Senior Data Engineer – Azure Databricks & PySpark (80–100%)

For our client, a leading financial institution in Zurich, we are looking for an experienced Senior Data Engineer to join a team responsible for developing and enhancing a modern Data Platform for Analytics and AI.

• Design, develop, and maintain scalable data pipelines, data products, and backend services supporting analytics, reporting, real-time processing, and AI use cases.
• Build and optimize large-scale data processing solutions using Databricks, Apache Spark, PySpark, Spark SQL, and Delta Lake.
• Develop and maintain efficient ETL/ELT pipelines and data transformation processes.
• Design and implement data models for analytical and operational workloads.
• Write high-quality, secure, maintainable, and well-tested code using Python, Scala, and SQL.
• Implement and enhance data quality, governance, and monitoring solutions.
• Contribute to CI/CD processes, automated deployments, and modern engineering practices.
• Collaborate closely with cloud, security, architecture, and business teams.

In this role, you will design and build scalable data pipelines, data products, and backend services using Azure Databricks, Apache Spark, PySpark, Python, and SQL. You will contribute to a modern cloud-based data architecture and help deliver reliable, high-performance data solutions.

Framework;

• Location: Zurich, Switzerland
• Start Date: ASAP (flexible for the right candidate)
• Contract Duration: 12 months, with possible extension
• Work Model: Initially 100% onsite in Zurich during onboarding, followed by a hybrid setup (60% onsite / 40% remote within Switzerland)

• 5+ years of professional experience in Data Engineering, including building enterprise-scale data platforms and ETL/ELT solutions.
• Strong hands-on experience with Apache Spark, PySpark, and/or Spark Streaming (must be clearly demonstrated in your CV).
• Proven experience working with Azure Databricks and large-scale data processing.
• Excellent programming skills in Python and SQL, including SQL performance optimization and troubleshooting.
• Experience with Spark SQL, Structured Streaming, and Delta Lake.
• Good understanding of data modelling, data warehousing, and modern data platform architectures.
• Experience with cloud-based data solutions, preferably Microsoft Azure.
• Familiarity with CI/CD pipelines, automated testing, and deployment practices.
• Strong analytical and problem-solving skills, with the ability to work independently and collaboratively.
• Excellent communication skills and experience working in Agile environments.

Nice to have:

• Experience in financial services, particularly Structured Products, Trading, or Capital Markets.
• Experience with Terraform and Infrastructure as Code (IaC).
• Knowledge of streaming technologies such as Kafka, Azure Event Hubs, Event Grid, or Apache Flink.
• Experience with dbt or Data Vault 2.0.
• Experience providing technical guidance and contributing to architectural decisions

• A challenging 12-month contract with the possibility of extension.
• The opportunity to work on a modern Azure Databricks Data Platform within a leading financial institution.
• Exposure to innovative Data Analytics and AI initiatives.
• A collaborative environment with experienced engineering, cloud, and architecture professionals.
• A hybrid working model following the initial onboarding period.
• A straightforward recruitment process consisting of two interview rounds.