Senior Data Platform Engineer (Python)
We are looking for a Senior Data Platform Engineer to design, build, and operate scalable data pipelines and lakehouse architectures across batch and streaming workloads.
You will work across Python, SQL, Airflow, Spark/PySpark, dbt, and modern AWS data services, with a strong focus on data quality, observability, and repeatable delivery.
Responsibilities
- Build and maintain scalable data pipelines in Python and SQL, orchestrated with Airflow, and delivered into S3-backed data lakes and Redshift/Athena warehouses.
- Design and implement lakehouse architectures using modern table formats such as Apache Iceberg and Delta Lake.
- Own data quality and reliability by implementing validation frameworks, data contracts, and observability practices including monitoring, alerting, SLIs, and SLOs.
- Optimize large-scale batch and streaming workloads with Spark/PySpark and dbt, balancing performance, cost, and maintainability across GBs to TBs of data.
- Drive infrastructure as code and CI/CD best practices for data workflows so deployments are repeatable, testable, and observable.
- Collaborate with analytics, product, and engineering stakeholders to turn raw data into dependable platform capabilities.
Requirements
- 5+ years of experience designing and building production ETL/ELT pipelines at scale.
- Hands-on expertise with Python, advanced SQL, and data modeling.
- Production experience with Airflow, data transformation, and Apache Spark/PySpark optimization.
- Practical AWS experience across S3 and data warehouses, plus CI/CD practices including Git, testing, and deployment automation.
- Experience building data quality frameworks, data contracts, and schema evolution strategies.
- Nice to have: modern table formats, medallion architecture, dbt, streaming platforms, and data observability tooling.
What will be your next steps?
Quick non-technical conversation
Our initial conversation is a brief, non-technical discussion to understand your background and career aspirations. We're keen to learn about your communication style and how you approach teamwork and decision-making.
60 to 90 minutes technical interview
This in-depth technical assessment, lasting 60 to 90 minutes, is designed to evaluate your specific skills and expertise. We will present you with challenges relevant to our client’s requirements.
Client interview
In this stage, you will meet directly with the client for a final technical discussion. This interview will be similar in format to our internal technical assessment, allowing the client to see firsthand how your expertise aligns with their specific project needs and team.
Offer
Congratulations on successfully completing our evaluation process. We are pleased to extend an offer and recommend you to our clients.
Apply for this role
Fill in your details below. We'll get back to you shortly.