7 of 7 Contract Apache Airflow Jobs in London

Data Engineer 6 Month Contract Location: London (Onsite 4 days a week) Rate: £700 - £800 per day (INSIDE IR35) Skills: Python, Azure, Kubernetes, Apache Airflow, Apache Spark, DV Clearance We are looking to recruit a DV Cleared Data Engineer for a leading technology consultancy. This will … Work across Azure environments on the low side Support data engineering workloads within Kubernetes environments on the high side Develop and maintain workflows using Apache Airflow Work with Apache Spark for large-scale data processing Support the ongoing development and improvement of established data pipelines Help ensure ...

running production pipelines end to end Excellent Python and SQL Solid grasp of data modelling, warehousing and lakehouse concepts Hands-on with orchestration (e.g. Airflow, Dagster) and distributed processing (e.g. Spark) Experience with cloud data platforms (AWS, Azure or GCP) and containerised deployment (Docker, Kubernetes) Good engineering practice: version ...

compliant solutions that align with GDPR and healthcare data governance standards Stack: Snowflake Snowflake Cortex AWS - S3, Glue, Lambda, Step Functions Python SQL dbt Airflow Terraform Docker GitHub CI/CD Requirements Must have advanced SQL and Python + experience delivering modern cloud-based data platforms Must have enterprise ...

compliant solutions that align with GDPR and healthcare data governance standards Stack: Snowflake | Snowflake Cortex | AWS - S3, Glue, Lambda, Step Functions | Python | SQL | dbt | Airflow| Terraform | Docker | GitHub CI/CD Requirements Must have advanced SQL and Python + experience delivering modern cloud-based data platforms Must have enterprise ...

Proven background as a Machine Learning Engineer Programming skills: Strong in SQL and Python (Pandas, Scikit-learn, Jupyter, Matplotlib) Data transformation & manipulation : experience with Airflow, DBT and Kubeflow Cloud: Experience with GCP and Vertex AI (developing ML services) Expertise: Solid understanding of computer science fundamentals and time-series forecasting ...

Docker and Kubernetes experience, including deployment and management Previous experience working within air-gapped environments Strong understanding of APIs and microservices Experience with Prefect, Airflow or similar data pipeline/orchestration technologies Experience configuring and integrating workflow tooling such as Jira Experience with database and storage technologies such ...

streaming pipelines and deliver scalable Databricks solutions using Delta Live Tables, Workflows, Unity Catalog, Python, PySpark, Terraform and Databricks Asset Bundles. Key requirements: Advanced Apache Spark and Spark SQL optimisation Production Spark Structured Streaming experience Deep Databricks platform knowledge Strong Python and PySpark engineering capability Delta Live Tables, Workflows … Databricks Asset Bundles CI/CD and GitHub Actions Ability to work autonomously in a fast-moving environment Experience with Databricks Apps, Lakebase, Airflow, Azure Data Factory, monitoring platforms or financial services would be beneficial. Location: London Rate: £800£900 per day IR35: Inside IR35 ...