18 of 18 Contract Apache Airflow Jobs

Data Engineer 6 Month Contract Location: London (Onsite 4 days a week) Rate: £700 - £800 per day (INSIDE IR35) Skills: Python, Azure, Kubernetes, Apache Airflow, Apache Spark, DV Clearance We are looking to recruit a DV Cleared Data Engineer for a leading technology consultancy. This will … Work across Azure environments on the low side Support data engineering workloads within Kubernetes environments on the high side Develop and maintain workflows using Apache Airflow Work with Apache Spark for large-scale data processing Support the ongoing development and improvement of established data pipelines Help ensure ...

Data Engineer 6 Month Contract Location: Hereford (Onsite 4 days a week) Rate: £700 - £800 per day (INSIDE IR35) Skills: Python, Azure, Kubernetes, Apache Airflow, Apache Spark, DV Clearance We are looking to recruit a DV Cleared Data Engineer for a leading technology consultancy. This will … Work across Azure environments on the low side Support data engineering workloads within Kubernetes environments on the high side Develop and maintain workflows using Apache Airflow Work with Apache Spark for large-scale data processing Support the ongoing development and improvement of established data pipelines Help ensure ...

running production pipelines end to end Excellent Python and SQL Solid grasp of data modelling, warehousing and lakehouse concepts Hands-on with orchestration (e.g. Airflow, Dagster) and distributed processing (e.g. Spark) Experience with cloud data platforms (AWS, Azure or GCP) and containerised deployment (Docker, Kubernetes) Good engineering practice: version ...

compliant solutions that align with GDPR and healthcare data governance standards Stack: Snowflake Snowflake Cortex AWS - S3, Glue, Lambda, Step Functions Python SQL dbt Airflow Terraform Docker GitHub CI/CD Requirements Must have advanced SQL and Python + experience delivering modern cloud-based data platforms Must have enterprise ...

compliant solutions that align with GDPR and healthcare data governance standards Stack: Snowflake | Snowflake Cortex | AWS - S3, Glue, Lambda, Step Functions | Python | SQL | dbt | Airflow| Terraform | Docker | GitHub CI/CD Requirements Must have advanced SQL and Python + experience delivering modern cloud-based data platforms Must have enterprise ...

Experience with cloud data platforms and warehouses, for example Snowflake, Databricks, BigQuery or Redshift. Exposure to modern data engineering tooling and practices, including dbt, Airflow, Git-based version control and CI/CD for data. Experience with IT cost management strategies, models and tooling, including chargeback and cost recovery ...

graph database tools. Essential -Python -Open-source data engineering technologies -Data pipeline development -Linux or Unix -Continuous integration and continuous delivery -Containerisation -Kubernetes Preferred -Apache Airflow -OpenSearch -Azure -Azure Stack Hub -Apache Kafka -Data cataloguing tools and practices -Apache NiFi -Graph databases Due to the nature ...

experience as a Data Engineer or Senior Data Engineer Expert-level Python development skills Extensive hands-on experience with Databricks in enterprise environments Strong Apache Spark and PySpark expertise Advanced SQL development and optimisation skills Experience designing and building large-scale ETL and data integration solutions Strong understanding … environments Exposure to risk, trading, treasury or regulatory data domains Experience with Kafka, streaming technologies or real-time data processing Databricks certifications Experience with Airflow, Databricks Workflows or similar orchestration tools CI/CD pipeline development and DevOps practices Docker and Kubernetes Experience supporting data science or machine learning ...

need both skillsets below - strength in one is enough: Hands-on: building AI/ML pipelines or prototypes; vector databases; prompt engineering; orchestration tools (Airflow/Prefect); Git Advisory: designing AI/data architectures; Agentic AI principles (LangGraph, CrewAI, AutoGen); mentoring specialists without formal authority; AI governance (security, privacy ...

Engineering Strong experience working with AWS in data engineering environments. Hands-on experience with AWS Glue for data integration and processing. Strong experience using Apache Airflow for workflow orchestration and pipeline management. Advanced Python development skills. Experience with Apache Spark or similar distributed data processing technologies would ...

live ML models. BAU Operator: High autonomy to own daily operations and deliver under tight deadlines. Cloud & BI Stack: Practical exposure to GCP (BigQuery), Airflow, or Looker . AI Tooling: Familiarity with Claude Code, OpenAI, Cursor , or agentic workflows to accelerate delivery. Compliance & Domain: Prior experience with FCA reporting ...

have skills: RabbitMQ Java/Microservices MongoDB/MySQL Kubernetes Grafana/Prometheus Gitlab Job Scheduling tooling (eg Berlin/JS7 Job Scheduler, TWS, Apache Airflow). Reporting and visualisation tooling (from Pentaho BA, Power BI, SAP Business Objects). Nice to have: Experience with databases/data ...

have skills: RabbitMQ Java/Microservices MongoDB/MySQL Kubernetes Grafana/Prometheus Gitlab Job Scheduling tooling (e.g. Berlin/JS7 Job Scheduler, TWS, Apache Airflow). Reporting and visualisation tooling (from Pentaho BA, Power BI, SAP Business Objects). Nice to have: Experience with databases/data ...

have skills: RabbitMQ Java/Microservices MongoDB/MySQL Kubernetes Grafana/Prometheus Gitlab Job Scheduling tooling (eg Berlin/JS7 Job Scheduler, TWS, Apache Airflow). Reporting and visualisation tooling (from Pentaho BA, Power BI, SAP Business Objects). Nice to have: Experience with databases/data ...

Proven background as a Machine Learning Engineer Programming skills: Strong in SQL and Python (Pandas, Scikit-learn, Jupyter, Matplotlib) Data transformation & manipulation : experience with Airflow, DBT and Kubeflow Cloud: Experience with GCP and Vertex AI (developing ML services) Expertise: Solid understanding of computer science fundamentals and time-series forecasting ...

support/SRE awareness * SC Security Clearance It would be an added advantage if you had any of the following: Jenkins/Kibana/Airflow/Docker/Containerisation/Grafana. Apply now for immediate interview! CBSbutler is operating and advertising as an Employment Agency for permanent positions ...

data infrastructure, while contributing to strong engineering practices and standards. The environment is heavily focused on Google Cloud Platform (GCP), with BigQuery, Dataform and Airflow/Cloud Composer forming key parts of the technology stack. WHAT YOU'LL BE DOING * Designing and implementing complex data flows across operational, analytics … clearly defined. ESSENTIAL EXPERIENCE * Strong hands-on experience with Google Cloud Platform (GCP). * Strong experience with BigQuery. * Experience using Dataform. * Experience with Airflow/Cloud Composer. * Strong data engineering fundamentals, including building scalable and robust production data solutions. * Experience working with data quality, governance and appropriate data controls. ...

streaming pipelines and deliver scalable Databricks solutions using Delta Live Tables, Workflows, Unity Catalog, Python, PySpark, Terraform and Databricks Asset Bundles. Key requirements: Advanced Apache Spark and Spark SQL optimisation Production Spark Structured Streaming experience Deep Databricks platform knowledge Strong Python and PySpark engineering capability Delta Live Tables, Workflows … Databricks Asset Bundles CI/CD and GitHub Actions Ability to work autonomously in a fast-moving environment Experience with Databricks Apps, Lakebase, Airflow, Azure Data Factory, monitoring platforms or financial services would be beneficial. Location: London Rate: £800£900 per day IR35: Inside IR35 ...