21 of 21 Contract Apache Airflow Jobs in the UK

Data Engineer 6 Month Contract Location: London (Onsite 4 days a week) Rate: £700 - £800 per day (INSIDE IR35) Skills: Python, Azure, Kubernetes, Apache Airflow, Apache Spark, DV Clearance We are looking to recruit a DV Cleared Data Engineer for a leading technology consultancy. This will … Work across Azure environments on the low side Support data engineering workloads within Kubernetes environments on the high side Develop and maintain workflows using Apache Airflow Work with Apache Spark for large-scale data processing Support the ongoing development and improvement of established data pipelines Help ensure ...

owned infrastructure. You'll work as part of a technical team building and supporting data pipelines using Python, Azure and Kubernetes, alongside technologies including Apache Airflow and Apache Spark. Key Responsibilities: Build and maintain new and existing data pipelines Develop data solutions using Python, Azure and Kubernetes … Work with Apache Airflow and Apache Spark Support the ongoing development and improvement of the data platform Ensure data remains securely hosted on MoD-owned infrastructure Collaborate with wider technical and delivery teams Troubleshoot and resolve technical issues across existing data pipelines Required Experience: Strong commercial experience ...

working technical solutions. Key Responsibilities: Build and maintain scalable applications using Python, SQL and PySpark Develop and support large-scale data processing pipelines using Apache Spark Use Apache Airflow for workflow orchestration and task scheduling Work with AWS EMR environments and review logs/output to troubleshoot … with stakeholders to translate customer requirements into technical functionality Follow engineering best practices across delivery team Technical Environment: Programming/Data: Python SQL PySpark Apache Spark Apache Airflow AWS/Cloud: AWS EMR S3 Glue CloudWatch IAM EC2 ECR DynamoDB Lake Formation Amazon Athena AI/Intelligent ...

Data Engineer 6 Month Contract Location: London (Onsite 4 days a week) Rate: £700 - £800 per day (INSIDE IR35) Skills: Python, Azure, Kubernetes, Apache Airflow, Apache Spark, DV Clearance We are looking to recruit a DV Cleared Data Engineer for a leading technology consultancy click apply ...

Data Engineer in secure government, defence, or intelligence settings. Strong Python and SQL development skills. Hands-on experience with data integration tools like Airflow, NiFi, Azure Data Factory, or AWS Glue. Expertise in Elasticsearch management, indexing, and query optimisation. Knowledge of data modelling, warehousing, and modern data platform architectures. ...

Google Cloud. Key skills: Strong Python & SQL experience Hands-on Azure Databricks, Delta Lake & Unity Catalog Strong GCP/BigQuery/GCS experience Apache Airflow/Cloud Composer for pipeline orchestration Experience building ELT/data pipelines and cross-cloud data transfers Understanding of IAM, RBAC, secrets management ...

with CI/CD and automated deployment practices. Experience supporting and maintaining production data platforms. Desirable Skills Scikit-Learn or Spark MLlib AWS EMR Apache Airflow Jenkins Hudi or other open table formats Kibana Grafana Understanding of Data Science principles If you receive suspicious outreach claiming ...

Optimise Python performance through effective use of loops , vectorization , threading , and understanding of the Global Interpreter Lock (GIL) . Desirable Experience Experience with Kubeflow , Airflow , or dbt . Hands-on experience with Google Cloud Platform (GCP) . Knowledge of containerisation (Docker, Kubernetes) and CI/CD practices. Familiarity with ...

code Requirements: - Must have strong, commercial AWS (contract) Data Engineering experience - Must bring solid knowledge of data warehousing and modelling - Experience with Airflow and infrastructure as a code (Terraform, CloudFormation) would be beneficial - Must have experience with Redshift Nice to have: - Experience with dbt/lake formation ...

preferred) or similar platforms: Snowflake, BigQuery, Redshift, Synapse, or equivalent Strong proficiency with: SQL (advanced level) Spark/PySpark Data pipeline orchestration tools (e.g., Airflow, ADF, etc.) Data Modelling Extensive experience designing: Fact and dimension tables Star and snowflake schemas Aggregation layers and semantic models Data Migration & Mapping Demonstrated ...

level proficiency in Python (3.x) for data engineering, with strong experience in Scripting, pandas, requests, SQLAlchemy, custom libraries, and building reusable, production-grade code. Apache Airflow Deep understanding of core concepts, including DAGs, tasks, operators, sensors, executors, hooks, and connections. Hands-on experience with Airflow architecture, scheduling … GitHub Actions for CI/CD, and collaborative development. Google Cloud Platform (GCP) Solid hands-on experience with GCP services, particularly Cloud Composer (managed Airflow), BigQuery (advanced SQL and Scripting), Cloud Storage, Dataflow, Pub/Sub, IAM, VPC, and Cloud Monitoring/Logging for pipeline observability. ...

enhancement programme. Investment Data Engineer, key skills: Strong commercial experience in Data Engineering, particularly using Snowflake Proven experience developing and maintaining workflows using Apache Airflow and DBT (Data Build Tool) Experience integrating and supporting Bloomberg data feeds, services or associated investment data solutions Strong SQL development and data … with minimal ramp-up time Strong analytical and problem-solving skills Excellent stakeholder communication and collaboration abilities Investment Data Engineer, Bloomberg, SQL, Python, DBT, Airflow Reasonable Adjustments: Respect and equality are core values to us. We are proud of the diverse and inclusive community we have built ...

background as a Machine Learning Engineer. Technical Skills: Strong in SQL and Python (Pandas, Scikit-learn, Jupyter, Matplotlib). Data transformation & manipulation : experience with Airflow, DBT and Kubeflow Cloud: Experience with GCP and Vertex AI (developing ML services). Expertise: Solid understanding of computer science fundamentals and time-series ...

management * Software testing lifecycle/defect management * CI/CD & DevOps practices Desirable: HMRC/public sector experience, AWS/Azure/SAS PaaS, Airflow or SAS Migration Factory experience. If you are interested in this role or wish to apply, please submit your CV. ...

technologies Awareness of Site Reliability Engineering principles Experience restoring services during outage situations Desirable skills and experience Docker and containerisation principles Jenkins Kibana Grafana Airflow Desired behaviours Proactive and self-motivated User-centric mindset Evidence-driven decision making Strong collaboration and communication skills Comfortable with pairing, demonstrations and knowledge ...

Scala Play or other MVC/Rest API frameworks SQL AWS Suite Continuous Integration Agile methodologies Desirable: Containerisation principles/Docker Jenkins Kibana Grafana Airflow ...

Essential: ScalaPlay or other MVC/Rest API frameworks SQL AWS Suite Continuous Integration Agile methodologies Desirable: Containerisation principles/Docker Jenkins Kibana Grafana Airflow If you are interested in this position and would like to learn more, please send through your CV and we will get in touch ...

Scala Play or other MVC/Rest API frameworks SQL AWS Suite Continuous Integration Agile methodologies Desirable: Containerisation principles/Docker Jenkins Kibana Grafana Airflow Please send CV for full details and immediate interviews. We are a preferred supplier to the client. ...

Scala Play or other MVC/Rest API frameworks SQL AWS Suite Continuous Integration Agile methodologies Desirable: Containerisation principles/Docker Jenkins Kibana Grafana Airflow If you receive suspicious outreach claiming to be from us, please contact us via the ManpowerGroup website. ...

Scala Play or other MVC/Rest API frameworks SQL AWS Suite Continuous Integration Agile methodologies Desirable: Containerisation principles/Docker Jenkins Kibana Grafana Airflow If you receive suspicious outreach claiming to be from us, please contact us via the ManpowerGroup website. ...

requiring strong data engineering expertise, experience working with financial data, and the ability to operate within a fast-paced investment environment. Key Skills Snowflake Airflow DBT Bloomberg Integrations Responsibilities Design, build and maintain scalable data pipelines and workflows. Develop and enhance data solutions using Snowflake, Airflow and DBT. … Drive best practices across data engineering, automation and platform support. Required Experience Experience within Investment Management, Asset Management or Financial Services Expertise with Snowflake, Airflow and DBT. Experience integrating Bloomberg data and services into enterprise platforms. Strong SQL and data modelling skills. Experience working with large-scale financial ...