1 to 25 of 110 Apache Airflow Jobs in England

data platform and engineering capability as the business continues to scale its trading operations. They are looking for a Data Engineer with strong Apache Airflow experience to help design and build the pipelines that power trading, analytics and commercial decision-making. This is a great opportunity to join … platform and the opportunity to work with modern open-source technologies . What You’ll Be Doing Designing and building data pipelines orchestrated with Apache Airflow Developing ETL/ELT workflows in Python and SQL to process trading and operational data Building reliable, scalable data infrastructure to support ...

technical delivery of data platforms and pipelines built in AWS-with a focus on AWS Glue, Managed Workflows for Apache Airflow (MWAA), and Python-and collaborate closely with Directors, Senior Architects, and Program Leadership to deliver business outcomes at scale. This is a player-coach role: you will … Your Responsibilities: Lead the design and implementation of scalable, secure, and cost-efficient ETL/ELT pipelines using AWS Glue, Python (PySpark), and MWAA (Airflow). Define solution architectures, data models, orchestration patterns, and CI/CD for data workflows. Own the technical roadmap, decomposition, and delivery plan-including ...

technical delivery of data platforms and pipelines built in AWS-with a focus on AWS Glue, Managed Workflows for Apache Airflow (MWAA), and Python-and collaborate closely with Directors, Senior Architects, and Program Leadership to deliver business outcomes at scale. This is a player-coach role: you will … Your Responsibilities: Lead the design and implementation of scalable, secure, and cost-efficient ETL/ELT pipelines using AWS Glue, Python (PySpark), and MWAA (Airflow). Define solution architectures, data models, orchestration patterns, and CI/CD for data workflows. Own the technical roadmap, decomposition, and delivery plan-including ...

Platform DevOps team to build and run highly reliable CI/CD pipelines and infrastructure for the bank’s modern data estate (AWS + Airflow + Databricks). Key Responsibilities Design, build and maintain Infrastructure as Code using CloudFormation and AWS services Manage and scale Apache Airflow … call rotation and incident resolution Essential Skills & Experience 5+ years as a DevOps/Platform Engineer Strong AWS experience + CloudFormation Production experience with Apache Airflow (MWAA) Lambda , Linux + Bash scripting CI/CD expertise (Git, Jenkins, GitHub Actions) IaC and GitOps practices Nice to Have Terraform ...

direction within your team Required Skills and Experience Strong analytic skills related to working with unstructured datasets Python (PySpark, Pandas, PyArrow) Distributed data processing (Apache Spark) Data ETL (Apache Airflow, AWS Step Functions, Apache NiFi) Cloud services (AWS, Azure or GCP) Messaging/Streaming (Kafka ...

governance, security, and regulatory compliance . Ability to work closely with technical and business teams to deliver impactful solutions. Nice-to-Haves: Orchestration tools (Airflow, dbt, Prefect) or streaming frameworks (Kafka, Kinesis). CI/CD experience and DevOps practices for data workflows. Exposure to data visualisation platforms (PowerBI ...

Location: London, UK (3 Days/Week Onsite) Type: Fixed Term Contract (6 Months) Mandatory Skills Needed : SQL ( Advanced) , AWS Cloud (Glue,Ethena,EMR), Airflow, Kafka/Kenesis ,Python, Terraform DBT & Snowflake (Basic) AWS Data Engineer Design, develop, and maintain data pipelines for collecting, transforming, and loading data into ...

driving innovation in data engineering. Desirable Criteria Experience delivering data pipelines on Hortonworks/Cloudera on-prem and leading cloud migration initiatives. Familiarity with: - Apache Airflow - Data modelling and metadata management Experience influencing enterprise data strategy and contributing to architectural governance. To apply for this position please submit ...

driving innovation in data engineering. Desirable Criteria Experience delivering data pipelines on Hortonworks/Cloudera on-prem and leading cloud migration initiatives. Familiarity with: Apache Airflow Data modelling and metadata management Experience influencing enterprise data strategy and contributing to architectural governance. To apply for this position please submit ...

driving innovation in data engineering. Desirable Criteria Experience delivering data pipelines on Hortonworks/Cloudera on-prem and leading cloud migration initiatives. Familiarity with: Apache Airflow Data modelling and metadata management Experience influencing enterprise data strategy and contributing to architectural governance. ...

/GCP beneficial) Strong data modelling knowledge Understanding of data governance and security best practices Ability to translate business needs into scalable solutions Desirable Airflow, dbt, Kafka, CI/CD, BI tools, governance platforms, financial services experience. Interested in building modern, enterprise-scale data platforms in a highly regulated ...

cloud data technologies such as Redshift, Glue, S3, Lambda or IAM. Proficiency in SQL and Python for data processing and orchestration tools such as Airflow Experience with modern modelling approaches and tooling such as dbt. Practical experience with infrastructure as code and CI/CD practices BENEFITS The successful ...

cloud data technologies such as Redshift, Glue, S3, Lambda or IAM. Proficiency in SQL and Python for data processing and orchestration tools such as Airflow Experience with modern modelling approaches and tooling such as dbt. Practical experience with infrastructure as code and CI/CD practices BENEFITS The successful ...

cloud data technologies such as Redshift, Glue, S3, Lambda or IAM. Proficiency in SQL and Python for data processing and orchestration tools such as Airflow Experience with modern modelling approaches and tooling such as dbt. Practical experience with infrastructure as code and CI/CD practices BENEFITS The successful ...

days per week in the office. • A collaborative culture and clear opportunities for technical growth. • Exposure to modern tooling including AWS, Redshift, Airflow, Terraform, and dbt. • The chance to shape a data platform and have meaningful influence on engineering standards. How to Apply If you are interested in this ...

data models and map various data sources to support downstream analytics. Pipeline Engineering: Build scalable ETL pipelines and data quality solutions using tools like Airflow, Luigi, or Azkaban. Codification: Translate intricate business logic into high-quality code using Python, Scala, or Java. Innovation: Evaluate and implement new technologies across ...

with dimensional modelling, star schemas, SCDs etc.Experience within regulated financial servicesDesirableExperience in pensions, asset management or institutional investingUnderstanding of performance measurement and attributionExposure to Airflow or dbtCloud platform familiarity (Azure or AWS)Knowledge of data governance best practiceAbout YouHighly detail-driven with a strong approach to data qualityComfortable operating ...

provisioning automation. Nice to Have Skills: Experience with Power BI, DAX, and data modeling for optimized reporting. Knowledge of big data orchestration tools like Apache Airflow. Previous exposure to regulated environments such as finance, healthcare, or gaming. Experience in a lottery or similar domain is a plus. Preferred Education ...

architecture to support data consumption for reporting systems, operational applications, and analytics platforms. Implement automation, orchestration, and scalable pipeline frameworks using tools such as Apache Airflow. Collaborate with business stakeholders and technical teams to align the data platform with enterprise data strategy. Support hospitality analytics use cases, including guest … predictive analytics, recommendation engines, and customer behaviour analytics. Ability to design AI-ready data platforms for future analytics use cases. Preferred Skills Experience with Apache Airflow for pipeline orchestration. Knowledge of CI/CD pipelines, DevOps, and Git-based development workflows. Experience with data governance, metadata management ...

modern cloud data warehouse (e.g., Snowflake, BigQuery, Redshift, or Azure Synapse). Experience with transformation tooling (e.g., dbt or equivalent) and orchestration (e.g., Airflow, Azure Data Factory, or similar). Proficiency with Git-based version control and CI/CD principles. Understanding of data governance, lineage, and metric definitions. ...

role Strong proficiency in SQL, Python and Spark (PySpark) Experience designing data models and implementing reliable pipeline patterns Knowledge of orchestration tools such as Airflow Experience with low-code or no-code integration tools Hands-on experience with cloud platforms (AWS preferred), including services for storage, transformation and warehousing ...

Excellence Lead the evolution of the enterprise data landscape using modern platforms such as Databricks, Snowflake, Azure Synapse, and BigQuery. Oversee data engineering with Apache Airflow, dbt, and Prefect, ensuring performance, governance, and alignment with enterprise metadata standards (Collibra, Alation, Microsoft Purview). Drive adoption of Delta Lake ...

working in fast-paced environments and able to contribute quickly without extensive onboarding. Nice to Have Experience with modern data stack tools (e.g., dbt, Airflow, etc.). Understanding of CI/CD pipelines and data infrastructure automation. Familiarity with data governance, security, and best practices in a cloud environment. ...

Skills and Experience * Strong commercial experience working with AWS in production environments. * Proficiency in Python and SQL for data processing and orchestration. * Experience with Airflow for workflow management. * Knowledge of cloud data warehousing, ideally Redshift. * Experience with infrastructure as code and CI/CD tooling such as Terraform ...

/CD for data pipelines. Good to have skills Although not essential, the following skills are desired by the client: Kafka/Kinesis, Airflow, CodePipeline. BI tools (Power BI/Tableau). Docker/OpenShift; metadata driven pipelines. 3-8+ years Data Engineering experience. Cloud data engineering ...