201 to 225 of 236 Apache Airflow Jobs in England

options Hybrid working - 1 day a week in a central London office High-growth scale-up with a strong mission and serious funding Modern tech stack: Python, SQL, Snowflake, Apache Iceberg, AWS, Airflow, dbt, Spark Work cross-functionally with engineering, product, analytics, and data science leaders What You'll Be Doing Lead, mentor, and grow a high-impact More ❯

best practices around CI/CD, infrastructure-as-code, and modern data tooling Introduce and advocate for scalable, efficient data processes and platform enhancements Tech Environment: Python, SQL, Spark, Airflow, dbt, Snowflake, Postgres AWS (S3), Docker, Terraform Exposure to Apache Iceberg, streaming tools (Kafka, Kinesis), and ML pipelines is a bonus What We're Looking For: 5+ years More ❯

respond to suspicious activity. We own the end-to-end platform that powers our real-time and batch monitoring capabilities, including: A custom alerting and orchestration platform, built on Airflow, that enables scalable, auditable detection pipelines. Data pipelines in DBT and Snowflake that serve both ML models and rule-based logic. Backend services and APIs that handle case management … case management, and customer termination. Build and maintain robust, well-tested code with a focus on performance, reliability, and operational efficiency. Maintain and evolve the engineering infrastructure behind our Airflow-based alerting platform, enabling analysts to deploy and manage DAGs safely and effectively. Contribute to the development and maintenance of DBT models and data pipelines integrated with Snowflake to … software design principles. Proficiency in at least one of the following languages: Python, Golang, Java. Experience with multiple languages is a plus. Familiarity with data pipeline tooling such as Airflow and DBT, and cloud data warehouses like Snowflake. Understanding of testing strategies, including unit, integration, and system testing (TDD/BDD is a plus). Experience with CI/ More ❯

build, and lead the development of scalable data pipelines and platforms on AWS. The ideal candidate will have deep expertise in PySpark, Glue, Athena, AWS LakeFormation, data modelling, DBT, Airflow, Docker and will be responsible for driving best practices in data engineering, governance, and DevOps. Key Responsibilities: • Lead the design and implementation of scalable, secure, and high-performance data … field. • 10+ years of experience in data engineering. • Strong hands-on experience with AWS services: S3, Glue, Lake Formation, Athena, Redshift, Lambda, IAM, CloudWatch. • Proficiency in PySpark, Python, DBT, Airflow, Docker and SQL. • Deep understanding of data modeling techniques and best practices. • Experience with CI/CD tools and version control systems like Git. • Familiarity with data governance, security More ❯

build, and lead the development of scalable data pipelines and platforms on AWS. The ideal candidate will have deep expertise in PySpark, Glue, Athena, AWS LakeFormation, data modelling, DBT, Airflow, Docker and will be responsible for driving best practices in data engineering, governance, and DevOps. Key Responsibilities: • Lead the design and implementation of scalable, secure, and high-performance data … field. • 10+ years of experience in data engineering. • Strong hands-on experience with AWS services: S3, Glue, Lake Formation, Athena, Redshift, Lambda, IAM, CloudWatch. • Proficiency in PySpark, Python, DBT, Airflow, Docker and SQL. • Deep understanding of data modeling techniques and best practices. • Experience with CI/CD tools and version control systems like Git. • Familiarity with data governance, security More ❯

build, and lead the development of scalable data pipelines and platforms on AWS. The ideal candidate will have deep expertise in PySpark, Glue, Athena, AWS LakeFormation, data modelling, DBT, Airflow, Docker and will be responsible for driving best practices in data engineering, governance, and DevOps. Key Responsibilities: • Lead the design and implementation of scalable, secure, and high-performance data … field. • 10+ years of experience in data engineering. • Strong hands-on experience with AWS services: S3, Glue, Lake Formation, Athena, Redshift, Lambda, IAM, CloudWatch. • Proficiency in PySpark, Python, DBT, Airflow, Docker and SQL. • Deep understanding of data modeling techniques and best practices. • Experience with CI/CD tools and version control systems like Git. • Familiarity with data governance, security More ❯

build, and lead the development of scalable data pipelines and platforms on AWS. The ideal candidate will have deep expertise in PySpark, Glue, Athena, AWS LakeFormation, data modelling, DBT, Airflow, Docker and will be responsible for driving best practices in data engineering, governance, and DevOps. Key Responsibilities: • Lead the design and implementation of scalable, secure, and high-performance data … field. • 10+ years of experience in data engineering. • Strong hands-on experience with AWS services: S3, Glue, Lake Formation, Athena, Redshift, Lambda, IAM, CloudWatch. • Proficiency in PySpark, Python, DBT, Airflow, Docker and SQL. • Deep understanding of data modeling techniques and best practices. • Experience with CI/CD tools and version control systems like Git. • Familiarity with data governance, security More ❯

build, and lead the development of scalable data pipelines and platforms on AWS. The ideal candidate will have deep expertise in PySpark, Glue, Athena, AWS LakeFormation, data modelling, DBT, Airflow, Docker and will be responsible for driving best practices in data engineering, governance, and DevOps. Key Responsibilities: • Lead the design and implementation of scalable, secure, and high-performance data … field. • 10+ years of experience in data engineering. • Strong hands-on experience with AWS services: S3, Glue, Lake Formation, Athena, Redshift, Lambda, IAM, CloudWatch. • Proficiency in PySpark, Python, DBT, Airflow, Docker and SQL. • Deep understanding of data modeling techniques and best practices. • Experience with CI/CD tools and version control systems like Git. • Familiarity with data governance, security More ❯

including technical design, coding standards, code review, source control, build, test, deploy, and operations Awesome If You: Are experienced in Rust/Java/Kotlin Have experience with AWS, Apache Kafka, Kafka Streams, Apache Beam/Flink/Spark - especially deployment, monitoring & debugging Have experience with productisation of Machine Learning research projects Are familiar with Airflow or More ❯

through data science projects Awareness of data security best practices Experience in agile environments You would benefit from having: Understanding of data storage and processing design choices Familiarity with Apache Spark or Airflow Experience with parallel computing Candidates should be able to reliably commute or plan to relocate to Coventry before starting work. The role requires a Data More ❯

of distributed systems, databases,(PostgreSQL, Databricks, Clickhouse, ElasticSearch), and performance tuning. Familiarity with modern web frameworks and front-end technologies (React, Vue, Angular, etc.) Experience with data processing frameworks (Apache Spark, Kafka, Airflow, Dagster or similar) Experience with cloud platforms (AWS, GCP, Azure) and containerization technologies (Docker/Kubernetes) Experience with testing frameworks Strong analytical skills and a More ❯

dynamic deep-tech ML startup, unblocking a high standard of execution across the company. Architecting and building our core data infrastructure for managing large-scale ML training datasets (e.g., Apache Iceberg, Parquet). Develop cloud-based data processing pipelines , that ingest and compute auxiliary metadata signals on image, video and 3D data (PySpark, Airflow). Develop a data More ❯

using cloud-based architectures and tools Experience delivering data engineering solutions on cloud platforms, preferably Oracle OCI, AWS, or Azure Proficient in Python and workflow orchestration tools such as Airflow or Prefect Expert in data modeling, ETL, and SQL Experience with real-time analytics from telemetry and event-based streaming (e.g., Kafka) Experience managing operational data stores with high … availability, performance, and scalability Expertise in data lakes, lakehouses, Apache Iceberg, and data mesh architectures Proven ability to build, deliver, and support modern data platforms at scale Strong knowledge of data governance, data quality, and data cataloguing Experience with modern database technologies, including Iceberg, NoSQL, and vector databases Embraces innovation and works closely with scientists and partners to explore More ❯

we are relied upon to ensure our systems are trusted, reliable and available.The technology underpinning these capabilities includes industry leading data and analytics products such as Snowflake, Astronomer/Airflow, Kubernetes , DBT, Tableau, Sisense , Collibra, and Kafka/Debezium . Our mission is to enable frictionless experiences for our AIS colleagues and customers so that they can openly and … governance by identifying capability gaps, implementing necessary tooling and processes, and promoting DataOps through leadership and user feedback initiatives. Requirements: Deploy and govern modern data stack technologies (e.g., Snowflake, Airflow, DBT, Fivetran, Airbyte, Tableau, Sisense, AWS, GitHub, Terraform, Docker) at enterprise scale for data engineering workloads. Develop deployable, reusable ETL/ELT solutions using Python, advanced SQL, and Jinja More ❯

a petabyte-scale Data Lake and create secure, efficient, and scalable environments for our data platforms. Leveraging cloud-native technologies and AWS tools such as AWS S3, EKS, Glue, Airflow, Trino, and Parquet, you will prepare to adopt Apache Iceberg for greater performance and flexibility. You'll address high-performance data workloads, ensuring seamless execution of massive queries More ❯

My client is based in the London area, is currently looking to recruit an experienced Data Engineer to join their AI Analytics team. They are one of the leaders within the AI space. They are currently going through a period More ❯

My client is based in the London area, is currently looking to recruit an experienced Data Engineer to join their AI Analytics team. They are one of the leaders within the AI space. They are currently going through a period More ❯

Lead Data Engineer £75,000-£85,000 ️ AWS, Python, SQL, Airflow Brighton, hybrid working Analyse customer behaviour using AI & ML We are partnered with a private equity backed company who provide an AI-powered, guided selling platform that helps businesses improve online sales and customer experience. They are looking for a Lead Data Engineer to lead a small team … experience in a Senior Data Engineering role. Comfortable owning and delivering technical projects end-to-end. Strong in Python, SQL, and cloud platforms (AWS or comparable). Experience with Airflow, Snowflake, Docker (or similar). Familiarity with coaching and mentoring more junior engineers, leading 1-1s and check ins. Wider tech stack : AWS, Python, Airflow, Fivetran, Snowflake … Enhanced parental leave and pay If you are interested in finding out more, please apply or contact me directly! Lead Data Engineer £75,000-£85,000 ️ AWS, Python, SQL, Airflow Brighton, hybrid working Analyse customer behaviour using AI & ML Burns Sheehan Ltd will consider applications based only on skills and ability and will not discriminate on any grounds. More ❯

optimise existing data pipelines, reducing manual effort and improving reliability Lead on structuring data to align with performance, delivery, and leadership metrics Work with cloud-native tooling (AWS, dbt, Airflow) to build scalable data infrastructure Collaborate with digital, performance, and portfolio teams to drive cross-functional delivery Champion data quality, completeness, and usability at every stage of the data … and maintaining automated data pipelines Advanced proficiency in Python and SQL Experience with dbt and strong understanding of data modelling (e.g., star schema) Proficient with orchestration tools such as Airflow Comfortable working with AWS services (Glue, S3, etc.) or similar cloud platforms Experience with Git, GitHub Actions and CI/CD workflows Confident engaging stakeholders to shape technical solutions More ❯

pipelines to power next-gen data products in the commodities industry. Ensure data quality using the latest analytics and monitoring tools. Design and build robust pipelines with tools like Airflow and DBT. Create scalable infrastructure on Azure using technologies like Terraform. Write clean, high-quality, reusable code aligned with best practices. Drive innovation by bringing your own ideas-your … in a fast-paced startup or agile environment. Strong background in schema design and dimensional data modeling. Able to communicate data architecture clearly with internal stakeholders. Experience with Azure, Airflow, DBT, Kubernetes, GitHub. Bonus points for: open-source contributions, an active GitHub profile, and curiosity for the latest in tech. A natural problem-solver who loves making things work. More ❯

pipelines to power next-gen data products in the commodities industry. Ensure data quality using the latest analytics and monitoring tools. Design and build robust pipelines with tools like Airflow and DBT. Create scalable infrastructure on Azure using technologies like Terraform. Write clean, high-quality, reusable code aligned with best practices. Drive innovation by bringing your own ideas-your … in a fast-paced startup or agile environment. Strong background in schema design and dimensional data modeling. Able to communicate data architecture clearly with internal stakeholders. Experience with Azure, Airflow, DBT, Kubernetes, GitHub. Bonus points for: open-source contributions, an active GitHub profile, and curiosity for the latest in tech. A natural problem-solver who loves making things work. More ❯

Join our rapidly expanding team as a hands-on Cloud Data Analytics Platform Engineer and play a pivotal role in shaping the future of data at Citi. We're building a cutting-edge, multi-cloud data analytics platform that empowers More ❯

Role Title: Infrastructure/Platform Engineer - Apache Duration: 9 Months Location: Remote Rate: £ - Umbrella only Would you like to join a global leader in consulting, technology services and digital transformation? Our client is at the forefront of innovation to address the entire breadth of opportunities in the evolving world of cloud, digital and platforms. Role purpose/summary ? Refactor … prototype Spark jobs into production-quality components, ensuring scalability, test coverage, and integration readiness. ? Package Spark workloads for deployment via Docker/Kubernetes and integrate with orchestration systems (e.g., Airflow, custom schedulers). ? Work with platform engineers to embed Spark jobs into InfoSum's platform APIs and data pipelines. ? Troubleshoot job failures, memory and resource issues, and execution anomalies … for logs, metrics, and error handling to support monitoring and incident response. ? Align implementations with InfoSum's privacy, security, and compliance practices. Required Skills and Experience: ? Proven experience with Apache Spark (Scala, Java, or PySpark), including performance optimization and advanced tuning techniques. ? Strong troubleshooting skills in production Spark environments, including diagnosing memory usage, shuffles, skew, and executor behavior. ? Experience More ❯

Role Title: Infrastructure/Platform Engineer - Apache Duration: 9 Months Location: Remote Rate: £ - Umbrella only Would you like to join a global leader in consulting, technology services and digital transformation? Our client is at the forefront of innovation to address the entire breadth of opportunities in the evolving world of cloud, digital and platforms. Role purpose/summary ? Refactor … prototype Spark jobs into production-quality components, ensuring scalability, test coverage, and integration readiness. ? Package Spark workloads for deployment via Docker/Kubernetes and integrate with orchestration systems (e.g., Airflow, custom schedulers). ? Work with platform engineers to embed Spark jobs into InfoSum's platform APIs and data pipelines. ? Troubleshoot job failures, memory and resource issues, and execution anomalies … for logs, metrics, and error handling to support monitoring and incident response. ? Align implementations with InfoSum's privacy, security, and compliance practices. Required Skills and Experience: ? Proven experience with Apache Spark (Scala, Java, or PySpark), including performance optimization and advanced tuning techniques. ? Strong troubleshooting skills in production Spark environments, including diagnosing memory usage, shuffles, skew, and executor behavior. ? Experience More ❯

Data Engineer (Snowflake & Airflow) London/Edinburgh 6 Month Contract Via Umbrella Our UK leading banking client are looking for a Data Engineer to join their team on an initial 6-month contract. Key Skills: Snowflake Agile methodology and working within small multiskilled feature teams. Comfortable with GIT and version control. Knowledge of older coding standards and practices. Airflow More ❯