1 to 25 of 196 Apache Airflow Jobs in England

governance, security, and regulatory compliance . Ability to work closely with technical and business teams to deliver impactful solutions. Nice-to-Haves: Orchestration tools (Airflow, dbt, Prefect) or streaming frameworks (Kafka, Kinesis). CI/CD experience and DevOps practices for data workflows. Exposure to data visualisation platforms (PowerBI ...

concepts and best practices Familiarity with CI/CD processes and DevOps practices Preferred Qualifications Snowflake certifications (SnowPro Core, Advanced) Experience with orchestration tools (Airflow, Prefect, etc.) Knowledge of data governance and compliance frameworks Experience with real-time streaming tools (Kafka, Kinesis) Experience supporting BI tools such as Tableau ...

reviews, pairing, and sharing best practices. What You'll Bring Technical Expertise: Proficiency in Python and SQL; experience with data processing frameworks such as Airflow, Spark, or TensorFlow. Data Engineering Fundamentals: Strong understanding of data architecture, data modelling, and scalable data solutions. You have hands-on experience deploying … running ML systems in production, including model serving and monitoring. Cloud Platforms: Familiarity with AWS or Azure, including services like Apache Airflow, Terraform, or SageMaker. Data Quality Management: Experience with data versioning and quality assurance practices. You enjoy working in fast-moving environments where you can have real ...

reviews, pairing, and sharing best practices. What You’ll Bring Technical Expertise: Proficiency in Python and SQL; experience with data processing frameworks such as Airflow, Spark, or TensorFlow. Data Engineering Fundamentals: Strong understanding of data architecture, data modelling, and scalable data solutions. You have hands-on experience deploying … running ML systems in production, including model serving and monitoring. Cloud Platforms: Familiarity with AWS or Azure, including services like Apache Airflow, Terraform, or SageMaker. Data Quality Management: Experience with data versioning and quality assurance practices. You enjoy working in fast-moving environments where you can have real ...

pipelines Proficiency with at least one cloud platform: Amazon Web Services (AWS) Microsoft Azure (Azure) Google Cloud Platform (GCP) Experience with orchestration tools (e.g., Apache Airflow) Experience with version control (e.g., Git) Solid understanding of data warehousing concepts and dimensional modeling To apply for this role please submit ...

/GCP beneficial) Strong data modelling knowledge Understanding of data governance and security best practices Ability to translate business needs into scalable solutions Desirable Airflow, dbt, Kafka, CI/CD, BI tools, governance platforms, financial services experience. Interested in building modern, enterprise-scale data platforms in a highly regulated ...

industry. Strong full-stack experience with Python , Java (Spring Boot) , React , SQL, and relational databases. Hands-on experience with modern data engineering tools (e.g. Airflow, Spark, Kafka, dbt, Snowflake or similar). Experience with cloud platforms , containerisation (Docker/Kubernetes) , and CI/CD pipelines . Solid understanding ...

provisioning automation. Nice to Have Skills: Experience with Power BI, DAX, and data modeling for optimized reporting. Knowledge of big data orchestration tools like Apache Airflow. Previous exposure to regulated environments such as finance, healthcare, or gaming. Experience in a lottery or similar domain is a plus. Preferred Education ...

Location: London, UK (3 Days/Week Onsite) Type: Fixed Term Contract (6 Months) Mandatory Skills Needed : SQL ( Advanced) , AWS Cloud (Glue,Ethena,EMR), Airflow, Kafka/Kenesis ,Python, Terraform DBT & Snowflake (Basic) AWS Data Engineer Design, develop, and maintain data pipelines for collecting, transforming, and loading data into ...

embedding models, and RAG architectures. Familiarity with document processing pipelines, chunking strategies, and semantic search patterns. Familiarity with data pipeline and orchestration tools (e.g., Airflow, dbt, Spark, Kafka, cloud-native ETL tools) and batch vs. streaming patterns. Understanding of data governance (catalog, lineage, security, RBAC, masking, compliance requirements like ...

embedding models, and RAG architectures. Familiarity with document processing pipelines, chunking strategies, and semantic search patterns. Familiarity with data pipeline and orchestration tools (e.g., Airflow, dbt, Spark, Kafka, cloud-native ETL tools) and batch vs. streaming patterns. Understanding of data governance (catalog, lineage, security, RBAC, masking, compliance requirements like ...

embedding models, and RAG architectures. Familiarity with document processing pipelines, chunking strategies, and semantic search patterns. Familiarity with data pipeline and orchestration tools (e.g., Airflow, dbt, Spark, Kafka, cloud-native ETL tools) and batch vs. streaming patterns. Understanding of data governance (catalog, lineage, security, RBAC, masking, compliance requirements like ...

embedding models, and RAG architectures. Familiarity with document processing pipelines, chunking strategies, and semantic search patterns. Familiarity with data pipeline and orchestration tools (e.g., Airflow, dbt, Spark, Kafka, cloud-native ETL tools) and batch vs. streaming patterns. Understanding of data governance (catalog, lineage, security, RBAC, masking, compliance requirements like ...

Skills & Experience Proven experience as a Data Engineer , BI Engineer , or similar role. Strong proficiency in SQL , Python , and modern data engineering frameworks (e.g., Airflow, dbt, Spark, etc.). Experience with cloud platforms such as AWS, Azure, or GCP. Solid understanding of data warehousing and ETL/ELT architecture. ...

Cassandra) Expertise in building and maintaining ETL pipelines and data workflows Familiarity with cloud data platforms (AWS, Azure, GCP) and data pipeline orchestration tools (Airflow, Prefect, etc.) Understanding of data modelling, schema design, and performance optimisation Experience with agile development methodologies, including Scrum and Kanban Familiarity with version control ...

cloud data technologies such as Redshift, Glue, S3, Lambda or IAM. Proficiency in SQL and Python for data processing and orchestration tools such as Airflow Experience with modern modelling approaches and tooling such as dbt. Practical experience with infrastructure as code and CI/CD practices BENEFITS The successful ...

cloud data technologies such as Redshift, Glue, S3, Lambda or IAM. Proficiency in SQL and Python for data processing and orchestration tools such as Airflow Experience with modern modelling approaches and tooling such as dbt. Practical experience with infrastructure as code and CI/CD practices BENEFITS The successful ...

days per week in the office. • A collaborative culture and clear opportunities for technical growth. • Exposure to modern tooling including AWS, Redshift, Airflow, Terraform, and dbt. • The chance to shape a data platform and have meaningful influence on engineering standards. How to Apply If you are interested in this ...

based data engineering, including Redshift, S3, IAM and related services Building and maintaining data pipelines using Python and SQL Orchestration tools such as Airflow Data modelling and warehouse architecture in modern cloud environments Git-based development and Infrastructure as Code (e.g. Terraform) Desirable experience includes exposure to regulated industries ...

Excellence Lead the evolution of the enterprise data landscape using modern platforms such as Databricks, Snowflake, Azure Synapse, and BigQuery. Oversee data engineering with Apache Airflow, dbt, and Prefect, ensuring performance, governance, and alignment with enterprise metadata standards (Collibra, Alation, Microsoft Purview). Drive adoption of Delta Lake ...

working in fast-paced environments and able to contribute quickly without extensive onboarding. Nice to Have Experience with modern data stack tools (e.g., dbt, Airflow, etc.). Understanding of CI/CD pipelines and data infrastructure automation. Familiarity with data governance, security, and best practices in a cloud environment. ...

/CD for data pipelines. Good to have skills Although not essential, the following skills are desired by the client: Kafka/Kinesis, Airflow, CodePipeline. BI tools (Power BI/Tableau). Docker/OpenShift; metadata driven pipelines. 3-8+ years Data Engineering experience. Cloud data engineering ...

with a strong focus on ETL/ELT. • Advanced SQL and Python development skills. • Hands-on experience with DBT, GIT, Terraform, Docker, IAM, and Airflow (Composer). • Proven experience working on cloud platforms – ideally GCP (BigQuery), but Azure or AWS also considered. • Strong understanding of Data Mesh, Test Driven ...

role Strong proficiency in SQL, Python and Spark (PySpark) Experience designing data models and implementing reliable pipeline patterns Knowledge of orchestration tools such as Airflow Experience with low-code or no-code integration tools Hands-on experience with cloud platforms (AWS preferred), including services for storage, transformation and warehousing ...

detailed technical design. Hands-on experience configuring multi-node Hadoop clusters, including resource management, security, and performance tuning. Strong Big Data engineering background using Apache Airflow, Spark, dbt, Kafka, and Hadoop ecosystem tools. Knowledge of RDBMS systems (PostgreSQL, SQL Server) and familiarity with NoSQL/distributed databases such … MongoDB. Proven delivery of streaming pipelines and real-time data processing solutions. Improved job efficiency and reduced runtimes through Apache Spark optimisation and development. Some experience with containerisation (Docker, Kubernetes) and CI/CD pipelines. Delivered streaming pipelines and real-time data processing solutions. Experience replacing legacy ETL tools ...