21 of 21 Apache Airflow Jobs in the North of England

lead quality assurance and testing activities within a large-scale cloud data engineering programme operating across a modern AWS-native technology ecosystem, including Apache Airflow, Amazon Athena, AWS Glue, S3, EMR, DynamoDB, and related cloud services. This role combines hands-on technical leadership with test strategy ownership . … reconciliation, and performance under load Oversee PySpark-based data validation and interrogation activities on AWS EMR Support testing and monitoring of workflows orchestrated through Apache Airflow Drive root cause analysis for production data issues and platform defects Risk Management & Reporting Identify, assess, and communicate quality risks across releases ...

software development team Hands‐on experience designing and developing data integration/ETL pipelines from diverse data sources and formats Hands‐on experience with Apache Airflow, dbt, and Python Strong database skills with Postgres SQL, DynamoDB, Snowflake, and Databricks Experience collaborating with Agile teams, product owners, and cross … highly scalable data engineering solutions that power Moody’s next‐generation digital content platform. Design, develop, and maintain scalable data pipelines using DataBricks, Snowflake, Apache Airflow, dbt (SQL), and Python within AWS Support platform optimization, infrastructure improvements, process control enhancements, and system upgrades Collaborate with Moody’s technical ...

multiple data sources Distributed computing for data intensive work such as Spark and Databricks Cloud platforms including AWS, Azure and GCP Orchestration tools like Airflow Modern DevOps practices including version control, CI/CD, and infrastructure as code using Terraform or Kubernetes Experience building scalable backend applications Understanding ...

writing automation scripts or custom AI agents using node or python. Data Engineering (30%): Experience deploying and maintaining data orchestration toolsets (e.g., dbt, Apache Airflow), ETL/ELT methodologies, real‐time streaming (e.g., AWS Kinesis, Apache Kafka), Vector databases, and RAG architectures. Observability & FinOps: Experience implementing modern ...

writing automation scripts or custom AI agents using node or python. Data Engineering (30%): Experience deploying and maintaining data orchestration toolsets (e.g., dbt, Apache Airflow), ETL/ELT methodologies, real‐time streaming (e.g., AWS Kinesis, Apache Kafka), Vector databases, and RAG architectures. Observability & FinOps: Experience implementing modern ...

implementing agentic guardrails. Data Engineering (30%): Practical experience implementing components of data pipelines, including real-time streaming tools (AWS Kinesis, Kafka), data orchestration (dbt, Airflow), and managing vector databases for RAG architectures. Observability & Cost Management: SRE/Platform experience with the practical application of real-time monitoring and cloud ...

implementing agentic guardrails. Data Engineering (30%): Practical experience implementing components of data pipelines, including real-time streaming tools (AWS Kinesis, Kafka), data orchestration (dbt, Airflow), and managing vector databases for RAG architectures. Observability & Cost Management: SRE/Platform experience with the practical application of real-time monitoring and cloud ...

implementing agentic guardrails. Data Engineering (30%): Practical experience implementing components of data pipelines, including real-time streaming tools (AWS Kinesis, Kafka), data orchestration (dbt, Airflow), and managing Vector Databases for RAG architectures. Observability & Cost Management: SRE/Platform experience with the practical application of real-time monitoring and cloud ...

Data Engineer to design, build, and maintain scalable data pipelines powering Moody’s next-generation digital content platform. You will work with DataBricks, Snowflake, Apache Airflow, dbt (SQL), and Python within AWS, driving platform optimizations and cross-functional collaboration. In this role, you’ll collaborate with product owners ...

concepts, experience with LLMs, prompt engineering, or integrating AI capabilities into applications• Data Technologies – Experience with Kafka, PySpark, Parquet and Avro data formats, Apache Airflow for workflow management, or Streamlit for building interactive data applications• Advanced Data Management – Strong SQL design, query optimization, and database architecture expertise• Observability ...

labeling workflowsFamiliarity with ML frameworks (e.g., PyTorch, TensorFlow)Experience with CI/CD tools and processesExperience with data pipelines and orchestration tools (e.g., Airflow, Metaflow)Familiarity with MLOps practices (model monitoring, drift, deployment checks)Experience with ML evaluation methods, metrics, and benchmarkingPassion for learning new technologies and improving existing ...

both Terraform and Terragrunt deploying data platforms (preferably Databricks Experience designing scalable data pipelines for machine learning workloads Experience implementing orchestration tools such as Apache Airflow or equivalent. Knowledge of data modelling, dimensional modelling and data warehousing principles Whether you're developing new products, building scalable platforms ...

Python Experience with AWS and modern data technologies Familiarity with version control systems such as Git Experience with orchestration tools such as Airflow Ability to troubleshoot and improve data platform performance Strong attention to detail and focus on data quality Excellent communication and collaboration skills Willingness to learn ...

demonstrable experience designing, deploying, and operating big data platforms at scale. You will have hands-on experience with OSS data ecosystems (e.g., Spark, Trino, Airflow, Apache Superset, and similar technologies). You will have knowledge of cloud platforms and traditional on-premise hosting; hands-on experience with Azure ...

building data pipelines Proficient in SQL, Python, AWS and other data management technologies Familiarity with version control systems like Git and orchestration tools like Airflow Ability to test and troubleshoot data platforms reliably. A track record of improving data reliability and quality Excellent communication skills and the ability ...

Data Engineer - FinTech - Newcastle (Tech stack: Data Engineer, SQL, Python, AWS, Git, Airflow, Data Pipelines, Data Platforms, Programmer, Developer, Architect, Data Engineer) Our client is a trailblazer in the FinTech space, known for delivering innovative technology solutions to global financial markets. They are expanding their engineering capability in Newcastle ...

stakeholder communication skills Desirable: Experience overseeing or supporting data ingestion pipelines, including APIs and event driven data sources Familiarity with orchestration tools such as Airflow and modern ELT architectures Experience governing or implementing data CI/CD pipelines (e.g. dbt tests, deployment pipelines, automated checks). We currently … design input, reviews, and mentoring Influence how analytics data is built, trusted, and operated at scale Work with a modern stack: AWS, Snowflake, dbt, Airflow, SQL, and Python Join us as we redefine travel experiences and create memories for millions of passengers. At Jet2.com and Jet2holidays , your potential ...

mentor teammates through code reviews and documentation. A bachelor’s degree or equivalent experience is required, with master’s a plus; AWS, Snowflake, Airflow, and Python experience preferred. #J-18808-Ljbffr ...

preferred Mastery of Python, SQL, and Spark – and strong opinions on how they should be used in production data pipelines Deep experience with dbt, Airflow, or equivalent orchestration – including how to build pipelines that are testable, observable, and operationally sound Strong understanding of data quality engineering – not just validation … leadership responsibility Primary cloud AWS – Glue, Lambda, Athena, S3, Redshift, RDS, Kinesis, MSK, Lake Formation, Step Functions, MWAA Streaming Languages Python, SQL, Spark dbt, Airflow/AWS MWAA, AWS Glue Platforms Databricks, Snowflake, Amazon Redshift Great Expectations, dbt tests, reconciliation frameworks, data observability tooling IaC/DevOps Terraform ...

preferred Mastery of Python, SQL, and Spark – and strong opinions on how they should be used in production data pipelines Deep experience with dbt, Airflow, or equivalent orchestration – including how to build pipelines that are testable, observable, and operationally sound Strong understanding of data quality engineering – not just validation … leadership responsibility Primary cloud AWS – Glue, Lambda, Athena, S3, Redshift, RDS, Kinesis, MSK, Lake Formation, Step Functions, MWAA Streaming Languages Python, SQL, Spark dbt, Airflow/AWS MWAA, AWS Glue Platforms Databricks, Snowflake, Amazon Redshift Great Expectations, dbt tests, reconciliation frameworks, data observability tooling IaC/DevOps Terraform ...

Months contract 2 Days a month onsite - Expenses to Newcastle depends on their hub. Start 3 from offer Experience required AWS Services Terraform Apache Spark Python Airflow Docker Nice to haves Gitlab Security Scanning Agents (eg Trivy, Trend Micro, Wiz etc) Jupyter Notebooks Integration with DWP Integration Team ...