1 to 25 of 96 Apache Airflow Jobs in the North of England

Senior Data Engineer/Data Analytics Engineer - Full Remote Working from anywhere in the UK SQL+Python+ETL+ Apache Airflow Our client is a global leading and fast growing high tech company: -Over 6,500 employees across 20+ offices; -300 Million+ active users on some of the platforms they developed … ensure data quality, and support business decision-making with high-quality datasets. -Work across technology stack: SQL, Python, ETL, Big Query, Spark, Hadoop, Git, Apache Airflow, Data Architecture, Data Warehousing -Design and develop scalable ETL pipelines to automate data processes and optimize delivery -Implement and manage data warehousing … solutions, ensuring data integrity through rigorous testing and validation -Lead, plan and execute workflow migration and data orchestration using Apache Airflow -Focus on data engineering and data analytics Requirements: -5+ years of experience in SQL -5+ years of development in Python -MUST have strong experience in More ❯

performance and responsiveness. Stay Up to Date with Technology: Keep yourself and the team updated on the latest Python technologies, frameworks, and tools like Apache Spark , Databricks , Apache Pulsar , Apache Airflow , Temporal , and Apache Flink , sharing knowledge and suggesting improvements. Documentation: Contribute to clear and … or Azure . DevOps Tools: Familiarity with containerization ( Docker ) and infrastructure automation tools like Terraform or Ansible . Real-time Data Streaming: Experience with Apache Pulsar or similar systems for real-time messaging and stream processing is a plus. Data Engineering: Experience with Apache Spark , Databricks , or similar … big data platforms for processing large datasets, building data pipelines, and machine learning workflows. Workflow Orchestration: Familiarity with tools like Apache Airflow or Temporal for managing workflows and scheduling jobs in distributed systems. Stream Processing: Experience with Apache Flink or other stream processing frameworks is a plus. More ❯

capabilities and opening new possibilities using machine learning and artificial intelligence. This approach encompasses a multitude of vendors (Microsoft, AWS, ESRI, Snowflake, IBM and Apache Software Foundation) and services (Azure, Microsoft Fabric, Apache Airflow, dbt Cloud, watsonx, Kafka, Kubernetes (EKS and AKS), Red Hat Open Shift, plus … good knowledge of Java, Web services, JavaScript, Json, XML, JDBC, and other related technologies. Experience in data and orchestration tools including some of dbt, Apache Airflow, Azure Data Factory. Experience with unit testing in a variety of languages and tools. Perhaps experience of Junit, NUnit, XUnit and Jest More ❯

configuration in YAML. Develop shell scripts and automation for deployment and operational workflows. Work with Data Engineering to integrate and manage data workflows using Apache Airflow and DAG-based models. Perform comprehensive testing, debugging, and optimization of backend components. Required Skills Bachelor's degree in Computer Science, Software … configurations and managing releases. Proficiency in shell scripting for automating deployment and maintenance tasks. Understanding of DAG (Directed Acyclic Graph) models and experience with Apache Airflow for managing complex data processing workflows. Familiarity with database systems (SQL and NoSQL) and proficiency in writing efficient queries. Solid understanding of More ❯

Databrew. They are experienced in developing batch and real-time data pipelines for Data Warehouse and Datalake, utilizing AWS Kinesis and Managed Streaming for Apache Kafka. They are also proficient in using open source technologies like Apache Airflow and dbt, Spark/Python or Spark/Scala More ❯

of SQL and experience in testing databases and data warehouses with dBt (e.g., Snowflake - Preferred, Redshift, BigQuery) Strong Knowledge of workload automation platforms like Apache Airflow and dBt (Data Build Tool) Familiarity with CI/CD tools (e.g. Azure DevOps – Preferred, Jenkins) and experience integrating automated tests into … and frameworks. Proficiency with automation testing frameworks (Cucumber, Gherkin, TestNG, or similar) for data testing workloads. Knowledge of performance testing and load testing tools (Apache JMeter or Gatling) Experience: Proven track record in supporting and improving test processes in data-related projects. Experience in leadership, mentoring, or training roles More ❯

object-oriented design principles, and data structures Extensive experience in developing microservices using Java, Python Experience in distributed computing frameworks like - Hive/Hadoop, Apache Spark. Good experience in Test driven development and automating test cases using Java/Python Experience in SQL/NoSQL (Oracle, Cassandra) database design … following cloud services: AWS Elastic Beanstalk, EC2, S3, CloudFront, RDS, DynamoDB, VPC, Elastic Cache, Lambda Working experience with Terraform Experience in creating workflows for Apache Airflow About Roku Roku pioneered streaming to the TV. We connect users to the streaming content they love, enable content publishers to build More ❯

working with data formats such as JSON, CSV, and Parquet. • Strong understanding of IT concepts, including security, IAM, Key Vault, and networking. • Exposure to Apache Airflow and DBT is a bonus. • Familiarity with agile principles and practices. • Experience with Azure DevOps pipelines. The "Nice to Haves": • Certification in More ❯

help you get there: Understanding of cloud computing security concepts Proficiency with IaC tools like Terraform or CloudFormation. Experience with workflow orchestration tools (e.g., Airflow, Dagster). Good understanding of Cloud providers – AWS, Microsoft Azure, Google Cloud Familiarity with DBT, Delta Lake, Databricks Experience working in Agile environments with More ❯

help you get there: Understanding of cloud computing security concepts Proficiency with IaC tools like Terraform or CloudFormation. Experience with workflow orchestration tools (e.g., Airflow, Dagster). Good understanding of Cloud providers – AWS, Microsoft Azure, Google Cloud Familiarity with DBT, Delta Lake, Databricks Experience working in Agile environments with More ❯

help you get there: Understanding of cloud computing security concepts Proficiency with IaC tools like Terraform or CloudFormation. Experience with workflow orchestration tools (e.g., Airflow, Dagster). Good understanding of Cloud providers – AWS, Microsoft Azure, Google Cloud Familiarity with DBT, Delta Lake, Databricks Experience working in Agile environments with More ❯

with cloud-based ML infrastructure, particularly GCP (Vertex AI, BigQuery), or equivalent (e.g. AWS, Azure) Exposure to orchestration tools such as Kubeflow pipelines or Airflow Familiarity with DBT or similar tools for modelling data in data warehouses Desire to build interpretable and explainable ML models (using techniques such as More ❯

with cloud-based ML infrastructure, particularly GCP (Vertex AI, BigQuery), or equivalent (e.g. AWS, Azure) Exposure to orchestration tools such as Kubeflow pipelines or Airflow Familiarity with DBT or similar tools for modelling data in data warehouses Desire to build interpretable and explainable ML models (using techniques such as More ❯

with cloud-based ML infrastructure, particularly GCP (Vertex AI, BigQuery), or equivalent (e.g. AWS, Azure) Exposure to orchestration tools such as Kubeflow pipelines or Airflow Familiarity with DBT or similar tools for modelling data in data warehouses Desire to build interpretable and explainable ML models (using techniques such as More ❯

data modeling Proficient in data warehouse technologies (such as Amazon Redshift, Google BigQuery, or Snowflake) Hands-on experience with ETL tools and frameworks, including Apache Airflow, Talend, or dbt Strong programming ability in Python or another data-focused language Knowledgeable about data management best practices, including governance, security More ❯

and support automated monitoring systems to detect data anomalies, system failures, and performance issues and leverage advanced scripting and orchestration tools (e.g., Python, Bash, Apache Airflow) to automate workflows and reduce operational overhead. Root Cause Analysis & Incident Management: Lead post-incident reviews, perform root cause analysis for data More ❯

and support automated monitoring systems to detect data anomalies, system failures, and performance issues and leverage advanced scripting and orchestration tools (e.g., Python, Bash, Apache Airflow) to automate workflows and reduce operational overhead. Root Cause Analysis & Incident Management: Lead post-incident reviews, perform root cause analysis for data More ❯

with big data technologies such as Hadoop, Spark, or Hive Familiarity with data warehousing and ETL tools such as Amazon Redshift, Google BigQuery, or Apache Airflow Proficiency in Python and at least one other programming language such as Java, or Scala Willingness to mentor more junior members of More ❯

improve engagement and performance Python (for scripting and data wrangling) SQL (strong experience in querying and transforming data) Experience with building analytics data layers Airflow, dbt (for orchestration and transformation) Cloud data warehouses like BigQuery or Redshift Terraform, Kubernetes (for infrastructure as code and deployment) Seniority level Seniority level More ❯

skills with Python (or C#) and SQL You have experience with SQL databases (e.g. Amazon Redshift, PostgreSQL) You have experience with data tooling (e.g. Airflow, DBT, AWS Kinesis) You have strong analysis and problem solving skills You're collaborative with excellent communication skills What's in it for you More ❯

Solid understanding of ETL concepts, data integration, and data warehousing best practices. Familiarity with version control systems (e.g., Git) and workflow orchestration tools (e.g., Airflow, dbt) is a plus. Excellent problem-solving skills and attention to detail. Strong communication and collaboration abilities. Preferred Qualifications: Experience working in agile or More ❯

operational targets crossing multiple departments Nice-to-have experience: Infrastructure as code (Terraform, cloud formation etc...) Cloud platforms (Aws, Azure, GCP etc..) Data orchestration (Airflow, Dagster etc...) The Team The Data & Analytics department serves as the central hub for data engineering and insights across our organization, comprising four essential More ❯

processes Hands-on experience with cloud platforms (GCP, AWS, Azure) and their data-specific services Proficiency in Python, SQL, and data orchestration tools (e.g., Airflow, DBT) Experience with modern data warehouse technologies (BigQuery, Snowflake, Redshift, etc.) Strong understanding of data modeling, data governance, and data quality principles Excellent communication More ❯

usage, and data governance considerations, promoting transparency and responsible AI use. 7. Automate ETL pipeline orchestration and data processing workflows: Leverage orchestration tools like Apache Airflow, Perfect to schedule, automate, and manage ETL jobs, reducing manual intervention and improving operational reliability. 8. Implement monitoring, alerting, and troubleshooting for … operating systems. Experience with Amazon Web Services cloud platform or another cloud platform. Proven work experience with containerization and orchestration tools such as Docker, Airflow, Prefect and Kubernetes. Knowledge of cybersecurity best practices. Excellent communication and collaboration skills. Ability to work effectively in a fast-paced and dynamic environment. More ❯

following: A track record of consolidating data sources, creating pipelines, and ideally a software engineering mindset. Excellent experience with Containers. Strong knowledge of Python, Airflow, SQL, and Linux. Proficient in AWS Lambda, ECS, Athena, S3, Kinesis, Glue, CloudWatch/Terraform/CDK, Tableau Online. They are particularly keen on More ❯