1 to 25 of 29 Permanent Apache Airflow Jobs in the North West

automation to ensure successful project delivery, adhering to client timelines and quality standards. Implement and manage real-time and batch data processing frameworks (e.g., Apache Kafka, Apache Spark, Google Cloud Dataproc) in line with project needs. Build and maintain robust monitoring, logging, and alerting systems for client projects … languages like Python, Bash, or Go to automate tasks and build necessary tools. Expertise in designing and optimising data pipelines using frameworks like Apache Airflow or equivalent. Demonstrated experience with real-time and batch data processing frameworks, including Apache Kafka, Apache Spark, or Google Cloud Dataflow. ...

automation to ensure successful project delivery, adhering to client timelines and quality standards. Implement and manage real-time and batch data processing frameworks (e.g., Apache Kafka, Apache Spark, Google Cloud Dataproc) in line with project needs. Build and maintain robust monitoring, logging, and alerting systems for client projects … languages like Python, Bash, or Go to automate tasks and build necessary tools. Expertise in designing and optimising data pipelines using frameworks like Apache Airflow or equivalent. Demonstrated experience with real-time and batch data processing frameworks, including Apache Kafka, Apache Spark, or Google Cloud Dataflow. ...

automation to ensure successful project delivery, adhering to client timelines and quality standards. Implement and manage real-time and batch data processing frameworks (e.g., Apache Kafka, Apache Spark, Google Cloud Dataproc) in line with project needs. Build and maintain robust monitoring, logging, and alerting systems for client projects … languages like Python, Bash, or Go to automate tasks and build necessary tools. Expertise in designing and optimising data pipelines using frameworks like Apache Airflow or equivalent. Demonstrated experience with real-time and batch data processing frameworks, including Apache Kafka, Apache Spark, or Google Cloud Dataflow. ...

experience designing and building modern data platforms in AWS. ETL/Orchestration Expertise: Expertise in ETL/ELT design and data orchestration, specifically with Apache Airflow. SQL Mastery: Strong SQL skills with significant experience in query tuning and performance optimisation. Programming Proficiency: Proficiency in Python and Bash (for data ...

experience designing and building modern data platforms in AWS. ETL/Orchestration Expertise: Expertise in ETL/ELT design and data orchestration, specifically with Apache Airflow. SQL Mastery: Strong SQL skills with significant experience in query tuning and performance optimisation. Programming Proficiency: Proficiency in Python and Bash (for data ...

similar language. Proficiency in database technologies (SQL, NoSQL, time-series databases) and data modelling. Strong understanding of data pipeline orchestration tools (e.g., Apache Airflow, Kubernetes). You thrive when working as part of a team Comfortable in a fast-paced environment Have excellent written and verbal English skills ...

similar language. Proficiency in database technologies (SQL, NoSQL, time-series databases) and data modelling. Strong understanding of data pipeline orchestration tools (e.g., Apache Airflow, Kubernetes). You thrive when working as part of a team Comfortable in a fast-paced environment Have excellent written and verbal English skills ...

similar language. Proficiency in database technologies (SQL, NoSQL, time-series databases) and data modelling. Strong understanding of data pipeline orchestration tools (e.g., Apache Airflow, Kubernetes). You thrive when working as part of a team Comfortable in a fast-paced environment Have excellent written and verbal English skills ...

using concepts, ETL/ELT pipelines and dimensional modelling Proficiency in advanced programming languages (Python/PySpark, SQL) Experience in data pipeline orchestration (e.g. Airflow, Data Factory) Familiarity with DevOps and CI/CD practices (Git, Azure DevOps etc) Ability to communicate technical concepts to both technical ...

cloud platform (Azure, AWS or GCP). Exposure to big data technologies such as Spark or Hadoop. Experience with ETL/orchestration tools like Airflow, Talend or Azure Data Factory. Understanding of data warehousing solutions (Snowflake, BigQuery, Redshift or similar). Good understanding of data governance, GDPR and modern ...

tuning. Hands-on expertise with Fivetran and dbt. Good understanding of data modelling, governance, and security best practices. Familiarity with orchestration tools such as Airflow or Prefect (advantageous). Experience working in Azure (or AWS/GCP). Strong analytical and collaboration skills, with great attention to detail. ...

maintain highly reliable and well-tested data and ML pipelines to extract, transform, and structure large datasets for ML applications. Expertise in using Apache Airflow (or similar orchestration tools like Prefect/Dagster) to define, schedule, and monitor complex data and ML workflows (DAGs). Implement comprehensive software … engineering teams. Bonus Skills to Stand Out (Optional): Experience with CI/CD tooling (e.g., GitHub Actions, Jenkins) specifically for ML pipelines and Airflow DAG deployment. Experience with data quality monitoring tools and frameworks. Master's or PhD in Computer Science, Data Science, or a related field. Relevant certifications ...

Design, SQL and database optimizations Experience of being a bar-raiser and helping improve engineering best practices Experience of orchestration tools like Databricks or Airflow Experience with cloud data warehouses like Snowflake or BigQuery Coaching, mentoring and growing an established mid-level team who are eager to learn from ...

strong ability to learn it quickly. Strong SQL skills; working familiarity with Python for scripting and DBT automation. Experience integrating DBT with orchestration tools (Airflow, Dagster, Prefect, etc.). Solid understanding of modern data engineering, ELT patterns, and version-controlled analytics development. Preferred Prior experience operationalizing ML workflows inside ...

strong ability to learn it quickly. Strong SQL skills; working familiarity with Python for scripting and DBT automation. Experience integrating DBT with orchestration tools (Airflow, Dagster, Prefect, etc.). Solid understanding of modern data engineering, ELT patterns, and version-controlled analytics development. Preferred Prior experience operationalizing ML workflows inside ...

strong ability to learn it quickly. Strong SQL skills; working familiarity with Python for scripting and DBT automation. Experience integrating DBT with orchestration tools (Airflow, Dagster, Prefect, etc.). Solid understanding of modern data engineering, ELT patterns, and version-controlled analytics development. Preferred Prior experience operationalizing ML workflows inside ...

Excellent problem-solving and communication skills, with a focus on delivering high-quality solutions. > Understanding/hands on experience in Orchestration solutions such as Airflow > Deep knowledge of key non-functional requirements such as availability, scalability, operability, and maintainability ...

data pipelines at scale, you'll thrive here. Tech stack: Micro-services Container Platforms (Kubernetes, CRC, Docker) Big Data Technologies (Apache.... Spark, Flink, HDFS, Airflow, Trino, Iceberg, Hive) Dependency injection frameworks (Spring) Observability (Loki/Grafana) Large scale data processing (Kafka) CI/CD Build tools ( Maven, Git, Jenkins ...

data pipelines at scale, you'll thrive here. Tech stack: Micro-services Container Platforms (Kubernetes, CRC, Docker) Big Data Technologies (Apache.... Spark, Flink, HDFS, Airflow, Trino, Iceberg, Hive) Dependency injection frameworks (Spring) Observability (Loki/Grafana) Large scale data processing (Kafka) CI/CD Build tools ( Maven, Git, Jenkins ...

strong ability to learn it quickly. Strong SQL skills; working familiarity with Python for scripting and DBT automation. Experience integrating DBT with orchestration tools (Airflow, Dagster, Prefect, etc.). Solid understanding of modern data engineering, ELT patterns, and version-controlled analytics development. Preferred Prior experience operationalizing ML workflows inside ...

strong ability to learn it quickly. Strong SQL skills; working familiarity with Python for scripting and DBT automation. Experience integrating DBT with orchestration tools (Airflow, Dagster, Prefect, etc.). Solid understanding of modern data engineering, ELT patterns, and version-controlled analytics development. Preferred Prior experience operationalizing ML workflows inside ...

strong ability to learn it quickly. Strong SQL skills; working familiarity with Python for scripting and DBT automation. Experience integrating DBT with orchestration tools (Airflow, Dagster, Prefect, etc.). Solid understanding of modern data engineering, ELT patterns, and version-controlled analytics development. Preferred Prior experience operationalizing ML workflows inside ...

. Develop robust backend systems and APIs to support workflows. Build and maintain CI/CD pipelines, infrastructure-as-code, and container orchestration (Kubernetes, Airflow, Databricks Jobs). Ensure best practices in model tracking, versioning, performance monitoring, logging, and retraining workflows. Collaborate with cross-functional teams to align technical ...

Principal Data Engineer: Experience as a lead or principal data engineer. Prior experience designing data platform(s) within GCP, working hands on with; Airflow, Big Query, Data Flow, Data Fusion, and Data Stream. Deep understanding of Data Mesh/decentralised design and Data Lake/Warehouse solutions. Previously ...

Principal Data Engineer: Experience as a lead or principal data engineer. Prior experience designing data platform(s) within GCP, working hands on with; Airflow, Big Query, Data Flow, Data Fusion, and Data Stream. Deep understanding of Data Mesh/decentralised design and Data Lake/Warehouse solutions. Previously ...