1 to 25 of 38 Apache Airflow Jobs in London

governance, security, and regulatory compliance . Ability to work closely with technical and business teams to deliver impactful solutions. Nice-to-Haves: Orchestration tools (Airflow, dbt, Prefect) or streaming frameworks (Kafka, Kinesis). CI/CD experience and DevOps practices for data workflows. Exposure to data visualisation platforms (PowerBI ...

governance, security, and regulatory compliance . Ability to work closely with technical and business teams to deliver impactful solutions. Nice-to-Haves: Orchestration tools (Airflow, dbt, Prefect) or streaming frameworks (Kafka, Kinesis). CI/CD experience and DevOps practices for data workflows. Exposure to data visualisation platforms (PowerBI ...

architecture, modelling, pipelines) (please be clear on years and/or projects) Strong Python and SQL Experience with orchestration and transformation frameworks such as Airflow, dbt, Dagster , or similar Experience designing and delivering data platforms or lakehouse-style architectures Solid understanding of CI/CD for data engineering (e.g. ...

industry. Strong full-stack experience with Python , Java (Spring Boot) , React , SQL, and relational databases. Hands-on experience with modern data engineering tools (e.g. Airflow, Spark, Kafka, dbt, Snowflake or similar). Experience with cloud platforms , containerisation (Docker/Kubernetes) , and CI/CD pipelines . Solid understanding ...

pipelines Proficiency with at least one cloud platform: Amazon Web Services (AWS) Microsoft Azure (Azure) Google Cloud Platform (GCP) Experience with orchestration tools (e.g., Apache Airflow) Experience with version control (e.g., Git) Solid understanding of data warehousing concepts and dimensional modeling To apply for this role please submit ...

data modelling Python preferred, or similar, for automation and pipeline development Cloud exposure across AWS, Azure, or GCP Familiarity with tools like dbt, Airflow, or equivalents Good habits around governance, security, documentation, Git, and CI/CD You’ll thrive if you’re Curious, motivated, and pragmatic, you care ...

data modelling Python preferred, or similar, for automation and pipeline development Cloud exposure across AWS, Azure, or GCP Familiarity with tools like dbt, Airflow, or equivalents Good habits around governance, security, documentation, Git, and CI/CD You’ll thrive if you’re Curious, motivated, and pragmatic, you care ...

Data Engineer: Experience with Databricks , Kafka , or Kinesis for real-time data streaming Knowledge of containerisation (Docker, ECS) and modern orchestration tools such as Airflow Familiarity with machine learning model deployment pipelines or data lakehouse architectures Data Engineer, AWS Data Engineer ...

working in fast-paced environments and able to contribute quickly without extensive onboarding. Nice to Have Experience with modern data stack tools (e.g., dbt, Airflow, etc.). Understanding of CI/CD pipelines and data infrastructure automation. Familiarity with data governance, security, and best practices in a cloud environment. ...

Python (and ideally C#) programming skills You have experience with data modelling, ETL pipelines and data integration You have a good knowledge of Dagster, Airflow or similar orchestration tool You're collaborative and pragmatic with great stakeholder communication skills What's in it for you: Salary to £120k Pension ...

working in fast-paced environments and able to contribute quickly without extensive onboarding. Nice to Have Experience with modern data stack tools (e.g., dbt, Airflow, etc.). Understanding of CI/CD pipelines and data infrastructure automation. Familiarity with data governance, security, and best practices in a cloud environment. ...

tooling Good scripting skills (Python/SQL/Bash) Strong understanding of AWS security, IAM, and networking Comfortable operating in Agile, delivery-led environments Apache Airflow for DAG Date-Led Platform/Cloud Engineering Key requirements: Active SC Clearance (essential) Permanent role Ideally you will be based around ...

Familiarity with cloud-based data tools and modern data ecosystems. Understanding of data modeling, warehousing concepts, and best practices Nice-to-Have Experience with Airflow, dbt, or similar orchestration/modeling tools. Experience with BI tools Knowledge of GCP services (Cloud Storage, Dataflow, Pub/Sub, etc.). Social ...

data structures, performance, and optimisation strategies Familiarity with GCP and ecosystem knowledge: BigQuery, Composer, Dataproc, Cloud Run, Dataplex Hands-on experience with orchestration (like Airflow, Dagster, Databricks Workflows) and distributed processing in a cloud environment Experience with analytical data modelling (star and snowflake schemas), DWH, ETL/ELT patterns ...

grade data pipelines for multiple internal teams. Develops and optimises central data models supporting analytics, ML, and operational workloads. Works with Databricks, AWS, dbt, Airflow, and Looker as part of a modern data stack. Leads complex technical initiatives and improves efficiency through CI/CD, observability, and tooling. Ensures ...

Data Engineer in a consultancy or complex delivery environment Deep, hands-on expertise with Snowflake Experience with the modern data stack, including dbt, Airflow (or similar) and Python Solid experience with AWS and/or Azure Strong SQL skills and performance optimisation experience Familiarity with CI/CD, DevOps ...

tasks, working with the Snowflake Data Marketplace, Performance Tuning and Knowledge of ETL/ELT pipelines and data integration tools (e.g., Informatica, Matillion, DBT, Airflow). Experience with APIs, stored procedures, and automation scripts is a plus. Strong analytical and problem-solving mindset. Clear communicator, able to work across ...

technical expertise with mentoring, guidance, and collaboration across teams. Core technologies include: Google Cloud Platform (BigQuery, Dataflow, Dataproc, Data Fusion, Data Streams, Cloud Functions, Airflow/Composer), SQL, Python, CI/CD tooling About you You'll have 10+ years experience in Data Engineering Experience operating in a Lead ...

Testing). Familiarity with C++ or Go is a significant plus. Tooling & Orchestration: Deep experience with containerization (Docker, Kubernetes) and orchestration tools like Kubeflow, Airflow, or Metaflow. Cloud Architecture: Expertise in deploying ML workloads on AWS, GCP, or Azure, utilizing managed services like SageMaker or Vertex AI. Data Foundations ...

experience with Google Cloud Platform (GCP), ideally BigQuery. Proficiency in SQL for data transformation and optimisation. Experience with automation tools such as Python, Airflow, dbt, Power Automate, or similar. Understanding of data security, permissions, and governance best practices. Soft Skills: Excellent communication and ability to translate complex data concepts ...

development experience. Experience building lakehouse architectures using Delta Lake. Strong cloud experience in Azure (preferred), AWS, or GCP. Experience with orchestration tools (ADF, Airflow, dbt, or similar). Certifications – Databricks Certified Data Engineer/Azure Data Engineer What’s in it for you: Fully remote role with flexible working ...

build of cloud data warehouses - Snowflake experience is desirable, though Redshift or Synapse are also suitable - Experience with data orchestration tools such as Airflow or Prefect - Strong coding ability in Python, Java, or Scala - Experience building data infrastructure from scratch - Comfortable working with both batch and streaming data - Experience ...

keys, sort keys , and workload management. Implement Redshift Spectrum to query data in S3 using external tables. 3. Data Lake & Iceberg Implement and maintain Apache Iceberg tables on AWS for schema evolution and ACID operations. Build Iceberg-based ingestion and transformation pipelines using Glue, EMR, or Spark. Ensure high … monitoring. Required Skills 8 10 years ' experience in data engineering. Strong hands-on experience with: AWS S3, Athena, Glue, Redshift, EMR/Spark Apache Iceberg Python/SQL Experience in telecom data pipelines and handling large-scale structured/semi-structured data. Strong problem-solving, optimization, and debugging skills. ...

clearly. Excellent problem-solving skills and the ability to translate business requirements into technical solutions. Languages: Python (primary), SQL, Bash Cloud: Azure, AWS Tools: Airflow, DBT Data Platforms: Snowflake, Delta Lake, Redis, Azure Data Lake Infrastructure and Operations: Terraform, GitHub Actions, Azure DevOps, Azure Monitor Desirable Skills and Experience … platforms such as Snowflake and Azure Data Lake. Understanding of monitoring, model performance tracking and observability best practices. Familiarity with orchestration tools such as Airflow or Azure Data Factory. ...

including managing dependencies and build pipelines without internet access Proven background building APIs for data enrichment services Orchestration & data pipeline experience with Prefect or Airflow, and K8s or Swarm Configuration and automation of business workflow tooling (e.g., Jira) Hands-on experience with a range of databases and storage technologies ...