1 to 25 of 26 Permanent Apache Airflow Jobs in London

governance, security, and regulatory compliance . Ability to work closely with technical and business teams to deliver impactful solutions. Nice-to-Haves: Orchestration tools (Airflow, dbt, Prefect) or streaming frameworks (Kafka, Kinesis). CI/CD experience and DevOps practices for data workflows. Exposure to data visualisation platforms (PowerBI ...

governance, security, and regulatory compliance . Ability to work closely with technical and business teams to deliver impactful solutions. Nice-to-Haves: Orchestration tools (Airflow, dbt, Prefect) or streaming frameworks (Kafka, Kinesis). CI/CD experience and DevOps practices for data workflows. Exposure to data visualisation platforms (PowerBI ...

industry. Strong full-stack experience with Python , Java (Spring Boot) , React , SQL, and relational databases. Hands-on experience with modern data engineering tools (e.g. Airflow, Spark, Kafka, dbt, Snowflake or similar). Experience with cloud platforms , containerisation (Docker/Kubernetes) , and CI/CD pipelines . Solid understanding ...

data modelling Python preferred, or similar, for automation and pipeline development Cloud exposure across AWS, Azure, or GCP Familiarity with tools like dbt, Airflow, or equivalents Good habits around governance, security, documentation, Git, and CI/CD You’ll thrive if you’re Curious, motivated, and pragmatic, you care ...

data modelling Python preferred, or similar, for automation and pipeline development Cloud exposure across AWS, Azure, or GCP Familiarity with tools like dbt, Airflow, or equivalents Good habits around governance, security, documentation, Git, and CI/CD You’ll thrive if you’re Curious, motivated, and pragmatic, you care ...

working in fast-paced environments and able to contribute quickly without extensive onboarding. Nice to Have Experience with modern data stack tools (e.g., dbt, Airflow, etc.). Understanding of CI/CD pipelines and data infrastructure automation. Familiarity with data governance, security, and best practices in a cloud environment. ...

Python (and ideally C#) programming skills You have experience with data modelling, ETL pipelines and data integration You have a good knowledge of Dagster, Airflow or similar orchestration tool You're collaborative and pragmatic with great stakeholder communication skills What's in it for you: Salary to £120k Pension ...

working in fast-paced environments and able to contribute quickly without extensive onboarding. Nice to Have Experience with modern data stack tools (e.g., dbt, Airflow, etc.). Understanding of CI/CD pipelines and data infrastructure automation. Familiarity with data governance, security, and best practices in a cloud environment. ...

tooling Good scripting skills (Python/SQL/Bash) Strong understanding of AWS security, IAM, and networking Comfortable operating in Agile, delivery-led environments Apache Airflow for DAG Date-Led Platform/Cloud Engineering Key requirements: Active SC Clearance (essential) Permanent role Ideally you will be based around ...

Familiarity with cloud-based data tools and modern data ecosystems. Understanding of data modeling, warehousing concepts, and best practices Nice-to-Have Experience with Airflow, dbt, or similar orchestration/modeling tools. Experience with BI tools Knowledge of GCP services (Cloud Storage, Dataflow, Pub/Sub, etc.). Social ...

grade data pipelines for multiple internal teams. Develops and optimises central data models supporting analytics, ML, and operational workloads. Works with Databricks, AWS, dbt, Airflow, and Looker as part of a modern data stack. Leads complex technical initiatives and improves efficiency through CI/CD, observability, and tooling. Ensures ...

Data Engineer in a consultancy or complex delivery environment Deep, hands-on expertise with Snowflake Experience with the modern data stack, including dbt, Airflow (or similar) and Python Solid experience with AWS and/or Azure Strong SQL skills and performance optimisation experience Familiarity with CI/CD, DevOps ...

technical expertise with mentoring, guidance, and collaboration across teams. Core technologies include: Google Cloud Platform (BigQuery, Dataflow, Dataproc, Data Fusion, Data Streams, Cloud Functions, Airflow/Composer), SQL, Python, CI/CD tooling About you You'll have 10+ years experience in Data Engineering Experience operating in a Lead ...

Testing). Familiarity with C++ or Go is a significant plus. Tooling & Orchestration: Deep experience with containerization (Docker, Kubernetes) and orchestration tools like Kubeflow, Airflow, or Metaflow. Cloud Architecture: Expertise in deploying ML workloads on AWS, GCP, or Azure, utilizing managed services like SageMaker or Vertex AI. Data Foundations ...

development experience. Experience building lakehouse architectures using Delta Lake. Strong cloud experience in Azure (preferred), AWS, or GCP. Experience with orchestration tools (ADF, Airflow, dbt, or similar). Certifications – Databricks Certified Data Engineer/Azure Data Engineer What’s in it for you: Fully remote role with flexible working ...

build of cloud data warehouses - Snowflake experience is desirable, though Redshift or Synapse are also suitable - Experience with data orchestration tools such as Airflow or Prefect - Strong coding ability in Python, Java, or Scala - Experience building data infrastructure from scratch - Comfortable working with both batch and streaming data - Experience ...

keys, sort keys , and workload management. Implement Redshift Spectrum to query data in S3 using external tables. 3. Data Lake & Iceberg Implement and maintain Apache Iceberg tables on AWS for schema evolution and ACID operations. Build Iceberg-based ingestion and transformation pipelines using Glue, EMR, or Spark. Ensure high … monitoring. Required Skills 8 10 years ' experience in data engineering. Strong hands-on experience with: AWS S3, Athena, Glue, Redshift, EMR/Spark Apache Iceberg Python/SQL Experience in telecom data pipelines and handling large-scale structured/semi-structured data. Strong problem-solving, optimization, and debugging skills. ...

Linux environment. - Experience in building automation tools and managing system configurations. - Knowledge of C++, KDB/q, and experience with technologies like Slurm, Airflow, Kafka, or AMPS. - Background in enhancing system stability, scalability, and performance while conducting root cause analyses to resolve incidents efficiently. - Observability skillset, monitoring and analysis ...

looking for Extensive experience in data engineering, including time in a lead role Strong Python and SQL skills Experience with orchestration tools (Airflow, Dagster, Prefect) Hands-on dbt , AWS, and modern data warehouses Strong data modelling and architectural capability Background scaling data platforms in high-growth environments Nice … have Snowflake or Apache Iceberg Infrastructure-as-Code (Terraform) Observability-driven data systems Package & Benefits Salary up to £120,000 Fully remote working Home-office setup Private healthcare and income protection 25 days holiday + enhanced parental leave Strong learning and development budget ...

objects (databases, schemas, tables, views, tasks, streams, resource monitors). ETL/ELT Pipelines: Develop and maintain robust data pipelines using tools like dbt, Airflow, Azure Data Factory, or similar. Data Modelling: Implement dimensional models (star/snowflake schemas), handle SCDs, and design efficient structures for analytics. Performance Tuning … Degree in IT Strong SQL skills (complex queries, performance tuning) and proficiency in Python for data processing. Experience with ETL/ELT tools (dbt, Airflow, ADF, Informatica, Matillion). Solid understanding of data warehousing concepts (Kimball, Data Vault, normalization). Familiarity with cloud platforms (Azure preferred; AWS/ ...

What you'll be doing: Design, develop, and maintain scalable data architectures and ETL pipelines Build and manage data models and data warehouse solutions (Airflow, dbt, and Redshift) Write clean, efficient Python and SQL code for data processing and transformation Integrate data from internal and third-party APIs … principles Ability to optimise data pipelines for performance and maintainability Strong collaboration skills and a proactive, problem-solving mindset Bonus Points For Experience with Airflow and/or dbt Experience working in Agile environments (Scrum/Kanban) Exposure to DevOps practices or CI/CD pipelines You'll join ...

What you'll be doing: Design, develop, and maintain scalable data architectures and ETL pipelines Build and manage data models and data warehouse solutions (Airflow, dbt, and Redshift) Write clean, efficient Python and SQL code for data processing and transformation Integrate data from internal and third-party APIs … principles Ability to optimise data pipelines for performance and maintainability Strong collaboration skills and a proactive, problem-solving mindset Bonus Points For Experience with Airflow and/or dbt Experience working in Agile environments (Scrum/Kanban) Exposure to DevOps practices or CI/CD pipelines You'll join ...

with engineers, architects, and product managers on data solutions. Support deployment, testing, and documentation of data products. Work with cloud-based tools, including BigQuery, Airflow, Terraform, DBT, and GIT. Continuously learn new tools and technologies, contributing to team knowledge and best practices. Key Skills & Experience: 1–3 years’ experience … roles. Strong SQL and Python skills. Exposure to cloud platforms (GCP preferred; Azure/AWS acceptable) . Familiarity with data tooling: DBT, GIT, Terraform, Airflow, Docker (nice to have). Good problem-solving skills, attention to detail, and collaborative mindset. Interested in Agile practices, Test-Driven Design, and modern ...

Company Description Who we are At The Stepstone Group, we have a simple yet very important mission: The right job for everyone. Using our data, platform, and technology, we create opportunities for jobseekers and companies ...

Senior DataEngineer-RemoteWorking LynxarecurrentlyworkingwithalargeITconsultancytohelpthemsourceaDataEngineer.OurclientislargeglobalConsultancywhoworkwithenterpriseclients,solvingtheirbusinessandtechnologyproblemsusingcuttingedgesolutions. Asa DataEngineer ,youwillplayakeyroleindesigning,building,andmaintainingrobustdataplatformsandpipelines.Youllworkalongsidedatascientists,analysts,andengineerstoensurereliable,scalable,andhigh-qualitydatasolutions. Whatyoullbedoing: Designingandbuildingdatapipelines,models,andarchitectures DevelopingandmaintainingETL/ELTprocessesusingmoderntools Implementingdatastoragesolutionsacrossrelational,non-relational,andcloudplatforms Ensuringdataquality,reliability,andoperationalexcellence Leadingthedesignanddeliveryofnewdataproductsandpipelines Collaboratingcloselywithstakeholdersandtechnicalteams Providingtechnicalleadershipandmentoringwithindeliveryteams Whattherelookingfor: Strongexperienceindataengineeringroles Provenleadershiporlinemanagementexperience ...