1 to 25 of 51 Apache Airflow Jobs in London

maintain scalable, cloud-based data pipelines and architectures to support advanced analytics and machine learning initiatives. Develop robust ELT workflows using tools like dbt, Airflow, and SQL (PostgreSQL, MySQL) to transform raw data into high-quality, analytics-ready datasets. Collaborate with data scientists, analysts, and software engineers to ensure … however Azure exposure is essential. Data Warehousing: Proven expertise with Snowflake – schema design, performance tuning, data ingestion, and security. Workflow Orchestration: Production experience with Apache Airflow (Prefect, Dagster or similar), including authoring DAGs, scheduling workloads and monitoring pipeline execution. Data Modeling: Strong skills in dbt, including writing modular ...

similarly regulated industry. Strong coding skills in Python or Java (Spring Boot); React experience is a plus. Proficiency with modern data tools: Airflow, Spark, Kafka, dbt, Snowflake or similar. Experience with cloud platforms (AWS, Azure, GCP), containerization (Docker, Kubernetes), and CI/CD. Data Quality: Proven ability to validate ...

coding skills in Java (Spring Boot) for data engineering and services. Proven background working within financial services environments. Hands-on experience building and managing Apache Airflow/Spark and Hadoop clusters. Deep experience in big data engineering, including Hadoop, Kafka, and Spark-based data pipelines. Strong SQL/ ...

experience designing and building modern data platforms in AWS. ETL/Orchestration Expertise: Expertise in ETL/ELT design and data orchestration, specifically with Apache Airflow. SQL Mastery: Strong SQL skills with significant experience in query tuning and performance optimisation. Programming Proficiency: Proficiency in Python and Bash (for data ...

solving capabilities. · Excellent communication and documentation skills. · Ability to work independently and lead cross-functional teams. · Adaptability to rapidly evolving technological environments PREFERRED SKILLS · Airflow, dbt, Spark, Kafka, RabbitMQ, Redis. · Git, CI/CD pipelines. · Experience with data warehousing solutions such as Snowflake, Redshift, BigQuery, or Synapse. · Exposure ...

Data Engineering (not building pipelines but designing and building the framework) DevOps MLOps/LLMOps Often work with the following technologies : Azure, AWS, GCP Airflow, dbt, Databricks, Snowflake, etc. GitHub, Azure DevOps and related developer tooling and CI/CD platforms, Terraform or other Infra-as-Code MLflow, AzureML ...

using concepts, ETL/ELT pipelines and dimensional modelling Proficiency in advanced programming languages (Python/PySpark, SQL) Experience in data pipeline orchestration (e.g. Airflow, Data Factory) Familiarity with DevOps and CI/CD practices (Git, Azure DevOps etc) Ability to communicate technical concepts to both technical ...

schema). Version Control Practical experience with Git (branching, merging, pull requests). Preferred Qualifications (A Plus) Experience with a distributed computing framework like Apache Spark (using PySpark). Familiarity with cloud data services ( AWS S3/Redshift, Azure Data Lake/Synapse, or Google BigQuery/Cloud Storage … . Exposure to workflow orchestration tools ( Apache Airflow, Prefect, or Dagster ). Bachelor's degree in Computer Science, Engineering, Information Technology, or a related field. ...

Snowflake Production experience with dbt (mandatory) Strong SQL and Python programming skills Experience with Git-based workflows and DevOps practices Familiarity with orchestration tools (Airflow, Prefect) and ETL/ELT patterns Knowledge of cloud platforms (AWS, Azure) and security best practices ...

modular, reusable transformations using SQL and Python. Implement CI/CD pipelines and manage deployments via Git. Automate workflows using orchestration tools such as Airflow or dbt Cloud. Configure and optimise Snowflake warehouses for performance and cost efficiency. Required Skills & Experience 7+ years in data engineering roles. 3+ years ...

technologies (NoSQL, time-series databases) and deep competence in data modeling principles. Workflow Orchestration: Strong working knowledge of industry-standard data orchestration tools (e.g., Apache Airflow or Kubernetes). The Environment We are seeking a highly collaborative professional with strong problem-solving skills, capable of thriving ...

MLflow), model registry, CI/CD for ML. Serving and orchestration: AWS SageMaker, EKS, batch & real-time inference. Data pipelines: Spark/SQL, orchestration (Airflow), monitoring for drift/perf. Nice to Have Cost/performance optimisation on AWS; GPU scheduling; quantisation/distillation. Experience working within Financial Services ...

challenge the status quo and drive improvements. Confidence in collaborating across teams and markets to deliver impactful solutions. Nice-to-haves: Familiarity with Airflow , DBT , BigQuery , Redshift , Terraform , or Kubernetes . BENEFITS The successful Data Engineer will receive the following benefits: Salary up to £75,000 Flexible working arrangements ...

days/week from Central London office Strong problem-solving skills, attention to detail, and ability to work independently Experience with workflow orchestration (Prefect, Airflow, Dagster), cloud data warehouses (BigQuery, Snowflake), or backend Python frameworks (Django, FastAPI) is a plus Benefits 25 days annual leave plus bank holidays Meaningful ...

similar) and how they consume data Experience working with cloud environments (AWS, GCP) Familiarity with basic data quality testing and workflow orchestration concepts (e.g., Airflow) Experience working in Agile, cross-functional teams Python experience for automation or scripting would be advantageous ...

with hands on Snowflake experience* Proven dbt experience at scale* Strong dimensional modelling using Kimball principles* Python for data engineering tasks* Experience with Airflow or similar orchestration tools* Git based workflows* Familiarity with BI tools such as Tableau, Looker, or Power BI* Experience working in cloud environments including ...

technologies . You will be utilising and playing with tools across the entire data ecosystem. 🚀Essential tech skills include proficiency in Python, Spark, and Airflow , alongside experience with major data platforms such as Snowflake, Databricks, and BigQuery . 🚀Familiarity with data transformation and infrastructure as code tools like ...

have indepth technical knowledge of Python for data workflows, SQL and Data Modelling, DBT, Cloud platforms (AWS, GCP or Azure), Workflow Orchestration tools (e.g. Airflow, Dagster, Prefect, DBT Cloud), CI/CD (GitHub Actions, GitLab CI, CircleCI) You have experience of leading technical deployments, running customer projects, diagnosing issues ...

pipelines in a production environment. Exposure to cloud-based data platforms (AWS, GCP, Azure, or similar). Familiarity with data orchestration tools (e.g. Airflow or equivalent). Solid understanding of data warehousing concepts, schema design, and data modelling. Oscar Associates (UK) Limited is acting as an Employment Agency ...

investment professionals, and who thrives in fast-paced financial environments... Key Responsibilities Help users adopt and make effective use of AWS, Kubernetes, Terraform, Airflow, Jenkins, Coder, and other cloud/platform technologies Streamline operational workflows by automating routine tasks, enhancing documentation, and building out knowledge bases Work with engineering ...

another language e.g. Python ELT/ETL CI/CD Data Modelling Data Architecture Data Frameworks dbt Data processing and orchestration using Airflow or similar Google BigQuery or Redshift Cloud based services: GCP (preferred) or AWS Agentic development frameworks Automations About You 6 years+ experience as a Data Engineer ...

peer to peer feedback and evolving healthy, curious and humble teams. Tech Stack: Python, Javascript/Typescript, React/React Native, AWS, GraphQL, Snowflake, Airflow, DDD. This is an incredible opportunity for a Senior Software Engineer to join a unique company as they embark on a period of significant ...

working as a Data Engineer or in a similar data-focused engineering role Strong Python and SQL skills Experience building and orchestrating data pipelines (Airflow or similar tools) Hands-on work with cloud platforms (AWS and/or Azure) Solid understanding of databases, data warehouses, and data lakes Experience ...

Azure) Nice to Have Experience with fraud, risk, credit, AML, pricing, or customer analytics use cases Familiarity with MLOps tools (MLflow, Kubeflow, Airflow, etc.) Docker and Kubernetes experience Exposure to model governance, explainability, or regulatory frameworks Contract Details £650–£750 per day (Outside IR35) Initial 6-month contract, with ...

data pipelines using Python and SQL to ensure clean, reliable, and well-structured data. You'll design and optimise workflows using tools such as Airflow and dbt to support scalable, high-quality analytics layers. You'll help architect and evolve cloud-based data platforms across technologies like BigQuery, Redshift ...