51 to 75 of 129 Apache Airflow Jobs in the UK

Skills & Experience Strong experience with Python and/or SQL in a production environment. Hands-on experience with modern data engineering frameworks such as: Apache Spark, Databricks Airflow/Prefect/Dagster Kafka/EventHub/Kinesis Experience building data solutions on at least one major cloud platform ...

Overview Lead a team of Data Analysts and own the delivery of high-impact analytical products using modern data engineering tooling (AWS, Databricks, dbt, Airflow). You will bridge the gap between business stakeholders and the data platform. Key Responsibilities Lead and mentor a team of Data Analysts Build … maintain self-serve analytical assets using dbt models and Databricks Design and orchestrate analytical pipelines in Airflow Translate complex business requirements into actionable insights and dashboards Perform deep-dive analysis across Retail, Commercial, Risk and Marketing domains Ensure all deliverables meet strict data governance and regulatory standards Work closely ...

Required Data Integration & Ingestion Data Manipulation & Processing GitHub, GitHub Actions, Azure DevOps Azure Data Factory Databricks SQL Database Azure Synapse Stream Analytics AWS Glue Apache Airflow AWS Kinesis Amazon Redshift SonarQube PyTest Optional/Nice to Have Experience managing projects or running Scrum teams Experience with BPC/ ...

Location: London, UK (3 Days/Week Onsite) Type: Fixed Term Contract (6 Months) Mandatory Skills Needed : SQL ( Advanced) , AWS Cloud (Glue,Ethena,EMR), Airflow, Kafka/Kenesis ,Python, Terraform DBT & Snowflake (Basic) AWS Data Engineer Design, develop, and maintain data pipelines for collecting, transforming, and loading data into ...

Greenfield project. What you'll need to succeed Strong Python and SQL are crucial to this role On-prem to Snowflake migration experience Airflow and DBT experience GitHub experience as you will be building CI/CD pipelines as part of this role Any experience within post-trade ...

analytics Fraud, AML, or transaction monitoring models ☁️ Cloud platforms (AWS, GCP, Azure) Experimentation & optimisation frameworks ⚙️ Typical Tech Stack Python • SQL • Spark • AWS/GCP • Airflow • Tableau/Power BI • Git 🚀 Why Join Work with large-scale behavioural datasets unique to iGaming Direct commercial impact on product & trading decisions Fast ...

commercial experience working as a Data Engineer in AWS. Solid programming skills in Python and strong SQL capability. Exposure to orchestration tools such as Airflow or Dagster. Experience with modern data warehouses such as Snowflake, Redshift or BigQuery. Infrastructure-as-code experience, ideally Terraform or Pulumi. BENEFITS The successful ...

Familiarity with cloud platforms such as AWS, Azure or Google Cloud and associated MLOps practices. Experience with automation and orchestration tooling such as UiPath, Airflow or Kubeflow. Experience and Capability Strong experience delivering AI or automation solutions within enterprise environments. Ability to work with cross functional teams and translate ...

commercial experience working as a Data Engineer in AWS. Solid programming skills in Python and strong SQL capability. Exposure to orchestration tools such as Airflow or Dagster. Experience with modern data warehouses such as Snowflake, Redshift or BigQuery. Infrastructure-as-code experience, ideally Terraform or Pulumi. BENEFITS The successful ...

commercial experience working as a Data Engineer in AWS. Solid programming skills in Python and strong SQL capability. Exposure to orchestration tools such as Airflow or Dagster. Experience with modern data warehouses such as Snowflake, Redshift or BigQuery. Infrastructure-as-code experience, ideally Terraform or Pulumi. BENEFITS The successful ...

commercial experience working as a Data Engineer in AWS. Solid programming skills in Python and strong SQL capability. Exposure to orchestration tools such as Airflow or Dagster. Experience with modern data warehouses such as Snowflake, Redshift or BigQuery. Infrastructure-as-code experience, ideally Terraform or Pulumi. BENEFITS The successful ...

Candidates will be asked to rate their knowledge of: Languages: Python, Java, Scala Compute/Distributed: RAY , Spark, Databricks, Hadoop Orchestration/Cloud: Kafka, Airflow, Prefect, AWS Storage/Warehouse: Clickhouse, Snowflake, Redshift, Greenplum Note: Experience with Petabyte/Exabyte scale is a strong signal for this role. Randstad ...

understanding ML pipelines and models in a production setting. Direct experience with Google Cloud Platform, BigQuery, and associated tooling. Experience with workflow tools like Airflow or Kubeflow. Familiarity with dbt (Data Build Tool). Please send your CV for more information on these roles. Reasonable Adjustments: Respect and equality ...

data platforms and backend systems. Familiarity with data governance frameworks, privacy compliance, and automated data quality checks. Hands-on experience with big data tools (Airflow, Databricks) and data observability platforms. Collaborative mindset and experience working with cross-functional teams including ML and analytics specialists. Curiosity and enthusiasm for continuous ...

mindset, with openness to feedback and diverse perspectives. Bonus points if you have the following. Experience building and maintaining production-grade data pipelines (e.g. Airflow, Dagster) or integrating data from APIs, web scraping, or third-party providers. Familiarity with cloud environments such as Google Cloud or AWS, and deploying ...

with Cloudera Data Platform (CDP) on AWS or recent Hadoop administration on similar platforms. AWS administration and platform management experience. Strong experience in Spark, Airflow, Apache Ranger, Impala (AWS CDK is a plus). Agile experience with Scrum, and tools including Jira, Slack. Familiarity with PowerBI, CI/ ...

morale Strong communication and stakeholder management skills Highly Desirable Experience in cloud-native or containerised environments Exposure to data tooling or workflow orchestration (e.g., Airflow, dbt) Experience scaling teams or improving engineering processes Experience mentoring early- to mid-career engineers Why Consider This Role? Real ownership over both technical ...

MLflow) Experience with cloud-based ML tools (e.g. SageMaker), data and model versioning (e.g. DVC), CI/CD (e.g. GitHub Actions), workflow orchestration (e.g. Airflow/Dagster) and containerised solutions (e.g. Docker, ECS) nice to have Experience in code testing (unit, integration, end-to-end tests) Strong data engineering ...

match/merge, survivorship, stewardship, integrations. Data integration: APIs/events, batch/streaming ingestion, CDC, lineage. Data engineering: data lake/lakehouse, orchestration (Airflow/MWAA), ETL/ELT (e.g., Glue), DQ frameworks. Discovery services: BI/analytics (Power BI/Tableau), ML feature pipelines, self-service data ...

team of 6–10 Data Engineers (onshore + offshore) Own the technical roadmap for the data platform (AWS + Databricks + dbt + Airflow) Architect scalable, cost-efficient and governed data solutions Drive best practices in engineering, testing, observability and documentation Act as senior technical point of contact … Experience 8+ years total experience with at least 3 years in a Lead/Principal Data Engineer role Expert-level AWS , Databricks , dbt and Apache Airflow Proven experience leading teams in a regulated financial services environment Strong stakeholder management and ability to translate business needs into technical solutions ...

SaaS systems Strong SQL and Python skills Understanding of cloud networking considerations for data ingestion Experience working in enterprise-scale data environments Experience with Airflow or other orchestration frameworks Exposure to distributed data processing frameworks such as Spark Google Associate Cloud Engineer or Professional Cloud Engineer certification (desirable) Please ...

custom ETL pipelines in a cloud-native environment. • Automating internal processes and integrations across tools and APIs. • Supporting workflow orchestration using technologies such as Airflow or AWS Step Functions. • Containerising and deploying services using Docker. • Collaborating with data scientists to streamline their data workflows. • Contributing to improvements across orchestration ...

custom ETL pipelines in a cloud-native environment. * Automating internal processes and integrations across tools and APIs. * Supporting workflow orchestration using technologies such as Airflow or AWS Step Functions. * Containerising and deploying services using Docker. * Collaborating with data scientists to streamline their data workflows. * Contributing to improvements across orchestration ...

background as a Machine Learning Engineer. Technical Skills: Strong in SQL and Python (Pandas, Scikit-learn, Jupyter, Matplotlib). Data transformation & manipulation : experience with Airflow, DBT and Kubeflow Cloud: Experience with GCP and Vertex AI (developing ML services). Expertise: Solid understanding of computer science fundamentals and time-series ...

testable systems Solid background in microservices architecture Proficiency with Postgres & MongoDB (relational + non-relational) Experience with event-driven architectures and asynchronous workflows (Kafka, Airflow, etc.) Solid coding practices (clean, testable, automated) The mindset of a builder: thrives in fast-paced startup environments, takes ownership, solves complex challenges Bonus ...