101 to 125 of 128 Apache Airflow Jobs in the UK

Tech Lead Manager in a data, platform or backend environment A background in data or backend engineering, such as AWS, Python or Scala, Spark, Airflow or similar Experience owning or supporting analytics, data or reporting products Strong people leadership skills including hiring, coaching and performance management A track record ...

healthcare, fintech, or other regulated environments Exposure to LLM infrastructure, model serving, or agentic AI frameworks in production Familiarity with data platforms (BigQuery, Airflow, data pipelines) Experience with CI/CD pipelines (GitHub Actions, Cloud Build) and policy-as-code tools Exposure to observability stacks (OpenTelemetry, Prometheus, Cloud Monitoring ...

Python and SQL. Strong experience building ETL pipelines and distributed data systems. Solid cloud experience — ideally AWS (open to GCP). Orchestration experience (Dagster, Airflow, or similar). Experience with modern data warehouses such as Snowflake, Redshift, or BigQuery. Infrastructure‐as‐code experience (Terraform or Pulumi). Strong data … development is highly desirable. Experience in high‐growth or smaller technology environments. Nice‐to‐haves Snowflake experience. Experience with dbt, Fivetran, AWS Glue, or Apache Iceberg. Prior leadership or mentoring experience (team lead/tech lead). What They Offer The opportunity to influence architecture, tooling, and engineering standards ...

visualisation tools. Background in Financial Services is a plus. Experience with cloud data environments (e.g. Redshift, Azure) welcomed. Knowledge of automation tools such as Airflow is beneficial. Excellent communication skills, confident engaging with senior stakeholders. Proactive, results-driven, and comfortable working in a fast-paced environment. ...

including managing dependencies and build pipelines without internet access Proven background building APIs for data enrichment services Orchestration & data pipeline experience with Prefect or Airflow, and K8s or Swarm Configuration and automation of business workflow tooling (e.g., Jira) Hands-on experience with a range of databases and storage technologies ...

AI. Comfortable working in a fast-paced, agile environment. Excellent communication and collaboration skills. Nice to Have Experience with workflow orchestration tools (e.g., Temporal, Airflow). Familiarity with vector databases or semantic search tools. Experience building internal tooling or developer productivity enhancements using AI. Exposure to customer support ...

What you'll be doing: Design, develop, and maintain scalable data architectures and ETL pipelines Build and manage data models and data warehouse solutions (Airflow, dbt, and Redshift) Write clean, efficient Python and SQL code for data processing and transformation Integrate data from internal and third-party APIs … principles Ability to optimise data pipelines for performance and maintainability Strong collaboration skills and a proactive, problem-solving mindset Bonus Points For Experience with Airflow and/or dbt Experience working in Agile environments (Scrum/Kanban) Exposure to DevOps practices or CI/CD pipelines You'll join ...

Senior Data Engineer (AWS, Airflow, Python) Based at client locations, working remotely, or based in our Godalming or Milton Keynes offices. Salary up to £65k plus company benefits. About Us Triad Group Plc is an award-winning digital, data, and solutions consultancy with over 35 years' experience primarily serving … maintain cloud data solutions. Maintain modular Terraform based IaC for reliable provisioning of AWS infrastructure. Develop, optimise and maintain robust data pipelines using Apache Airflow. Implement data transformation processes using Python to clean, preprocess, and enrich data for analytical use. Collaborate with data analysts, data scientists, developers, and other ...

e.g. AWS, GCP, Azure), infrastructure-as-code, and scalable architecture patterns Hands-on experience with SQL, Python, and modern data integration frameworks such as Apache Kafka, airflow or dbt Proven ability to deliver enterprise-grade customer data solutions, including customer data platforms and or API based integration Excellent ...

deliver reliable, high-performance data solutions using Databricks and modern cloud technologies. Key Responsibilities Design, develop, and maintain scalable data pipelines using Databricks, Apache Spark, and Delta Lake Build and optimize ETL/ELT workflows for batch and streaming data Develop data models to support analytics, reporting, and machine … data pipelines, architectures, and best practices Required Qualifications 3+ years of experience in data engineering or a similar role Strong experience with Databricks and Apache Spark Proficiency in Python and SQL Experience with Delta Lake, data modeling, and performance tuning Hands-on experience with cloud platforms (AWS, Azure ...

looking for Extensive experience in data engineering, including time in a lead role Strong Python and SQL skills Experience with orchestration tools (Airflow, Dagster, Prefect) Hands-on dbt , AWS, and modern data warehouses Strong data modelling and architectural capability Background scaling data platforms in high-growth environments Nice … have Snowflake or Apache Iceberg Infrastructure-as-Code (Terraform) Observability-driven data systems Package & Benefits Salary up to £120,000 Fully remote working Home-office setup Private healthcare and income protection 25 days holiday + enhanced parental leave Strong learning and development budget ...

with engineers, architects, and product managers on data solutions. Support deployment, testing, and documentation of data products. Work with cloud-based tools, including BigQuery, Airflow, Terraform, DBT, and GIT. Continuously learn new tools and technologies, contributing to team knowledge and best practices. Key Skills & Experience: 1–3 years’ experience … roles. Strong SQL and Python skills. Exposure to cloud platforms (GCP preferred; Azure/AWS acceptable) . Familiarity with data tooling: DBT, GIT, Terraform, Airflow, Docker (nice to have). Good problem-solving skills, attention to detail, and collaborative mindset. Interested in Agile practices, Test-Driven Design, and modern ...

with engineers, architects, and product managers on data solutions. Support deployment, testing, and documentation of data products. Work with cloud-based tools, including BigQuery, Airflow, Terraform, DBT, and GIT. Continuously learn new tools and technologies, contributing to team knowledge and best practices. Key Skills & Experience: 1–3 years' experience … roles. Strong SQL and Python skills. Exposure to cloud platforms (GCP preferred; Azure/AWS acceptable) . Familiarity with data tooling: DBT, GIT, Terraform, Airflow, Docker (nice to have). Good problem-solving skills, attention to detail, and collaborative mindset. Interested in Agile practices, Test-Driven Design, and modern ...

Desired Skills (Bonus Points) Experience with MLOps tooling (MLflow, Kubeflow, Weights & Biases, or similar) Building data pipelines (real-time or batch) using tools like Apache Spark, Kafka, Airflow, or dbt Familiarity with recommender systems, transformers, or graph neural networks Exposure to model optimisation techniques (quantisation, distillation, efficient inference ...

objects (databases, schemas, tables, views, tasks, streams, resource monitors). ETL/ELT Pipelines: Develop and maintain robust data pipelines using tools like dbt, Airflow, Azure Data Factory, or similar. Data Modelling: Implement dimensional models (star/snowflake schemas), handle SCDs, and design efficient structures for analytics. Performance Tuning … Degree in IT Strong SQL skills (complex queries, performance tuning) and proficiency in Python for data processing. Experience with ETL/ELT tools (dbt, Airflow, ADF, Informatica, Matillion). Solid understanding of data warehousing concepts (Kimball, Data Vault, normalization). Familiarity with cloud platforms (Azure preferred; AWS/ ...

objects (databases, schemas, tables, views, tasks, streams, resource monitors). ETL/ELT Pipelines: Develop and maintain robust data pipelines using tools like dbt, Airflow, Azure Data Factory, or similar. Data Modelling: Implement dimensional models (star/snowflake schemas), handle SCDs, and design efficient structures for analytics. Performance Tuning … Degree in IT Strong SQL skills (complex queries, performance tuning) and proficiency in Python for data processing. Experience with ETL/ELT tools (dbt, Airflow, ADF, Informatica, Matillion). Solid understanding of data warehousing concepts (Kimball, Data Vault, normalization). Familiarity with cloud platforms (Azure preferred; AWS/ ...

model serving platforms such as BentoML, MLflow, or SageMaker . Hands-on work with streaming and batch data pipelines using tools like Spark, Airflow, or Apache Beam . Background in front-end or mobile development , enabling end-to-end solution thinking. Prior experience in healthcare, life sciences ...

version management, model registry, canary/blue-green rollouts, and rollback plans. Data engineering: Build reliable data pipelines (batch/stream) using Spark/Airflow/Beam; ensure data quality, lineage, and governance. Technical leadership: Lead design reviews, mentor engineers, enforce coding standards, documentation, and SRE best practices. Partner … prompt/context engineering, multimodal workflows, and observability. Experience with MLOps/model serving (e.g., MLflow, SageMaker, Vertex AI) and data pipelines (e.g., Spark, Airflow, Beam). Security-first mindset and familiarity with compliance (PII handling, RBAC/IAM,key management). Nice-to-Have Experience with function/ ...

Data Factory) and modern data warehouse technologies (Databricks, Snowflake) Experience with database technologies such as RDBMS (SQL Server, Oracle) or NoSQL (MongoDB) Knowledge in Apache technologies such as Spark, Kafka and Airflow to build scalable and efficient data pipelines Ability to design, build, and deploy data solutions that ...

facing features and reporting. Continuously improve reliability, performance, deployment, and monitoring as the platform scales. What We're Looking For: Tech Stack: Backend: FastAPI, Airflow Databases: PostgreSQL, MongoDB Messaging & Streaming: Kafka Ideal Experience: 5+ years' experience building and maintaining mission-critical backend systems. Strong background in microservices-based architectures. … interested in the above role, click the 'Apply Now' button for immediate review. Senior Backend Engineer, Backend Engineer, Python, FastAPI, Kafka, Airflow, PostgreSQL, MongoDB, Microservices, Event-Driven Architecture, Cybersecurity, SaaS Circle Recruitment is acting as an Employment Agency in relation to this vacancy. Earn yourself a referral bonus ...

security. Drive modernisation by transitioning from legacy systems to a lean, scalable platform. Act as a lead expert for technologies such as AWS, DBT, Airflow, and Databricks. Establish best practices for data modelling, ingestion, storage, streaming, and APIs. Governance & Standards Ensure all technical decisions are well-justified, documented … engineering, including data ingestion, transformation, and storage. Significant hands-on experience with AWS and its data services. Expert-level skills in SQL, Python, DBT, Airflow and Redshift. Confidence in coding, scripting, configuring, versioning, debugging, testing, and deploying. Ability to guide and mentor others in technical best practices. A product ...

setting. MSc in Computer Science, Machine Learning, or a related field. Experience building data pipelines (realtime or batch) & data quality using modern toolchain (e.g., Apache Spark, Kafka, Airflow, dbt). Strong foundational knowledge of machine learning and deep learning algorithms, including deep neural networks, supervised/unsupervised learning ...

Proven experience in recommender systems, behavioural AI, and/or reinforcement learning. Building data pipelines (realtime or batch) & data quality using modern toolchain (e.g., Apache Spark, Kafka, Airflow, dbt). PhD in Computer Science, Machine Learning, or a closely related field What We Offer: Opportunity to build technology ...

Skills/Knowledge/Experience (Up to 10, Avoid repetition) Familiar with any Cloud environment and knowledge on Kubernetes Familiarity with orchestration tools like Airflow Familiarity with engineering in a regulatory controlled environment Good to Have (As Applicable) Familiar with AWS via hands on experience or certification Familiarity with … orchestration tools like Airflow Familiarity with BASEL regulatory reporting framework Familiarity with engineering in a regulatory controlled environment Managing expectations of non-technical stakeholders ...

than headcount substitution, aligning with Outside IR35 status . Key responsibilities include: Designing, building, and delivering data pipelines and orchestration workflows using Prefect or Airflow Developing Python-based services and APIs for data ingestion and enrichment Building and deploying containerised workloads using Docker and Kubernetes (or Swarm) Managing dependencies … UKSV DV clearance (minimum 12 months remaining) Strong Python development experience in secure or restricted environments Proven delivery of data pipelines using Prefect or Airflow Hands-on experience with Docker, Kubernetes , and container orchestration Experience operating in air-gapped systems , managing builds without external connectivity Strong understanding of data ...