51 to 72 of 72 Apache Airflow Jobs in London

track record of deploying AI into production at scale. Technical Skills: Strong backend Python; expertise with ML frameworks (TensorFlow, PyTorch, Scikit-Learn, etc.); familiarity with modern data pipeline tools (Airflow, Spark, Kafka) and workflow orchestration frameworks such as n8n or LangGraph. Applied AI: Background in enterprise applications such as chatbots, workflow automation, RAG pipelines, or intelligent document processing. Domain More ❯

track record of deploying AI into production at scale. Technical Skills: Strong backend Python; expertise with ML frameworks (TensorFlow, PyTorch, Scikit-Learn, etc.); familiarity with modern data pipeline tools (Airflow, Spark, Kafka) and workflow orchestration frameworks such as n8n or LangGraph. Applied AI: Background in enterprise applications such as chatbots, workflow automation, RAG pipelines, or intelligent document processing. Domain More ❯

and leadership with collaborators across Marketing, Product, and Engineering, including influencing senior and executive partners. Nice to have: Experience with marketing tech stacks, event-level data, Spark/Databricks, Airflow, generative AI for CRM applications, or two-sided marketplace environments. Please see the Talent Privacy Notice for information regarding how eBay handles your personal data collected when you use More ❯

Infrastructure automation: develop CI/CD pipelines for research processes and live trading apps develop robust monitoring solutions for infrastructure and deployed applications automate recurring jobs with tools like Airflow/Prefect Performance engineering: Be familiar with best practices for profiling, monitoring performance to assist with performance investigations Develop solutions that empower researchers and developers to understand the performance More ❯

early-stage B2B SaaS (ideally cybersecurity, threat intelligence, or offensive security tools). Deep technical expertise across: Cloud: AWS, Azure IaC: Terraform, Ansible Platforms: Kubernetes (EKS), ECS, Docker, Lambda, Airflow Observability: EFK, Prometheus, Grafana, Jaeger/OpenTelemetry Persistence: MySQL, PostgreSQL, MongoDB, Redis, Elasticsearch, Kafka Pipelines: GitHub Actions, AWS CodeBuild Runtimes: Python, Node.js, Golang Strong networking fundamentals, especially OSI layer More ❯

early-stage B2B SaaS (ideally cybersecurity, threat intelligence, or offensive security tools). Deep technical expertise across: Cloud: AWS, Azure IaC: Terraform, Ansible Platforms: Kubernetes (EKS), ECS, Docker, Lambda, Airflow Observability: EFK, Prometheus, Grafana, Jaeger/OpenTelemetry Persistence: MySQL, PostgreSQL, MongoDB, Redis, Elasticsearch, Kafka Pipelines: GitHub Actions, AWS CodeBuild Runtimes: Python, Node.js, Golang Strong networking fundamentals, especially OSI layer More ❯

libraries, and container environments for production readiness. Deployment & Delivery Oversight Provide technical leadership across the full deployment life cycle. Partner with datacenter operations to ensure correct rack layouts, cabling, airflow and power design. Support delivery teams during build-out phases, ensuring the design is executed correctly. Participate in factory acceptance tests (FAT), site acceptance tests (SAT), and operational readiness More ❯

data infrastructure for the organisation. What you'll be doing: Design, develop, and maintain scalable data architectures and ETL pipelines Build and manage data models and data warehouse solutions (Airflow, dbt, and Redshift) Write clean, efficient Python and SQL code for data processing and transformation Integrate data from internal and third-party APIs and services Optimise data pipelines for … data security, privacy, and compliance principles Ability to optimise data pipelines for performance and maintainability Strong collaboration skills and a proactive, problem-solving mindset Bonus Points For Experience with Airflow and/or dbt Experience working in Agile environments (Scrum/Kanban) Exposure to DevOps practices or CI/CD pipelines You'll join a business that values innovation More ❯

London office. About you : You have strong Python backend software engineer skills You have experience working with large data sets You have experience of using PySpark and ideally also Apache Spark You believe in automating wherever possible You're a collaborative problem solver with great communication skills Other technology in the stack includes: FastAPI, Django, Airflow, Kafka, ETL More ❯

when needed). The Role Build and optimise scalable data pipelines for a new Data Lake Work with Python, SQL, Spark, AWS, Docker, CI/CD Orchestrate workflows using Airflow, Prefect, or Dagster Support ETL, API integrations, and high-quality data validation Translate business needs into technical solutions with full ownership What You Need STEM degree (Computer Science, Engineering … etc.) 2+ years in data engineering/analytics/software Strong Python + SQL 2+ years ETL, APIs, CI/CD, Docker, AWS Ideally but not essential experience with Airflow/Prefect/Dagster Background in Financial Services, FinTech, Insurance, PE/VC, or Banking Package & Benefits High performance bonus 25 days holiday Pension & private healthcare Join at inception More ❯

excellent career progression opportunities for the right individual. You will play a pivotal role in evolving the company's data stack, working with modern tools such as DBT and Airflow, and developing scalable, high-quality data pipelines and infrastructure. Key Responsibilities: Lead the evolution of the company's data stack, with a focus on automation and scalability. Design, build … SQL and Python. Proven experience in managing complex data estates and developing automated data pipelines. Good understanding of testing principles and deployment processes. Experience with technologies such as Redshift, Airflow, DBT, and AWS. Strong grasp of data modelling, database architecture, and governance principles. Hands-on experience with Git and modern development workflows. Excellent problem-solving skills, resilience, and a More ❯

self-organising, fast-paced environment. Nice-to-Have Skills: Experience with GCP/BigQuery or other cloud data warehouses (e.g., Snowflake, Redshift). Familiarity with data orchestration tools (e.g., Airflow). Experience with data visualisation platforms (such as Preset.io/Apache Superset or other). Exposure to CI/CD pipelines, ideally using GitLab CI. Background working with More ❯

self-organising, fast-paced environment. Nice-to-Have Skills: ·Experience with GCP/BigQuery or other cloud data warehouses (e.g., Snowflake, Redshift). ·Familiarity with data orchestration tools (e.g., Airflow). ·Experience with data visualisation platforms (such as Preset.io/Apache Superset or other). ·Exposure to CI/CD pipelines, ideally using GitLab CI. ·Background working with More ❯

a scaling startup in the AI and Healthcare technology sector Salary up to £150k + Equity Paddington-based (4-5 days per week in-office) Tech environment: Python | SQL | Airflow | dbt | GCP | AWS | Data Architecture | Kafka If you wish to keep your CV/data private, feel free to WhatsApp your details/CV to me, Dan - WHO WE … Senior or Lead Data Engineer , Data Manager, or similar leadership role. Strong proficiency in Python (or Scala/Java) and SQL . Hands-on experience with data orchestration tools (Airflow, dbt, Dagster, or Prefect). Solid understanding of cloud data platforms (AWS, GCP, or Azure) and data warehousing (Snowflake, BigQuery, Redshift). Experience with streaming technologies (Kafka, Kinesis, or … are giving express consent for us to process (subject to required skills) your application to our client in conjunction with this vacancy only. KEY SKILLS: Data Management | Python | SQL | Airflow | dbt | GCP | AWS | Data Architecture | Kafka | Governance | Data Quality | Cloud Data Platforms More ❯

be doing: Managing and maintaining a Python-based data warehouse , ensuring reliable data ingestion and transformation. Developing and monitoring data pipelines using Meltano and orchestration tools like Dagster or Airflow . Designing and optimising SQL transformations (DBT experience a plus). Working within Azure cloud infrastructure following Infrastructure-as-Code (IaC) principles. Managing Power BI semantic models , DAX measures … data stack for ETL/ELT processes and data warehousing. Strong SQL and Python skills, with an understanding of Kimball-style data modelling . Experience with DBT, Dagster, or Airflow for transformation and orchestration. Hands-on experience with Azure (preferred), or other cloud environments (GCP/AWS). Familiarity with Git-based version control , CI/CD, and DevOps More ❯

cross-functional teams to build scalable data pipelines and contribute to digital transformation initiatives across government departments. Key Responsibilities Design, develop and maintain robust data pipelines using PostgreSQL and Airflow or Apache Spark Collaborate with frontend/backend developers using Node.js or React Implement best practices in data modelling, ETL processes and performance optimisation Contribute to containerised deployments … essential Operate within Agile teams and support DevOps practices What We're Looking For Proven experience as a Data Engineer in complex environments Strong proficiency in PostgreSQL and either Airflow or Spark Solid understanding of Node.js or React for integration and tooling Familiarity with containerisation technologies (Docker/Kubernetes) is a plus Excellent communication and stakeholder engagement skills Experience More ❯

contributing to how data is used to drive commercial decisions - particularly around pricing, revenue, and customer insight. Key responsibilities include: Manage and maintain the company's data warehouse (Python, Airflow, DBT, Kimball) Ensure data pipelines are robust, accurate, and performant Maintain and develop cloud infrastructure using Infrastructure as Code (Terraform) Identify opportunities to improve data processes, architecture, and efficiency … and data during the ongoing merger YOUR SKILLS AND EXPERIENCE: A successful Data Engineer will bring: Strong SQL and Python skills Experience managing or building data warehouses Familiarity with Airflow and modern data engineering workflows Interest in cloud infrastructure and IaC principles Proactive mindset - not just maintaining systems, but improving them THE BENEFITS: You will receive a salary of More ❯

customers, by leveraging data to improve the OTM users and agents experience. This role would be suited to an individual with a background in Python, SQL, AWS, Looker, PyTorch, Airflow and Redshift/Postgres. The candidate must be able to exhibit proficiency in working with large volumes of data, automating data loads, and the ability to understand end user … requirements with the goal of building a solution to provide high quality data processing and analysis as well as building APIs. RESPONSIBILITIES D evelop data pipelines using Python and Airflow Data warehousing (Redshift/Spectrum) Develop data solutions to address business needs Maintain existing reports and improve their performance QUALIFICATIONS Bachelors degree in Computer Science, Data Science, Mathematics, Engineering … warehouses such as Amazon Redshift, and Google BigQuery. Building visualizations using tools such as Looker studio or equivalent Experience in designing ETL/ELT solutions, preferably using tools like Airflow or AWS Glue Solid analytical and technical skills Good communications skills both verbally and written; this role will be working closely with business users. NICE TO HAVE Development & Deployment More ❯

About JustPark JustPark is the premier partner offering both B2B solutions for destinations and B2C services for drivers, giving us the best of both worlds. We simplify the entire parking experience. From venues and councils to private driveways, our platform More ❯

streamline data workflows, optimize performance, and ensure reliability across cloud platforms-while guiding teams with strong technical leadership.Key Responsibilities* Pipeline Automation: Build and orchestrate data workflows using tools like Airflow, Prefect, or Dagster.* Containerization: Package and deploy data applications using Docker and Kubernetes (including EKS and AKS).* CI/CD for Data: Implement and maintain automated pipelines for … Use Terraform and Ansible to provision and manage data infrastructure.* Performance Optimization: Enhance data processing for speed, scalability, and reliability.What We're Looking For* Strong experience with orchestration tools (Airflow, Prefect, Dagster).* Expertise in Docker and Kubernetes.* Solid understanding of CI/CD principles and tooling.* Familiarity with open-source data technologies (Spark, Kafka, PostgreSQL).* Knowledge of … your application with you before presenting it to any potential employer.Please note we are on the client's supplier list for this position.KeywordsLead DataOps Engineer, DataOps, Data Pipeline Automation, Airflow, Prefect, Dagster, Docker, Kubernetes, EKS, AKS, CI/CD, Terraform, Ansible, Grafana, Prometheus, Spark, Kafka, PostgreSQL, Infrastructure as Code, Cloud Data Engineering, Hybrid Working, Security Clearance, Leadership, DevOps, Observability More ❯

run seamlessly within an AWS environment. Day to day, you'll: Oversee end-to-end data operations, ensuring all loads complete successfully and identifying and resolving workflow issues within Airflow Manage and triage incident tickets, investigating discrepancies and implementing fixes to maintain data accuracy and performance Drive automation initiatives, helping to reduce manual intervention and improve system reliability Collaborate … continuous improvement within a large-scale data transformation programme Your Skills and Experience 5+ years' experience across Data Engineering, BI or Data Operations Strong hands-on experience with AWS , Airflow , and Power BI Solid understanding of data pipelines, ETL, and debugging within cloud environments Strong problem-solving and stakeholder management skills Experience managing or working with offshore teams A More ❯

to grow your career. The Role * Build and lead the development of self serve dashboards to support commercial and customer decision making* Lead workflow automation and integration using Databricks, Airflow and Power Platform* Support data enablement and data architecture across the business* Own customer data tools that help understand customer behaviour and improve experience* Work cross functionally with marketing … Drive the data and automation roadmap while promoting best practice across the function Skills and Experience * Strong stakeholder and team leadership experience* Power BI and SQL expertise* Databricks and Airflow experience* Experience building data products and workflow automation* Python would be beneficial* Ability to influence and work across customer, commercial and data teams Salary and Benefits £70,000 to More ❯