51 to 75 of 197 Apache Airflow Jobs in London

stack Python and associated ML/DS libraries (scikit-learn, numpy, LightlGBM, Pandas, LangChain/LangGraph TensorFlow, etc ) PySpark AWS cloud infrastructure: EMR, ECS, Athena, etc. MLOps: Terraform, Docker, Airflow, MLFlow More information: Enjoy fantastic perks like private healthcare & dental insurance, a generous work from abroad policy, 2-for-1 share purchase plans, an EV Scheme to further reduce More ❯

through coursework, Kaggle competitions, or personal data projects You've shown initiative in teaching yourself new technical tools or concepts beyond what was required - such as exploring BigQuery, dbt, Airflow, Docker, or other data engineering technologies on your own time Progression This is an initial six-month engagement. If you perform well, the expectation is that you'll move More ❯

Product Owner or Product Owner for data/analytics platforms. Understanding of the software development lifecycle with a data-centric lens. Familiarity with data engineering workflows, data tools (dbt, Airflow), and cloud data platforms (AWS, GCP, Azure). Proven experience translating data and analytics requirements into actionable backlog items. Knowledge of regulatory and compliance frameworks (e.g., GDPR, CCPA) as More ❯

in trust metrics or customer experience analysis Knowledge of dashboard design and data visualization best practices Experience with cloud-based data infrastructure (AWS) Familiarity with modern data stack tools (Airflow, dbt, etc.) Why This Role Matters Judge.me is at an inflection point. As the market leader in Shopify reviews, we've chosen to build our future with Shopify because More ❯

such as Tableau, with a focus on optimizing underlying data structures for dashboard performance. Ingestion and orchestration tools: Skilled in using pipeline orchestration and data ingestion tools such as Airflow and Stitch, along with Python scripting for integrating diverse data sources. Large-scale data processing: Proficient with distributed query engines like AWS Athena or SparkSQL for working with datasets More ❯

systems (e.g. Git) Desirables: Experience with cloud-based ML infrastructure, particularly GCP (Vertex AI, BigQuery), or equivalent (e.g. AWS, Azure) Exposure to orchestration tools such as Kubeflow pipelines or Airflow Familiarity with DBT or similar tools for modelling data in data warehouses Desire to build interpretable and explainable ML models (using techniques such as SHAP) Desire to quantify the More ❯

systems (e.g. Git) Desirables: Experience with cloud-based ML infrastructure, particularly GCP (Vertex AI, BigQuery), or equivalent (e.g. AWS, Azure) Exposure to orchestration tools such as Kubeflow pipelines or Airflow Familiarity with DBT or similar tools for modelling data in data warehouses Desire to build interpretable and explainable ML models (using techniques such as SHAP) Desire to quantify the More ❯

Experience designing scalable ML infrastructure on cloud platforms (AWS SageMaker, GCP AI Platform, Azure ML, or equivalent). Solid understanding of data-engineering concepts: SQL/noSQL, data pipelines (Airflow, Prefect, or similar), and batch/streaming frameworks (Spark, Kafka). Leadership & Communication: Proven ability to lead cross-functional teams in ambiguous startup settings. Exceptional written and verbal communication More ❯

and evaluation through continuous monitoring and scaling. Build & Optimise AI models in Python: fine-tune state-of-the-art architectures on our in-house GPU cluster. Orchestrate Workflows with Apache Airflow: schedule, monitor, and maintain complex data and model pipelines. Engineer Cloud Services on AWS (Lambda, ECS/EKS, S3, Redshift, etc.) and automate deployments using GitHub Actions … testing, and monitoring. Startup mindset: proactive, resourceful, ambitious, driven to innovate, eager to learn, and comfortable wearing multiple hats in a fast-moving environment. Desirable: hands-on experience with Apache Airflow, AWS services (especially Redshift, S3, ECS/EKS), and IaC tools like Pulumi. Why Permutable AI? Hybrid Flexibility: Spend 2+ days/week in our Vauxhall hub. More ❯

and reliability across our platform. Working format: full-time, remote. Schedule: Monday to Friday (the working day is 8+1 hours). Responsibilities: Design, develop, and maintain data pipelines using Apache Airflow . Create and support data storage systems (Data Lakes/Data Warehouses) based on AWS (S3, Redshift, Glue, Athena, etc.). Integrate data from various sources, including … pipeline into a standalone, scalable service. What we expect from you: 4+ years of experience as a Data Engineer, including 1+ year at a Senior level. Deep knowledge of Airflow : DAGs, custom operators, and monitoring. Strong command of PostgreSQL databases; familiarity with the AWS stack (S3, Glue or Redshift, Lambda, CloudWatch) is a significant plus. Excellent SQL skills and More ❯

DATA ENGINEER - DBT/AIRFLOW/DATABRICKS 4-MONTH CONTRACT £450-550 PER DAY OUTSIDE IR35 This is an exciting opportunity for a Data Engineer to join a leading media organisation working at the forefront of data innovation. You'll play a key role in designing and building the data infrastructure that supports cutting-edge machine learning and LLM … to accelerate delivery of critical pipelines and platform improvements. THE ROLE You'll join a skilled data team to lead the build and optimisation of scalable pipelines using DBT, Airflow, and Databricks. Working alongside data scientists and ML engineers, you'll support everything from raw ingestion to curated layers powering LLMs and advanced analytics.Your responsibilities will include: Building and … maintaining production-grade ETL/ELT workflows with DBT and Airflow Collaborating with AI/ML teams to support data readiness for experimentation and inference Writing clean, modular SQL and Python code for use in Databricks Contributing to architectural decisions around pipeline scalability and performance Supporting the integration of diverse data sources into the platform Ensuring data quality, observability More ❯

pipelines to power next-gen data products in the commodities industry. Ensure data quality using the latest analytics and monitoring tools. Design and build robust pipelines with tools like Airflow and DBT. Create scalable infrastructure on Azure using technologies like Terraform. Write clean, high-quality, reusable code aligned with best practices. Drive innovation by bringing your own ideas-your … in a fast-paced startup or agile environment. Strong background in schema design and dimensional data modeling. Able to communicate data architecture clearly with internal stakeholders. Experience with Azure, Airflow, DBT, Kubernetes, GitHub. Bonus points for: open-source contributions, an active GitHub profile, and curiosity for the latest in tech. A natural problem-solver who loves making things work. More ❯

pipelines to power next-gen data products in the commodities industry. Ensure data quality using the latest analytics and monitoring tools. Design and build robust pipelines with tools like Airflow and DBT. Create scalable infrastructure on Azure using technologies like Terraform. Write clean, high-quality, reusable code aligned with best practices. Drive innovation by bringing your own ideas-your … in a fast-paced startup or agile environment. Strong background in schema design and dimensional data modeling. Able to communicate data architecture clearly with internal stakeholders. Experience with Azure, Airflow, DBT, Kubernetes, GitHub. Bonus points for: open-source contributions, an active GitHub profile, and curiosity for the latest in tech. A natural problem-solver who loves making things work. More ❯

within ambitious software businesses. Specifically, you can expect to be involved in the following: Designing and developing full-stack data pipelines and platforms using modern tools such as dbt, Airflow, and cloud infrastructure Cleansing, enriching and modelling data to generate commercial insights and power C-level dashboards Delivering scalable solutions that support internal use cases and extend directly to … sales) and building tools that serve business needs Background in startups or scale-ups with high adaptability and a hands-on approach Experience with modern data tools (e.g. dbt, Airflow, CI/CD) and at least one cloud platform (AWS, GCP, Azure) Strong communication skills and a track record of credibility in high-pressure or client-facing settings BENEFITS More ❯

Data Storage & Databases: SQL & NoSQL Databases: Experience with databases like PostgreSQL, MySQL, MongoDB, and Cassandra. Big Data Ecosystems: Hadoop, Spark, Hive, and HBase. Data Integration & ETL: Data Pipelining Tools: Apache NiFi, Apache Kafka, and Apache Flink. ETL Tools: AWS Glue, Azure Data Factory, Talend, and Apache Airflow. AI & Machine Learning: Frameworks: TensorFlow, PyTorch, Scikit-learn, Keras More ❯

Strong SQL and Python skills for building and optimising data pipelines Experience working with cloud platforms (e.g., AWS, GCP, or Azure) Familiarity with modern data stack tools (e.g., dbt, Airflow, Snowflake, Redshift, or BigQuery) Understanding of data modelling and warehousing principles Experience working with large datasets and distributed systems What's in it for you? Up to £70k Hybrid More ❯

experience with Cloud infrastructure (ideally AWS), DevOps technologies such as Docker or Terraform and CI/CD processes and tools. Have previously worked with MLOps tools like MLFlow and Airflow, or on common problems such as model and API monitoring, data drift and validation, autoscaling, access permissions Have previously worked with monitoring tools such as New Relic or Grafana … and associated ML/DS libraries (scikit-learn, numpy, pandas, LightGBM, LangChain/LangGraph, TensorFlow, etc ) PySpark AWS cloud infrastructure: EMR, ECS, ECR, Athena, etc. MLOps: Terraform, Docker, Spacelift, Airflow, MLFlow Monitoring: New Relic CI/CD: Jenkins, Github Actions More information: Enjoy fantastic perks like private healthcare & dental insurance, a generous work from abroad policy, 2-for More ❯

platform, ensuring scalability, reliability, and security. Drive modernisation by transitioning from legacy systems to a lean, scalable platform. Act as a lead expert for technologies such as AWS, DBT, Airflow, and Databricks. Establish best practices for data modelling, ingestion, storage, streaming, and APIs. Governance & Standards Ensure all technical decisions are well-justified, documented, and aligned with business needs. Lead … in data engineering and cloud engineering, including data ingestion, transformation, and storage. Significant hands-on experience with AWS and its data services. Expert-level skills in SQL, Python, DBT, Airflow and Redshift. Confidence in coding, scripting, configuring, versioning, debugging, testing, and deploying. Ability to guide and mentor others in technical best practices. A product mindset, focusing on user needs More ❯

engineers Collaborate across engineering, data science, and product teams to deliver business impact Skills & Experience: Expert in SQL , dbt , and cloud data warehouses (e.g., BigQuery, Redshift) Strong experience with Airflow , Python , and multi-cloud environments (AWS/GCP) Proven background in designing and scaling analytics solutions in agile environments Proven experience as an Analytics Engineer Nice to Have: Experience More ❯

attributes of the suitable Analytics Engineer include: A few years of experience in analytics or data engineering, with strong SQL and Python skills, and hands-on experience with dbt, Airflow, and cloud platforms (AWS, GCP, or Azure). You should be confident designing ELT/ETL pipelines and working across varied technical stacks. Strong communication skills and experience working More ❯

attributes of the suitable Analytics Engineer include: A few years of experience in analytics or data engineering, with strong SQL and Python skills, and hands-on experience with dbt, Airflow, and cloud platforms (AWS, GCP, or Azure). You should be confident designing ELT/ETL pipelines and working across varied technical stacks. Strong communication skills and experience working More ❯

oversight across the data platform, including data pipelines, orchestration and modelling. Lead the team in building and maintaining robust data pipelines, data models, and infrastructure using tools such as Airflow, AWS Redshift, DBT and Looker.Ensuring the team follows agile methodologies to improve delivery cadence and responsiveness. Contribute to hands-on coding, particularly in areas requiring architectural input, prototyping, or … mentoring skills and ability to foster team growth and development Strong understanding of the data engineering lifecycle, from ingestion to consumption Hands-on experience with our data stack (Redshift, Airflow, Python, DVT, MongoDB, AWS, Looker, Docker) Understanding of data modelling, transformation, and orchestration best practices Experience delivering both internal analytics platforms and external data-facing products Knowledge of modern More ❯

trusted, reliable and available . The technology underpinning these capabilities includes industry leading data and analytics products such as Snowflake, Tableau, DBT, Talend, Collibra, Kafka/Confluent , Astronomer/Airflow , and Kubernetes . This forms part of a longer-term strategic direction to implement Data Mesh, and with it establish shared platform s that enables a connected collection of … and driv ing a culture of iterative improvemen t. Modern data stack - hands-on deploy ment and govern ance of enterprise technologies at scale (e.g. Snowflake, Tableau, DBT, Fivetran , Airflow, AWS , GitHub, Terraform, etc ) for self-service workloads . Thought leadership and influencing - deep interest in data platforms landscape to build well-articulated proposals that are supported by strong More ❯

and meeting deadlines. Proficiency in SQL (BigQuery), Python, Git/GitHub, and preferably Looker (Tableau or PowerBI are acceptable as well) Above average knowledge of DBT, Docker, GCP, and Airflow Experience in the cryptocurrency industry, fintech sector, or platform-type businesses is preferred but not required. Personal Attributes Analytical mindset with a passion for data-driven decision-making. Strong … ambitious with a results-oriented attitude and continuous improvement mindset Technologies you will work with Python SQL (BigQuery) GCP EPPO for experimentation DBT, Docker, Cloud Run/Kubernetes, and Airflow for data orchestration and data pipelines Looker data visualization Git and GitHub for code collaboration Ability to leverage AI tools such as Cursor and LLMs in the day-to More ❯

data accessibility and quality. Key Skills & Experience Required – About You: Essential: Strong proficiency in Python, SQL, and Jinja Experience with cloud platforms (preferably AWS) Familiarity with orchestration tools (ideally Airflow) Experience building CI/CD workflows (ideally using GitHub Actions) Up-to-date knowledge of modern data engineering practices Proven ability to design and develop scalable data pipelines Strong More ❯