24 of 24 Permanent Apache Airflow Jobs in Central London

optimisation and administration Strong programming experience with Python, Go and/or SQL Experience using dbt for data modelling and transformation Experience working with Apache Airflow or similar orchestration tooling Proven experience designing and building production systems on GCP and/or AWS Strong understanding of event-driven … distributed systems Experience deploying and supporting batch and real-time machine learning workloads Experience working with distributed processing technologies such as Spark or Apache Beam Strong understanding of data quality principles and engineering best practices Experience optimising database performance, troubleshooting issues and maintaining production data systems Ability to learn ...

principles. Comfortable working within Agile delivery environments. Experience with CI/CD pipelines, ideally using Azure DevOps or similar. Nice to Have Experience with Apache Airflow or dbt. Exposure to BigQuery/GCP. Azure or other cloud platform certifications Knowledge of data governance and data management frameworks Ideal ...

automated testing and data quality practice. A collaborative mindset and a genuine interest in building things well. Desirable Experience with orchestration tools such as Airflow, Dagster, Prefect or Azure Data Factory. Experience working within AWS, Azure, GCP or another enterprise cloud ecosystem. Experience with complex customer, transactional, digital ...

Computer Science, Information Systems, Engineering or a related discipline, or equivalent practical experience • Strong data engineering experience, including SQL, Python, DBT, Snowflake and Airflow • Experience building and maintaining ETL/ELT pipelines and data models • Experience with Power BI/BI and ideally semantic models and self-service analytics ...

offer the following skills & experience. Strong experience in Python, PySpark/Spark, Databricks, Azure Data Factory, SQL DB, Synapse, Stream Analytics, Glue, Airflow, Kinesis, Redshift, SonarQube, PyTest. Experience designing, building and supporting scalable data pipelines and data products in cloud environments. Experience developing and maintaining REST APIs using frameworks ...

Comfortable building from scratch in a fast-moving, product-led environment Experience with vector databases and vector search in modern AI systems Desirable: Spark, Airflow, Kafka, Elasticsearch/OpenSearch, event-driven architectures Cloud experience (AWS, GCP or Azure), GCP is a plus Why join High impact and ownership ...

times series data, they use TimescaleDB but other experience with similar platforms is fine You have experience with Orchestration platforms such as Dagster or Airflow You're collaborative with great communication skills What's in it for you: Salary to £120k + bonus X3 weeks per year work anywhere ...

Commercial Data to build scalable data products supporting AI/ML, analytics, and commercial insights. ✅ Snowflake ✅ SQL & Python ✅ Collibra ✅ IQVIA/Veeva ✅ dbt/Airflow ✅ Pharma Commercial Data 📩 Interested candidates can share their updated resume to manasa@cygnuspro.com ...

Shape the engineering approach across Google Cloud Platform (GCP) , working with technologies including BigQuery, Cloud Storage, Pub/Sub, Datastream and Cloud Composer/Airflow, alongside Terraform and infrastructure-as-code. Set and champion high engineering standards across data modelling, dbt (data build tool) transformations, testing, monitoring, documentation, deployment … BigQuery, and a good working knowledge of the wider Google Cloud Platform (GCP) ecosystem. Experience with modern data engineering tools and practices such as Airflow/Cloud Composer, dbt, Git/GitHub, CI/CD and Terraform or other infrastructure-as-code tooling . A solid understanding of data ...

product partners Preferred Qualifications, Capabilities, and Skills Experience building agentic tools and AI agents that support engineering workflows in production environments Experience with Airflow ABOUT US J.P. Morgan is a global leader in financial services, providing strategic advice and products to the world's most prominent corporations, governments, wealthy ...

will include Design, build and maintain robust, scalable data pipelines that support data science workflows and advanced analytics use cases Develop and manage complex Airflow orchestration, ensuring pipelines are reliable, observable and maintainable Work hands-on with Databricks suite to deliver high-quality data engineering solutions for data science ...

Experience using Google Cloud Platform and/or Google Cloud Engineer qualifications Experience of PySpark, BigQuery, Git and any/all of: DataFlow, DataProc, Airflow (or equivalents) We believe great things happen when teams connect. Our schedule is built around 4 days of high-impact, in-office collaboration (Monday ...

economy. Familiarity with Supabase specifically, or with similar BaaS platforms. BigQuery experience. Familiarity with n8n. Equivalent experience with code‐first orchestrators such as Temporal, Airflow or Prefect is welcome. Design sensibility. You care how things look and feel, not just how they function. Experience replacing legacy or external vendor ...

CDO. What you’ll need for this role Hands-on experience designing and operating large-scale data pipelines on GCP (BigQuery, Cloud Composer/Airflow, GCS), with proven knowledge of Apache Kafka and real-time streaming architectures Demonstrated experience leading or technically mentoring a team of data engineers ...

/Java and/or Python. Experience managing batching and/or streaming data pipelines and infrastructure using technologies such as GCP, Dataflow/Apache Beam, Dataproc, Spark/Flink, dbt; Working knowledge of SQL and shell scripting; Experience with Docker/k8s and "infrastructure as a code" tools … partners across the organization via Technical Design Documents. This is a great opportunity to further your experience across technologies such as Cloud Composer, Airflow, Dataflow, Spark, Terraform & Google Cloud Platform. Benefits Hybrid working Long contract Award winning work environment #J-18808-Ljbffr ...

integration failures, data inconsistencies, and production issues Key Skills Python & SQL (production-grade pipeline development) REST APIs, webhooks, OAuth, event-driven architecture Orchestration tools: Airflow, Prefect, or Dagster Cloud platforms: AWS, GCP, or Azure Docker & Kubernetes Microsoft 365 & Microsoft Copilot Desirable Skills Vector databases and embedding pipelines Real-time … robust, production-grade data pipelines from scratch. Deep familiarity with integration patterns: REST APIs, webhooks, OAuth, and event-driven architectures. Experience with orchestration tools (Airflow, Prefect, or Dagster) and transformation frameworks (dbt or similar). Comfortable across cloud platforms (AWS, GCP, or Azure) and with containerisation (Docker, Kubernetes). ...

days on site A London Trading firm is building a trading data platform from scratch, in house, on open source. Spark, Kafka, Polaris, Ranger, Airflow, MinIO. This is the person who builds it. Why now The analytics function is running on legacy warehouses that were never built … ingestion pipelines for both streaming and batch sources Write the Spark jobs that clean, transform and land data in the lakehouse Build and own Airflow orchestration Work on cataloguing and access control with Polaris and Ranger Help migrate data off the legacy warehouses and retire them What we need ...

driven user experience. Hybrid role with three on-site days in City of London. Responsibilities include building scalable data pipelines (BigQuery, Dataflow/Apache Beam, Airflow), ensuring data quality and observability (Looker, Monte Carlo), and collaborating with product engineering and data science teams to plan data tracking ...

value, and/or FICC options trading Experience with Linux-based, concurrent, high-throughput, low-latency software systems Experience with pipeline orchestration frameworks (e.g. Airflow, Dagster) Experience with streaming platforms (e.g. Kafka), data lake platforms (e.g. Delta Lake, Apache Iceberg), and relational databases Have a Bachelor or advanced ...

value, and/or FICC options trading Experience with Linux‐based, concurrent, high‐throughput, low‐latency software systems Experience with pipeline orchestration frameworks (e.g. Airflow, Dagster) Experience with streaming platforms (e.g. Kafka), data lake platforms (e.g. Delta Lake, Apache Iceberg), and relational databases Have a Bachelors or advanced ...

ship confidently. Key Responsibilities Own and evolve the core AI platform infrastructure supporting production ML services. Build and maintain orchestration pipelines using Dagster , Airflow or similar tooling. Deploy and manage workloads using Kubernetes and cloud-native infrastructure. Develop platform tooling and services in Python . Implement and maintain Terraform … + 10% bonus Working Model: Hybrid, 1-2 days per week onsite in Central London Tech Stack: Python, Azure, Kubernetes, Terraform, Dagster, Airflow, ML Infrastructure, LLMs, Vector Databases Visa Sponsorship: Exceptional candidates only Interested? Please apply below. ...

experienced Full Stack Data Engineer to join our Customer Data Science team. You will design, build and maintain robust data pipelines, orchestration with Airflow, and Databricks-based solutions to empower data scientists with reliable data foundations and scalable platforms. You will collaborate across product and delivery partners, contribute ...

design, API development, and data modelling for production machine learning systems. Build and operate production solutions using technologies such as Python, TensorFlow, PyTorch, Redis, Airflow, Docker, Kubernetes, and cloud-native infrastructure. Design and deploy scalable batch and real-time ML pipelines supporting customer-facing applications. Develop and operationalise … designing, building, or scaling RAG systems. Experience in search, ranking, recommendations, personalisation, content intelligence, moderation, NLP, computer vision, or related AI domains. Experience with Airflow or comparable workflow orchestration platforms. Experience with cloud-native environments and container technologies, including Docker and Kubernetes. Experience developing model evaluation frameworks ...

SENIOR SCALA ENGINEER £590 per day via Umbrella 6-12 month contract Stratford, London – hybrid remote working SC clearance required ABOUT THE CLIENT Our client is a major UK public sector organisation delivering digital services ...