151 to 175 of 281 Apache Airflow Jobs in London

organisation Deep understanding & hands‐on leverage of Modern, Cloud scale Data Technologies, with specific expertise in Distributed Data Processing architectures (Spark, PySpark, Python, Airflow), ELT data processing underpinning Data LakeHouses and Data Products for scaled use within ML & AI Expertise in Streaming Data, Event Driven Design, Data Governance ...

Shape the engineering approach across Google Cloud Platform (GCP), working with technologies including BigQuery, Cloud Storage, Pub/Sub, Datastream and Cloud Composer/Airflow, alongside Terraform and infrastructure-as-code. Set and champion high engineering standards across data modelling, dbt (data build tool) transformations, testing, monitoring, documentation, deployment … BigQuery, and a good working knowledge of the wider Google Cloud Platform (GCP) ecosystem. Experience with modern data engineering tools and practices such as Airflow/Cloud Composer, dbt, Git/GitHub, CI/CD and Terraform or other infrastructure‐as‐code tooling . A solid understanding of data ...

will be used across the organisation, including taking ownership of our data transformation and orchestration tooling; batch and streaming infrastructure and exploration tools (Databricks, Airflow, dbt) and look after our Datalake (ingestion, storage, governance, privacy). You’ll work with a modern, cutting‐edge data stack and play … analytics engineers and have a strong grasp of their needs and how they operate Big data technologies, with expertise in tools & platforms such as Airflow, dbt, Kafka, Databricks and data observability & catalogue) solutions (e.g. Monte Carlo, Atlan, Datahub) Cloud Platform proficiency: familiarity with AWS, GCP, or Microsoft Azure, with ...

understanding of data management and data quality principles, with experience developing data quality checks or reports Prior experience using the Bloomberg Terminal Experience with Apache Airflow or similar tools For more information about DRW's processing activities and our use of job applicants' data, please view our Privacy ...

business intelligence in a security-conscious environment. The role requires 3+ years in data engineering, strong experience with Spark, Python, Kafka, SQL and Airflow, and a willingness to work hybrid in London or Cheltenham. You will contribute to client engagements and growth of the company. #J-18808-Ljbffr ...

5+ years of data engineering experience with Spark, strong Python and SQL, extensive AWS services, and hands-on experience with orchestration tools (Docker, Kubernetes, Airflow, Prefect, Dagster). Hybrid work model in London. #J-18808-Ljbffr ...

intensive project with a globally leading AI lab. You will build and own production data pipelines using SQL and Python, with orchestration tools like Airflow and streaming frameworks such as Spark/Kafka. The role involves collaboration across teams and potential documentation tasks to structure work for frontier ...

development experience, with strong skills in Python. Hands‐on experience with building and deploying data pipelines Familiarity with modern infrastructure (CI/CD, Kafka, Airflow) Excellent communication and collaboration skills Detail orientated, with a commitment to best engineering practices The ability to prioritise, plan and deliver to projects ...

development experience, with strong skills in Python. Hands-on experience with building and deploying data pipelines Familiarity with modern infrastructure (CI/CD, Kafka, Airflow) Excellent communication and collaboration skills Detail orientated, with a commitment to best engineering practices The ability to prioritise, plan and deliver to projects ...

containers and production monitoring. Clear communication and an interest in how global events affect financial markets. Experience with multi-agent workflows, model fine-tuning, Airflow, AWS, GitHub Actions, Pulumi or GPU workloads would be useful. We care more about what you have built and operated than whether you have ...

similar databases. Experience designing data models, schemas and transformation layers for complex datasets. Experience with directed acyclic graph–based orchestration tools such as Airflow, Dagster, Prefect or similar systems. Familiarity with data warehouses, object storage, queues, APIs and cloud infrastructure. Experience with data quality testing, lineage, observability, versioning ...

product partners Preferred Qualifications, Capabilities, and Skills Experience building agentic tools and AI agents that support engineering workflows in production environments Experience with Airflow ABOUT US J.P. Morgan is a global leader in financial services, providing strategic advice and products to the world's most prominent corporations, governments, wealthy ...

product partners Preferred Qualifications, Capabilities, and Skills Experience building agentic tools and AI agents that support engineering workflows in production environments Experience with Airflow ABOUT US Our client is a global leader in financial services, providing strategic advice and products to the world's most prominent corporations, governments, wealthy ...

product partners Preferred Qualifications, Capabilities, and Skills Experience building agentic tools and AI agents that support engineering workflows in production environments Experience with Airflow ABOUT US Our client is a global leader in financial services, providing strategic advice and products to the world's most prominent corporations, governments, wealthy ...

taking an AI‐first approach to analytical problems. Data governance frameworks, marketing analytics and attribution. Experience with tools and technologies such as dbt, Airflow, Git, Google Cloud Storage (GCS), BigQuery, or ClickHouse. Familiarity with data governance and cataloguing tools such as Atlan, Alation, or OpenMetadata. What we offer Real ...

APIs can be exposed to LLM-based agents. Experience with search relevance & recommendation systems — ranking algorithms, semantic search, personalisation. Hands-on with Snowflake, Databricks, Airflow, Kafka . Why Join Us Work on firm-wide platforms used by thousands of data scientists, ML engineers, and analysts across JPMC Shape ...

ability to initiate and drive projects proactively Strong AWS knowledge, including Kinesis, Athena, S3, etc. Extensive data warehousing experience, Snowflake, Redshift etc. Experience with Airflow or a similar ETL tool, Python experience and data modelling knowledge Enjoy working with other engineers and stakeholders Our Benefits: The flexibility ...

forecasting, prioritisation and scenario modelling. You’re proficient in SQL and know your way around data pipeline management tools such as Snowflake, dbt, Git, Airflow, Python and Lightdash. You’re comfortable owning analytics from data transformation through to actionable insights. You’re an expert at crafting dashboards and visualisations ...

functionally with Product/Design/Engineering and bring teams along on decisions. To share extra context, our current tech stack includes GCP, BigQuery, Airflow, Python, SQL on the Data side and AWS, Kotlin, Javascript, Typescript on the Product side while our infrastructure is containerised with Kubernetes. Experience across ...

with expertise in Python and experience building reusable tooling and platform services. Experience with modern platform and engineering tooling such as GitHub Actions, Apigee, Airflow, and related cloud‐native technologies. Expertise with observability platforms such as Data Dog, Prometheus, Grafana, ELK, Splunk or equivalent, including monitoring, logging, tracing, reliability ...

field Production ML engineering excellence – Deep understanding of ML operations, including containerization (Docker, Kubernetes/AWS EKS), cloud platforms (AWS/Azure), workflow orchestration (Airflow), automated testing for ML systems, and API development for model deployment Technical leadership and influence – Track record of mentoring and coaching engineers, driving ...

field Production ML engineering excellence – Deep understanding of ML operations, including containerization (Docker, Kubernetes/AWS EKS), cloud platforms (AWS/Azure), workflow orchestration (Airflow), automated testing for ML systems, and API development for model deployment Technical leadership and influence – Track record of mentoring and coaching engineers, driving ...

cloud‐native data architecture (AWS/GCP/Snowflake). Architect and implement scalable ingestion and transformation pipelines using Fivetran, Databricks, and Airflow/DBT. Design and code modular data services and APIs that unify campaign, conversion, and audience datasets across 20+ marketing platforms. Build and optimize schemas, transformations … Hands‐on experience as a Data Engineer or Data Platform Developer, including direct contributions to production codebases. Expert‐level proficiency in Python, SQL, and Apache Spark for data pipeline development. Proven experience building ELT workflows using Fivetran, Airflow, DBT, or Databricks. Solid understanding of data modeling, dimensional design ...

services, a good understanding of cloud architecture, and proficiency with Terraform for provisioning and managing cloud infrastructure Experience building streaming data pipelines using Apache Flink, Apache Kafka, Spark Structured Streaming, or similar technologies Knowledge of modern data lakehouse architectures and ecosystem such as Kafka, Flink, Spark, Databricks, Snowflake … Airflow, Debezium, Delta/Iceberg, and comfort with Python/Java and SQL Experience with networking and security concepts within AWS, including VPCs, subnets, routing, security groups, IAM, etc. Nice to have Experience: Exposure to or interest in the cryptocurrency technology ecosystem Experience working with different blockchain technologies ...

database design principles Experience with Microsoft Azure and Azure Data Lake Storage (ADLS) Experience with data engineering tools such as Azure Data Factory, Databricks, Apache Spark, dbt or Airflow Experience with cloud data platforms such as Azure Synapse Analytics or Snowflake Experience using Git for version control Experience ...