76 to 100 of 171 Apache Airflow Jobs in England

join them 5+ years’ experience in Data Engineering Experience designing scalable data pipelines for machine learning workloads Experience implementing orchestration tools such as Apache Airflow or equivalent. Knowledge of data modelling, dimensional modelling and data warehousing principles. Whether you’re developing new products, building scalable platforms or unlocking ...

both Terraform and Terragrunt deploying data platforms (preferably Databricks Experience designing scalable data pipelines for machine learning workloads Experience implementing orchestration tools such as Apache Airflow or equivalent. Knowledge of data modelling, dimensional modelling and data warehousing principles Whether you're developing new products, building scalable platforms ...

technical trade‐offs, backed by a previous background as an experienced software engineer. Experience with data platform and ETL/orchestration technologies such as Apache Airflow, Spark, AWS Glue, and Snowflake. Experience with or strong advocacy for AI developer tooling such as Claude Code, Copilot, Windsurf, and Gemini. ...

such as customer lifetime value, attribution modelling, real-time bidding, and audience targeting is a plus Familiarity with data pipeline orchestration tools such as Apache Airflow and Kubernetes is a plus You do your best work as part of a team and enjoy sharing knowledge just as much ...

Strong hands‐on background with modern cloud data platforms (AWS/GCP/Azure) Expertise across Python, SQL and data orchestration tools such as Airflow Experience building scalable data pipelines and data platforms in a fast‐paced fintech or technology environment A collaborative leadership style with the ability ...

Python Experience with AWS and modern data technologies Familiarity with version control systems such as Git Experience with orchestration tools such as Airflow Ability to troubleshoot and improve data platform performance Strong attention to detail and focus on data quality Excellent communication and collaboration skills Willingness to learn ...

learning services. Experience with machine learning frameworks such as TensorFlow, PyTorch, or scikit-learn. Knowledge of pipeline orchestration and data workflow tools such as Airflow, Dataform, or dbt. Experience delivering advanced measurement and modelling solutions, including marketing mix modelling, incrementality testing, causal impact analysis, attribution modelling, propensity modelling, churn ...

implementing batch and stream data processing systems. Experience with Datadog, SQL, Terraform, Docker, AWS (ECS, Kinesis, SQS, CloudWatch), GCP (BigQuery). Knowledge of Airflow, Kafka, Flink, dbt, Kubernetes is highly beneficial. Good understanding of AI platforms, experience building infrastructure for LLM orchestration (e.g., vector databases, RAG pipelines, GPU scaling ...

APIs can be exposed to LLM‐based agents. Experience with search relevance & recommendation systems — ranking algorithms, semantic search, personalisation. Hands‐on with Snowflake, Databricks, Airflow, Kafka . Why Join Us Work on firm‐wide platforms used by thousands of data scientists, ML engineers, and analysts across JPMC. Shape ...

when it isn’t. The ability to communicate model behaviour and limitations clearly to non‐technical stakeholders. Familiarity with data pipeline tooling (e.g. Airflow, dbt, Spark) and an understanding of how ML fits within a broader data platform. There’s always room to grow and learn with our roles ...

operating large‐scale data transfer pipelines at multi‐TB or PB scale. Experience orchestrating parallelised transfer jobs: Flyte preferred; other workflow orchestration tools (e.g. Airflow, Prefect) will be considered. Experience building reliable, observable data systems , including retry strategies, backfills, data integrity checks, lifecycle/state tracking and operational alerting. ...

forecasting, and statistical modeling in a product environment. Expertise in Python or R, fluency in SQL, and experience with large‐scale data tech (Databricks, Airflow, dBT a plus). Experience with FinTech, payments, crypto, or blockchain data is a strong plus. Advanced degree (MS, PhD) in a quantitative field ...

organisation* Deep understanding & hands-on leverage of Modern, Cloud scale Data Technologies, with specific expertise in Distributed Data Processing architectures (Spark, PySpark, Python, Airflow), ELT data processing underpinning Data LakeHouses and Data Products for scaled use within ML & AI* Expertise in Streaming Data, Event Driven Design, Data Governance ...

organisation Deep understanding & hands-on leverage of Modern, Cloud scale Data Technologies, with specific expertise in Distributed Data Processing architectures (Spark, PySpark, Python, Airflow), ELT data processing underpinning Data LakeHouses and Data Products for scaled use within ML & AI Expertise in Streaming Data, Event Driven Design, Data Governance ...

organisation Deep understanding & hands-on leverage of Modern, Cloud scale Data Technologies, with specific expertise in Distributed Data Processing architectures (Spark, PySpark, Python, Airflow), ELT data processing underpinning Data LakeHouses and Data Products for scaled use within ML & AI Expertise in Streaming Data, Event Driven Design, Data Governance ...

/CD, testing frameworks, type-hinting, code reviews); and can build scalable, maintainable solutions Nice-to-have skills: Experience with DevOps tools (Airflow, Kubernetes, Terraform, GCP), data integration/ETL and pipeline optimization, or reinforcement learning algorithms Customer collaborator: Comfortable working directly with clients and cross-functional teams, aligning ...

well-governed data products. Job responsibilities Design and implement batch and streaming data pipelines using Databricks (Spark), Delta Lake , and orchestrators (e.g., Workflows , Airflow, ADF). Develop and optimize Spark jobs (PySpark/Scala) and SQL transformations for performance, reliability, and cost efficiency. Build and maintain curated data models … automation capabilities, to improve the value realized by automation Required qualifications, skills and capabilities Experience with building data pipelines on Databricks and/or Apache Spark in production. Strong coding skills in Python (PySpark) and SQL (Scala a plus). Hands-on experience with Delta Lake (MERGE/UPSERT ...

well-governed data products. Job responsibilities Design and implement batch and streaming data pipelines using Databricks (Spark), Delta Lake , and orchestrators (e.g., Workflows , Airflow, ADF). Develop and optimize Spark jobs (PySpark/Scala) and SQL transformations for performance, reliability, and cost efficiency. Build and maintain curated data models … automation capabilities, to improve the value realized by automation Required qualifications, skills and capabilities Experience with building data pipelines on Databricks and/or Apache Spark in production. Strong coding skills in Python (PySpark) and SQL (Scala a plus). Hands-on experience with Delta Lake (MERGE/UPSERT ...

demonstrable experience designing, deploying, and operating big data platforms at scale. You will have hands-on experience with OSS data ecosystems (e.g., Spark, Trino, Airflow, Apache Superset, and similar technologies). You will have knowledge of cloud platforms and traditional on-premise hosting; hands-on experience with Azure ...

3+ years experience working in version control environments using platforms such as Gitlab or Github 2+ years experience working with job orchestration tools like Apache Airflow, both as a pipeline developer and reviewer 2+ years experience with data ingestion and transformation frameworks (e.g., dbt, Fivetran, or custom/ ...

development experience, with strong skills in Python. Hands‐on experience with building and deploying data pipelines Familiarity with modern infrastructure (CI/CD, Kafka, Airflow) Excellent communication and collaboration skills Detail orientated, with a commitment to best engineering practices The ability to prioritise, plan and deliver to projects ...

Experience Data Modeling Data Integration & Ingestion Data Manipulation and Processing GitHub, GitHub Actions, Azure DevOps Data Factory, Databricks, SQL DB, Synapse, Stream Analytics, Glue, Airflow, Kinesis, Redshift, SonarQube, PyTest Key Characteristics Technology expert who constantly pursues knowledge enhancement and has an inherent curiosity to understand work from multiple dimensions. ...

will shape what modern data infrastructure looks like in this space. You'll take full ownership of the backend, building custom data pipelines, orchestrating Airflow style workflows, and turning messy real world inputs into structured data ready for downstream AI models. You'll design APIs, optimise low latency queries ...

building data pipelines Proficient in SQL, Python, AWS and other data management technologies Familiarity with version control systems like Git and orchestration tools like Airflow Ability to test and troubleshoot data platforms reliably. A track record of improving data reliability and quality Excellent communication skills and the ability ...

Backend Engineering background Experience building APIs and system integrations Experience working within complex, data-intensive environments Strong understanding of software engineering best practices Desirable Airflow Snowflake AWS Exposure to data warehousing and modern analytics platforms Experience supporting reporting, commercial analytics, or performance data solutions IDEAL BACKGROUND We're especially ...