101 to 125 of 149 Apache Airflow Jobs in the UK excluding London

product partners Preferred Qualifications, Capabilities, and Skills Experience building agentic tools and AI agents that support engineering workflows in production environments Experience with Airflow ABOUT US J.P. Morgan is a global leader in financial services, providing strategic advice and products to the world's most prominent corporations, governments, wealthy ...

Batch, ECS, EKS, S3, RDS or Postgres, Athena, SageMaker, Bedrock, or equivalent services. Scientific workflow and data orchestration frameworks such as Nextflow, Dagster, Airflow or similar. Platform security, privacy or governance controls for sensitive biomedical or clinical data. Identity platforms and federation technologies such as Entra ...

data storytelling. A Data Architect: You have a high bar for data quality and have experience working within modern data stacks (e.g BigQuery, dbt, Airflow) What’s In It For You A range of flexible working options to dedicate time to what matters to you Competitive compensation package + ...

Batch, ECS, EKS, S3, RDS or Postgres, Athena, SageMaker, Bedrock, or equivalent services. Scientific workflow and data orchestration frameworks such as Nextflow, Dagster, Airflow or similar. Platform security, privacy or governance controls for sensitive biomedical or clinical data. Identity platforms and federation technologies such as Entra ...

Hybrid : London or Manchester - 2 days per week. This role requires the ability to hold an active security clearance. Skills : Python and SQL . Apache Airflow for DAG orchestration and monitoring. AWS data services: Redshift , OpenSearch , Lambda , Glue , Step Functions , Batch . CI/CD pipelines and YAML ...

Data Engineer - FinTech - Newcastle (Tech stack: Data Engineer, SQL, Python, AWS, Git, Airflow, Data Pipelines, Data Platforms, Programmer, Developer, Architect, Data Engineer) Our client is a trailblazer in the FinTech space, known for delivering innovative technology solutions to global financial markets. They are expanding their engineering capability in Newcastle ...

diligence work. Experience with MLOps or model lifecycle tooling such as MLflow, model registries, or feature stores. Experience with workflow orchestration tools such as Airflow, dbt, or Databricks Workflows. Experience with data product or data mesh operating models. Knowledge of European banking or EU regulatory frameworks. Experience with Teradata ...

creating stakeholder‐ready visualizations and narratives Ability to connect insights to business outcomes and influence strategy Familiarity with modern data platforms (e.g., Spark, Snowflake, Airflow) and ML Ops tooling Passion for fostering inclusive team environments and valuing diverse perspectives Clear communicator who can translate complex technical concepts into actionable ...

schemas to support business requirements Develop and maintain data ingestion and processing systems using various tools and technologies, such as SQL, NoSQL, ETL, Luigi, Airflow, Argo, etc. Implement data storage solutions using different types of databases, such as relational, non-relational, or cloud-based. Working collaboratively with the client …/Azure SQL, PostgreSQL) You have framework experience within either Flask, Tornado or Django, Docker Experience working with ETL pipelines is desirable e.g. Luigi, Airflow or Argo Experience with big data technologies, such as Apache Spark, Hadoop, Kafka, etc. Data acquisition and development of data sets and improving ...

data processing. Proficiency in SQL, Python, and PySpark, with experience in query optimization and performance tuning. Hands-on experience with data lake platforms (Databricks, Apache Spark, or similar). Experience with AWS cloud services (S3, ECS, SNS/SQS, Lambda, etc.). Strong understanding of data quality, security … with business and technical stakeholders. Preferred qualifications, capabilities, and skills Knowledge of regulatory reporting and financial data aggregation techniques. Experience with data orchestration tools (Airflow, Step Functions, etc.). Databricks or AWS certifications. Demonstrated proficiency in cloud-native development (e.g., cloud, artificial intelligence, machine learning). Experience with Parquet ...

data processing. Proficiency in SQL, Python, and PySpark, with experience in query optimization and performance tuning. Hands-on experience with data lake platforms (Databricks, Apache Spark, or similar). Experience with AWS cloud services (S3, ECS, SNS/SQS, Lambda, etc.). Strong understanding of data quality, security … with business and technical stakeholders. Preferred qualifications, capabilities, and skills Knowledge of regulatory reporting and financial data aggregation techniques. Experience with data orchestration tools (Airflow, Step Functions, etc.). Databricks or AWS certifications. Demonstrated proficiency in cloud-native development (e.g., cloud, artificial intelligence, machine learning). Experience with Parquet ...

stakeholder communication skills Desirable: Experience overseeing or supporting data ingestion pipelines, including APIs and event-driven data sources Familiarity with orchestration tools such as Airflow and modern ELT architectures Experience governing or implementing data CI/CD pipelines (e.g. dbt tests, deployment pipelines, automated checks). We currently … design input, reviews, and mentoring Influence how analytics data is built, trusted, and operated at scale Work with a modern stack: AWS, Snowflake, dbt, Airflow, SQL, and Python Join us as we redefine travel experiences and create memories for millions of passengers. At Jet2.com and Jet2holidays, your potential ...

devices and platforms to surface key metrics. The role emphasizes coding in Python, strong SQL, and experience with large data ecosystems (HDFS, Hive, Spark, Airflow). You will influence architecture and own initiatives from concept to delivery. Hybrid work from Cambridge Office. #J-18808-Ljbffr ...

design, build, and maintain data pipelines and infrastructure to support secure, reliable, and scalable public sector services. You will work with AWS, Terraform, Spark, Airflow, Docker, and GitLab CI/CD within a hybrid model, contributing to secure coding practices and collaborating across cross-functional teams. #J-18808-Ljbffr ...

platform and expanding their engineering capability. The Role Own and develop the MLOps infrastructure supporting AI services. Build and maintain orchestration pipelines using Dagster, Airflow, or Prefect. Deploy and manage ML workloads on Kubernetes and cloud platforms. Implement CI/CD, observability, and monitoring across production systems. Develop infrastructure ...

given real ownership, exposure to experienced director level mentors & Senior Data Engineers, alongside the opportunity to grow towards senior-level responsibilities. Tech StackSpark | Airflow | Kubernetes | Azure | CubeJSWhat You'll Be DoingData Pipeline Development – designing & developing efficient, scalable & robust Data PipelinesSupport and main the data platform to ensure security, reliability ...

ideally within a large‐scale or regulated environment. AWS Experience: Strong working knowledge of AWS and cloud‐first data pipeline orchestration tools such as Airflow, Glue, or EMR. A Security Mindset: Understanding of compliance and security when managing sensitive data pipelines. Strong Automation Skills: Comfortable using Python and Shell ...

large‐scale or regulated environment. AWS Experience: You have a strong working knowledge of AWS and cloud‐first data pipeline orchestration tools (like Airflow, Glue, or EMR). A Security Mindset: You understand the importance of compliance and security when managing sensitive data pipelines. Strong Automation Skills ...

data engineering Experience in large-scale data processing, microservices, API design, Kafka, Redis, MemCached, observability tools (Dynatrace, Splunk, Grafana), and orchestration frameworks (Airflow, Temporal) Advanced working knowledge of relational and NoSQL databases, vector stores, data lake architectures, and data governance Practical cloud-native experience (AWS, Azure, or GCP) Ability … hands-on experience with Spark/PySpark and other big data processing technologies Expertise in open-source table formats and catalog services such as Apache Iceberg Experience with LLM orchestration frameworks and model serving infrastructure or managed endpoints (AWS Bedrock, Azure OpenAI) Familiarity with emerging technologies in data engineering ...

Skill Requirements:*** Strong experience in **Solution Architecture** and **Cloud-Native Architecture Design*** Hands-on expertise in **Snowflake Architecture*** Experience with **Data Lakehouse architectures**, including **Apache Iceberg*** Proficiency in designing and managing **S3-based Data Lakes*** Experience with **Apache Airflow/MWAA** for workflow orchestration* Strong knowledge … large-scale data processing* Ability to create **Architecture Design Documents (ADDs)*** Deep understanding of **cloud-native design patterns*** Experience in **performance tuning** across:* Snowflake* Airflow* Iceberg* Focus on **platform reliability, scalability, and observability*** Experience designing and operating **data platforms** in production environments #J-18808-Ljbffr ...

experience GIS or geospatial data handling Machine learning or data science background Time-series data and databases (e.g. InfluxDB, TimescaleDB) Data workflow orchestration (e.g. Airflow, Dagster) This role is 100% onsite, so you need to live within commutable distance. If you are interested, please apply to Matt Andrews ...

management * Software testing lifecycle/defect management * CI/CD & DevOps practices Desirable: HMRC/public sector experience, AWS/Azure/SAS PaaS, Airflow or SAS Migration Factory experience. If you are interested in this role or wish to apply, please submit your CV. ...

codebase and pipelines Hands-on experience maintaining production ML models: refresh, retrain, monitor, and troubleshoot, rather than building from scratch Working knowledge of Looker, Airflow and Google Cloud Advanced degree in a quantitative discipline (Computer Science, Mathematics, Statistics, Physics, or similar), or equivalent experience Benefits Hybrid working (3 days ...

data engineering Experience in large-scale data processing, microservices, API design, Kafka, Redis, MemCached, observability tools (Dynatrace, Splunk, Grafana), and orchestration frameworks (Airflow, Temporal) Advanced working knowledge of relational and NoSQL databases, vector stores, data lake architectures, and data governance Practical cloud-native experience (AWS, Azure, or GCP) Ability … hands-on experience with Spark/PySpark and other big data processing technologies Expertise in open-source table formats and catalog services such as Apache Iceberg Experience with LLM orchestration frameworks and model serving infrastructure or managed endpoints (AWS Bedrock, Azure OpenAI) Familiarity with emerging technologies in data engineering ...

data engineering Experience in large-scale data processing, microservices, API design, Kafka, Redis, MemCached, observability tools (Dynatrace, Splunk, Grafana), and orchestration frameworks (Airflow, Temporal) Advanced working knowledge of relational and NoSQL databases, vector stores, data lake architectures, and data governance Practical cloud-native experience (AWS, Azure, or GCP) Ability … hands-on experience with Spark/PySpark and other big data processing technologies Expertise in open-source table formats and catalog services such as Apache Iceberg Experience with LLM orchestration frameworks and model serving infrastructure or managed endpoints (AWS Bedrock, Azure OpenAI) Familiarity with emerging technologies in data engineering ...