51 to 75 of 376 Apache Airflow Jobs in England

like Spark, Flink etc., Solid grasp of Kubernetes, CI/CD and Terraform Working knowledge of ETL and ELT frameworks and orchestration tools like Airflow and dbt Ability to provide clear input, guide, empower junior members of the team to achieve desired outcomes Familiarity with machine learning ...

practices. Strong hands-on Snowflake experience, including SQL optimisation and data pipeline development. Experience building and orchestrating data pipelines using tools such as dbt, Airflow or similar. Strong SQL skills and proficiency in Python or another data engineering language. Experience implementing automated data quality checks, monitoring and validation frameworks. ...

cloud data warehouses (e.g. BigQuery, Snowflake, Redshift, ADW), including performance, cost, and data layout considerations. Proficiency with data integration and orchestration tools (e.g. Fivetran, Airflow, managed ELT platforms), and an understanding of how ingestion, transformation, and orchestration fit together. Experience building and maintaining an analytics or semantic modelling layer ...

Kubernetes Spark & distributed computing dbt Terraform Data Warehousing (we use Databricks but experience with other platforms is fine) MLOps implementation (desirable but not required) Airflow (or other orchestration tooling) Governance Things to note Just to let you know, we are dog friendly company ...

shared infrastructure and pipelines for reliability, monitoring and scale. Requirements: Production data platforms with high-volume ETL/ELT, distributed processing and orchestration (e.g. Airflow, Dagster, Prefect), as distinct from analytics pipelines. Python and SQL at a high level, plus Bash automation. Cloud infrastructure, with AWS preferred (S3, Lambda ...

Kanban. Strong analytical and problem‐solving skills, with a focus on delivering high-quality solutions. Experience with modern data engineering tools such as Airflow, dbt, Dataform, or similar technologies. Excellent communication and stakeholder management skills, with the ability to explain technical concepts to non‐technical audiences. What we offer ...

data platforms (AWS, GCP, or Azure) Technical credibility with data engineering practices: strong grasp of SQL, data modeling, pipeline design, and tools like dbt, Airflow, Fivetran, and similar Hands-on experience with cloud data warehouses (BigQuery, Redshift, or Snowflake); understanding of performance optimization, partitioning, and data quality patterns Ability ...

Google Cloud Platform ecosystem, like Cloud Run, Cloud Functions, Cloud Storage, Pub/Sub, and Cloud Scheduler Data orchestration tools such as Cloud Composer (Airflow), Prefect, or equivalent Looker/Data Studio and “Conventional analytics” (aka. BQ agenta) Building or maintaining a data warehouse in a regulated or healthcare ...

relational database design Engineering-SQL, Python, ETL/ELT, APIs, data modelling, data warehousing, testing and observability Orchestration-Azure Data Factory, Prefect, Airflow or comparable workflow platforms Business intelligence-Power BI, SSRS, QlikView, semantic modelling and governed self‐service reporting Development-Git/GitHub, CI/CD, VB/ ...

Strong hands-on experience building and operating data platforms on GCP. Working experience with GCP data services such as BigQuery, Dataflow, Dataproc, Cloud Composer (Airflow), Pub/Sub, Cloud Storage, Bigtable, and Cloud SQL. Proficiency in Python and SQL, with experience in Spark or Apache Beam. Experience designing ...

data engineering, with demonstrated expertise in designing and implementing scalable, flexible modern data architectures Experience building modern data lake/lakehouse architectures (e.g., Apache Iceberg) in cloud environments Strong knowledge of AWS (S3, Redshift) or Azure data services Experience with ETL/ELT processes and data ingestion Proven expertise … using transformation tools such as dbt Experience with data pipeline orchestration tools (such as Apache Airflow) Experience with best practices applied to data (e.g., testing, CI/CD) Advantageous Experience developing dashboards and reporting solutions using data visualization tools such as Power BI or Amazon QuickSight Experience with ...

Ansible for repeatable, automated provisioning of GPU clusters and AI platform environments; GitOps (ArgoCD) for continuous, declarative platform delivery. Pipeline orchestration: Kubeflow Pipelines, Apache Airflow, or Argo Workflows to orchestrate multi-stage training, fine-tuning, and inference pipelines. Cluster & workload scheduling: Slurm, Run:ai, and NVIDIA Base Command ...

implementing enterprise-scale data solutions. The ideal candidate will possess strong expertise in Snowflake and hands‐on experience with Python, PySpark, AWS, and Airflow for building scalable, high-performance data platforms. The role demands strong technical leadership, problem‐solving capabilities, and the ability to collaborate effectively with cross‐functional ...

core stack of SQL and Python, with experience across ETL and ELT, orchestration, streaming and data warehousing, using tools such as Spark, Kafka, Airflow, dbt, Snowflake, BigQuery or Databricks on AWS, GCP or Azure. Some roles focus on hands-on delivery, others on documenting and structuring that work. Commitment ...

members Required qualifications, capabilities and skills Proven experience in designing and building data pipelines (ETL/ELT) using modern technologies (e.g., Python, SQL, Spark, Airflow, etc.). Strong analytical skills with the ability to interpret complex data and deliver business value. Experience integrating data from multiple sources and systems. ...

Following Distributed computing technologies such as Spark and Databricks Cloud platforms including AWS, Azure, or GCP Workflow orchestration tools such as Airflow CI/CD, version control, and infrastructure as code Experience within financial services, consulting, or risk environments What’s on Offer Opportunity to make an impact from ...

Senior Data Engineer: Passionfor building scalable, resilient cloud data platforms Strongexperience with Databricks or Snowflake on AWS ProvenPython skills, including Spark and Airflow expertise AdvancedSQL skills and end-to-end data modelling experience Experiencebuilding batch and real-time data integrations Hands-onCICD skills with Git, Jenkins or similar Abilityto ...

Redshift), and data transformation tools (such as dbt). Pipeline mastery: Practical experience building and operating batch or streaming pipelines using orchestration tools like Airflow, Prefect, or Dagster. Strong communicator: Skilled at writing clear technical specifications and explaining complex trade-offs to both technical and non-technical colleagues. Growth ...

automated testing and data quality practice. A collaborative mindset and a genuine interest in building things well. Desirable Experience with orchestration tools such as Airflow, Dagster, Prefect or Azure Data Factory. Experience working within AWS, Azure, GCP or another enterprise cloud ecosystem. Experience with complex customer, transactional, digital ...

data modelling Python, Git, Docker ETL pipelines and data platforms Relational database engines, including PostgreSQL and DuckDB Orchestration (e.g. Step Functions, Airflow) AI-accelerated development workflows and tools including Claude Code and Codex AWS cloud infrastructure, including Lambda and ECS End-to-end data pipeline testing strategy and implementation ...

maintaining scalable data platforms. Strong SQL and data modelling skills, with experience designing analytical datasets. Experience with modern data stack tools (e.g. dbt, Spark, Airflow, or similar orchestration and transformation frameworks). Experience with cloud‐based data platforms, ideally AWS. Understanding of medallion architecture or similar data layering approaches. ...

experience you need Strong experience in Data Engineering and cloud-based data platforms. Hands-on knowledge of Databricks, Spark, SQL and Python. Understanding of Airflow, Terraform and modern engineering practices. Experience building large-scale batch and real-time data pipelines. Strong problem-solving and decision-making skills. Ability ...

with modern data warehouse technologies, particularly Snowflake.* Exposure to technologies such as Azure Data Factory, Synapse Analytics, Databricks, Spark, SQL, and Python. Experience with Airflow would be advantageous.Data Engineering* Proven experience applying data engineering techniques to design and develop scalable solutions.* Ability to build and maintain robust data pipelines ...

formats like Parquet. MLOps Exposure: Experience with tools and frameworks for training, tracking, deploying, and monitoring models in production (e.g., MLflow, Optuna, Prefect, or Airflow). Background in supply chain forecasting, enterprise AI, or retail analytics M.S. or Ph.D. in Computer Science, Artificial Intelligence, Statistics, Data Science ...

frameworks for large-scale batch and near real-time processing, with hands-on experience using orchestration and transformation technologies such as Airflow and dbt. Strong knowledge of software engineering best practices, including coding standards, code reviews, testing strategies, version control systems such as Git, and CI/CD processes. ...