26 to 50 of 80 Apache Airflow Jobs in the UK

skills Experience building data pipelines and integrating data from multiple sources Exposure to technologies such as Power BI, Snowflake, SQL Server, PostgreSQL, Spark, Kafka, Airflow, Docker, and Kubernetes Experience within engineering, manufacturing, industrial, or other complex regulated environments would be advantageous Excellent stakeholder management and problem-solving abilities What ...

pipelines end to end. Modern data stack Experience with modern ELT, transformation and orchestration technologies. Our environment includes technologies such as dbt, Dagster/Airflow and Snowflake, although experience with comparable technologies is equally relevant. Cloud infrastructure Experience designing and implementing secure, scalable solutions on a major cloud platform. ...

with reverse ETL and moving data between warehouses and operational systems Strong understanding of APIs Familiarity with modern data engineering tools such as dbt, Airflow, Fivetran or Census Excellent stakeholder management skills If you're an experienced Senior Data Engineer who enjoys solving complex integration challenges, working with modern ...

initial 6-month contract. Core Technical Requirements Strong experience with Snowflake for data warehousing, including writing efficient SQL and managing schemas. Proficiency in Airflow for orchestration and workflow management. Hands-on experience with AWS services , particularly S3 for storage and Lambda for serverless processing. Solid understanding of data modelling ...

analyse large datasets and communicate insights effectively. Experience with LLMs, generative AI, and prompt engineering. Knowledge of MLOps tools and frameworks (MLflow, Kubeflow, Airflow). Familiarity with SQL, NoSQL, and distributed data technologies. Experience deploying AI models in production environments. Publications, patents, or open-source contributions in ML/ ...

common CI/CD stack (e.g., Jenkins, CircleCI, GitLab, Azure DevOps) Experience in workflow orchestration with tools such as Argo Workflow, Airflow, and scientific workflow tools such as Nextflow, Snakemake, VisTrails, or Cromwell Experience with application performance tuning and optimization, including in parallel and distributed computing paradigms and communication ...

Snowpark Iceberg Tables Secure Data Sharing Snowflake Cortex Data Integration Fivetran SnapLogic Enterprise SaaS integration patterns Data Transformation & Orchestration dbt Cloud/dbt Core Apache Airflow (MWAA) CI/CD pipelines Event-driven orchestration Data Governance & Observability Collibra Data Catalogue Data Lineage Data Governance Frameworks Acceldata Data Observability ...

need both skillsets below - strength in one is enough: Hands-on: building AI/ML pipelines or prototypes; vector databases; prompt engineering; orchestration tools (Airflow/Prefect); Git Advisory: designing AI/data architectures; Agentic AI principles (LangGraph, CrewAI, AutoGen); mentoring specialists without formal authority; AI governance (security, privacy ...

pipelines Strong programming skills in Python and SQL or similar Experience with technologies such as Spark, Databricks, AWS, Azure, GCP and/or Airflow Strong focus on data quality, testing and reliability Solid software engineering fundamentals and high engineering standards Strong communication and collaborative skills Comfortable working with ambiguity ...

hands-on experience in at least one data area and one platform area below. Data experience Orchestration: operating pipelines with tools such as Airflow, Dagster or Prefect, including dependencies, retries, backfills, reruns or schema changes. Storage and modelling: designing schemas and storage for relational databases, warehouses or lakehouses ...

lake, lakehouse, warehouse or large-scale analytical platform delivery. • Python, Bash or other scripting and automation approaches. • Workflow orchestration and scheduling tools, such as Airflow, TWS, JS7 or similar. • CI/CD, version control and modern engineering delivery practices. • Data quality, testing, observability and operational resilience. • Working with enterprise ...

models, schemas to support business requirements Develop and maintain data ingestion and processing systems using various tools andtechnologies, such as SQL, NoSQL, ETL, Luigi, Airflow, Argo, etc. Implement data storage solutions using different types of databases, such asrelational, non-relational, or cloud-based. Working collaboratively with the client …/Azure SQL, PostgreSQL) You have framework experience within either Flask, Tornado or Django, Docker Experience working with ETL pipelines is desirable e.g. Luigi, Airflow or Argo Experience with big data technologies, such as Apache Spark, Hadoop, Kafka, etc. Data acquisition and development of data sets and improving ...

concepts, experience with LLMs, prompt engineering, or integrating AI capabilities into applications • Data Technologies – Experience with Kafka, PySpark, Parquet and Avro data formats, Apache Airflow for workflow management, or Streamlit for building interactive data applications • Advanced Data Management – Strong SQL design, query optimization, and database architecture expertise • Observability ...

solution design responsibility. You'll ideally have: Strong commercial experience with Python & Snowflake Experience mentoring or technically leading other engineers Experience with dbt, Airflow or similar data engineering tooling A good understanding of data modelling and modern data platform architecture Experience working with AWS (ideally) or another major cloud ...

orchestration It would be nice if you could bring (desirables): Professional experience with PySpark, MySQL, Docker, Kubernetes Experience with best-practice pipeline orchestration software (Airflow, Dagster). Experience migrating code from prototype notebooks to .py, while maintaining logic and transparency of underlying code Owned a project across a whole ...

times series data, they use TimescaleDB but other experience with similar platforms is fine You have experience with Orchestration platforms such as Dagster or Airflow You're collaborative with great communication skills What's in it for you: Salary to £85k + bonus X3 weeks per year work anywhere ...

cloud-native architectures, with experience shaping technical direction across teams and stakeholders Solid expertise in relevant technologies such as AWS, Terraform, Kafka/MSK, Airflow, Glue or Spark, and building scalable data solutions Experience with modern data ecosystems including data lakes/lakehouse architectures, Iceberg or similar table formats ...

building data pipelines Proficient in SQL, Python, AWS and other data management technologies Familiarity with version control systems like Git and orchestration tools like Airflow Ability to test and troubleshoot data platforms reliably. A track record of improving data reliability and quality Excellent communication skills and the ability ...

building data pipelines Proficient in SQL, Python, AWS and other data management technologies Familiarity with version control systems like Git and orchestration tools like Airflow Ability to test and troubleshoot data platforms reliably. A track record of improving data reliability and quality Excellent communication skills and the ability ...

pipelines. Proficiency in SQL, Python, AWS, and other modern data engineering technologies. Experience with version control tools (e.g., Git) and workflow orchestration platforms (e.g., Airflow). Proven ability to test, troubleshoot, and improve data platforms, ensuring high levels of reliability and data quality. Strong interpersonal and communication skills with ...

necessary infrastructure, resources, and interfaces to enable data loading and LLM workflows. Use Python/Java and large-scale data workflow orchestration platforms e.g. Airflow to construct software artifacts for ETL, interfacing with diverse data formats and storage technologies, and incorporate them into robust data workflows and dynamic systems ...

Data Engineer Location: Remote – with requirement to visit London based office twice a month Hours: 37.5 Hours a week (Monday – Friday 9:00am – 17:00pm) Salary Range: £40,000 - £80,000 per annum (Depending on ...

Nice to have Consulting or client-facing delivery experience Experience evaluating and monitoring LLM systems (testing, guardrails, cost control) MLOps tools such as MLflow, Airflow or CI/CD pipelines We Are Dcoded is acting as an employment agency in relation to this permanent vacancy. ...

Data Engineer within GCP. Experience building production-grade data pipelines and platforms. Expertise with technologies such as BigQuery, Dataflow, Cloud Composer/Airflow, Pub/Sub and Cloud Storage. Strong SQL and data engineering fundamentals. An engineering mindset and a passion for solving problems. Confidence working independently while remaining ...

operate across engineering, product, and business functions Desirable Experience LLM applications and document-processing platforms Retrieval-Augmented Generation (RAG) React and Next.js Snowflake Airflow Data engineering and ETL pipelines Kubernetes and cloud-native infrastructure Supply chain, procurement, or operational systems Ideal Background Suitable candidates may come from: Product ...