26 to 50 of 79 Apache Airflow Jobs in the UK

Experience Extensive experience in data engineering Strong programming expertise in Python Hands-on experience with Databricks (workflows, notebooks, jobs, Delta Lake) Strong experience with Apache Spark/PySpark Experience building scalable ETL pipelines Proficiency in SQL and relational databases Experience with cloud platforms (AWS/Azure/GCP) Experience … with modern data warehousing solutions (e.g., Snowflake, BigQuery, Redshift) Familiarity with orchestration tools (e.g., Airflow, Databricks Workflows) Strong understanding of data modelling and lakehouse architecture Databricks certifications (e.g., Databricks Certified Data Engineer) Experience with streaming data pipelines (Kafka, Structured Streaming) Experience with CI/CD and DevOps practices Knowledge ...

Python for data engineering and strong SQL skills for data modelling and analysis Experience building and orchestrating data pipelines using tools such as Airflow or dbt within cloud-based environments Expertise in developing clear, user-friendly dashboards using tools such as Tableau or Looker Studio Deep understanding of music ...

environments Solid experience with SQL and data modelling concepts Strong analytical and problem-solving skills Nice to Have Skills Experience with data pipeline tools (Airflow, Kafka, Spark) Knowledge of cloud platforms (AWS, Azure or GCP) Familiarity with relational and NoSQL databases Additional programming experience (Python, Scala) Essential Requirements Active ...

Nice to Have: * Microsoft Azure Solutions Architect Expert* AWS Solutions Architect Professional* Databricks Certifications* CDMP Certification* Terraform, Bicep or Infrastructure-as-Code experience* dbt, Airflow or modern orchestration frameworks* Experience supporting AI, GenAI or RAG-based architectures* Experience within Financial Services, Healthcare, Life Sciences, Energy, Retail or Manufacturing Robert ...

Redux, Redux Hooks, and other modern state management libraries. Kotlin on Android and Swift on iOS. Data Analytics/Engineering: SQL, Python, Snowflake, dbt, Airflow & Looker. AI at Marshmallow: Sierra.ai for conversational AI, Python and LangChain for agentic AI solutions, SageMaker and MLflow for Machine Learning, Cursor and Claude ...

Proven experience in a Lead/Senior Data Engineering role Strong handson experience with Snowflake Expertise in ETL/ELT and orchestration tools (e.g. Airflow, dbt, Matillion) Strong SQL and Python skills Experience building scalable data platforms in cloud environments Previous experience leading or mentoring engineers Strong communication ...

Snowflake Toolkit: Hands-on with Snowpipe, Streams, Tasks, Dynamic Tables, and Stored Procedures. Nice to Have: Exposure to DBT, Snowpark, Snowflake AI/ML, Airflow, and Snowflake administration (RBAC, warehouse sizing). Mindset: Strong problem-solving abilities with an ownership mindset for end-to-end delivery. This ...

warehousing concepts Exposure to data transformation tools such as DBT is beneficial Basic knowledge of Python, ideally in the context of orchestration tools like Airflow Confident communication skills and the ability to work effectively in a remote team A proactive approach with the ability to take ownership and solve ...

production-grade data pipelines Solid understanding of integration patterns including REST APIs, webhooks, OAuth and event-driven architectures Experience with orchestration tools such as Airflow, Prefect or Dagster Familiarity with cloud environments (AWS, GCP or Azure) Experience with containerisation tools such as Docker and Kubernetes Proven experience integrating multiple ...

necessary infrastructure, resources, and interfaces to enable data loading and LLM workflows. Use Python/Java and large-scale data workflow orchestration platforms e.g. Airflow to construct software artifacts for ETL, interfacing with diverse data formats and storage technologies, and incorporate them into robust data workflows and dynamic systems ...

necessary infrastructure, resources, and interfaces to enable data loading and LLM workflows. Use Python/Java and large-scale data workflow orchestration platforms e.g. Airflow to construct software artifacts for ETL, interfacing with diverse data formats and storage technologies, and incorporate them into robust data workflows and dynamic systems ...

common CI/CD stack (e.g., Jenkins, CircleCI, GitLab, Azure DevOps) Experience in workflow orchestration with tools such as Argo Workflow, Airflow, and scientific workflow tools such as Nextflow, Snakemake, VisTrails, or Cromwell Experience with application performance tuning and optimization, including in parallel and distributed computing paradigms and communication ...

combines deep data expertise with broader engineering capability, get in touch with Justin Donaldson for a confidential chat. Skills: dbt spark sql snowflake apache airflow cloud TLNT1 ...

combines deep data expertise with broader engineering capability, get in touch with Justin Donaldson for a confidential chat. Skills: dbt spark sql snowflake apache airflow cloud JBLE1 ...

/SD/PP etc.) required for December cutover. Build scalable ingestion pipelines using Delta Lake/Parquet and modern orchestration tools (ADF, dbx, Airflow, etc.). Collaborate with SAP functional teams to validate business logic, deltas, and data quality. Implement CI/CD and DevOps practices for data ...

preferred) or similar platforms: Snowflake, BigQuery, Redshift, Synapse, or equivalent Strong proficiency with: SQL (advanced level) Spark/PySpark Data pipeline orchestration tools (e.g., Airflow, ADF, etc.) Data Modelling Extensive experience designing: Fact and dimension tables Star and snowflake schemas Aggregation layers and semantic models Data Migration & Mapping Demonstrated ...

/ELT development, orchestration frameworks, cloud-based data platforms, and data quality management. You will be comfortable working with technologies such as dbt, Airflow, PostgreSQL, cloud warehouses or lakehouses, and modern data architecture patterns. Most importantly, you'll be someone who enjoys ownership, takes pride in building reliable solutions ...

deep knowledge in core processing and orchestration products such as Big Query, Data Flow, Data Fusion, Data Stream , Cloud Functions, Data Proc and Airflow/Composer . You will have excellent problem-solving skills, a rigorous approach to code checks/peer reviews and have the strength of character ...

organisation Deep understanding & hands-on leverage of Modern, Cloud scale Data Technologies, with specific expertise in Distributed Data Processing architectures (Spark, PySpark, Python, Airflow), ELT data processing underpinning Data LakeHouses and Data Products for scaled use within ML & AI Expertise in Streaming Data, Event Driven Design, Data Governance ...

data modelling Experience integrating messy external APIs and bulk datasets Strong Postgres Experience with a transformation framework ( dbt or SQLMesh ) and an orchestrator ( Dagster , Airflow, Prefect, or similar). Modern Python data tooling; DuckDB for analytical compute. Financial Services/Wealth domain would be ideal but not essential ...

retrieval augmented generation (RAG) architectures. * Strong knowledge of cloud platforms including AWS or Azure. * Experience with data engineering and orchestration tools such as Databricks, Airflow, Docker, and Kubernetes is desirable. * Must have previous experience working within a small to medium-sized business (SME), start-up, or scale-up environment. ...

platforms and pipelines (batch and real-time ingestion). Hands-on familiarity with: Snowflake (or equivalent cloud data warehouse), Looker (or other BI tools), Airflow, Pub/Sub, or similar orchestration and ingestion frameworks. Experience with data infrastructure monitoring and optimization. Strong communication skills. Ability to translate business goals ...

Kubernetes and modern containerised deployment patterns Strong experience with CI/CD pipelines, infrastructure-as-code and workflow orchestration Experience with tools such as Airflow or similar platform and orchestration technologies Good understanding of model observability, data quality, feature pipelines, lineage and reproducibility Experience designing scalable infrastructure ...

libraries (scikit-learn, NumPy, LightGBM, Pandas, LangChain/LangGraph, TensorFlow, etc.) PySpark AWS cloud infrastructure: EMR, ECS, Athena, etc MLOps/DevOps: Terraform, Docker, Airflow, MLFlow Monitoring and evaluation platforms including New Relic and Braintrust More information Enjoy fantastic perks like private healthcare & dental insurance, a generous work from ...

demonstrable performance metrics. Preferred Bachelor's degree in Computer Science or Software Engineering. Experience deploying AI within AWS and MS Azure. Experience using Docker, Airflow and Openshift. Cloud Certification. Data science model review, code refactoring, model optimization, containerisation, deployment, versioning, monitoring of model quality and non-functional requirements. ...