176 to 200 of 475 Apache Airflow Jobs in the UK

/ELT frameworks, and cloud data platforms such as Azure, AWS, or GCP. Data modelling and pipeline orchestration using tools such as Airflow, Data Factory, dbt, or similar. Integrate CDP data sources across digital, CRM, transactional, and behavioural data; support identity resolution, consent handling, audience activation, journey events, trigger ...

scalable access Ensure systems are secure, reliable and cost-effective Collaborate with analysts, ML engineers and platform teams You might work with: Python, Spark, Airflow Snowflake, Databricks, BigQuery, Neo4J dbt, Fivetran, Terraform Your work matters and so do you. That’s why we back your skills with a structure ...

systems, inventory extraction from application metadata tables (reporting opening). Columnar and lake formats (Parquet or similar), large object stores, orchestration platforms (Airflow or similar). Working-level knowledge of graph databases; data contracts and data quality frameworks; catalogue platforms (DataHub or similar) on the ingestion side. ...

ontologies, and versioned/temporal data. Understanding of how data structures feed RAG and LLM retrieval quality. SQL proficiency; experience with orchestration tools (e.g., Airflow) and cloud data services. Familiarity with data governance, security, and privacy practices. Strong documentation and cross-team communication skills. WHO YOU ARE You think ...

Snowflake and dbt experience: modelling, semantic layers, agents & skills, testing, performance and cost tuning. Production experience with a pipeline orchestrator, ideally Dagster (or similar Airflow, Prefect), including dependent workloads, partial failures, and observability. Modern cloud infrastructure (Azure/AWS/GCP) and Kubernetes experience deploying and operating containerised workloads ...

comparable transformation framework – is a strong advantage and an area we’d invest in developing. Experience with data orchestration tools such as Airflow or equivalent. Python or another scripting language for data manipulation and automation. Background in consumer lending, payments, or embedded finance products. Experience working in an Analytics ...

metrics layers. Experience applying AI to Analytics Engineering workflows or developing AI-ready data products. Knowledge of orchestration and ingestion tools such as Airflow, Dagster or Fivetran. Python experience for automation or data engineering tasks. Experience working in regulated financial services or another highly data-driven organisation. Snowflake experience ...

data platforms, or significantly evolving existing ones. Experience with data quality and observability, using dbt or complementary tooling. Familiarity with orchestration tools such as Airflow, Dagster, or Prefect. Experience with semantic layers, metrics frameworks, or analytics consumption patterns. Exposure to data governance, access control, or analytics enablement at scale. ...

Application Resiliency, and Security. Preferred Qualifications, Capabilities, And Skills Production experience with Databricks (Delta Lake, Unity Catalog, MLflow, Databricks Jobs) and workflow orchestration with Apache Airflow. Experience with evaluation frameworks and approaches (golden datasets, LLM-as-judge, human-in-the-loop review, red teaming). Solid grounding in data ...

Working knowledge of Kubernetes/EKS, comfortable deploying to and reasoning about containerised workloads in a managed K8s environment. Experience orchestrating pipelines with Dagster, Airflow, Prefect, or similar. Solid grasp of ETL concepts and production‐grade pipeline design. Beneficial Skills Interest in chemistry, life sciences, robotics, or manufacturing. Hands ...

various platforms Experience briefing, designing and implementing ETL solutions end-to-end, from requirements gathering through to deployment Experience with data orchestration tools (e.g. Airflow, Prefect) and building scalable data pipelines Strong Git and collaborative development skills (e.g. GitHub, GitLab) Strong understanding of data fundamentals: MDM, data integrity best ...

analytics Model and manage data across data warehouses and lakes and operational databases Build batch and streaming workflows, orchestrated with tools such as Airflow, Prefect, Dagster, or dbt Own data quality, validation, lineage, and observability ensuring the data feeding our models is accurate, timely, and trustworthy Engineer features … similar) Technical Skills Python (essential) – pandas, NumPy, and the modern data/ML ecosystem SQL – advanced querying, modelling, and optimisation Data pipelines & orchestration – Airflow, Prefect, Dagster, or dbt Data warehouses/lakes – BigQuery, Snowflake, Redshift, or Databricks ML frameworks – PyTorch, TensorFlow, or scikit‐learn LLM & AI tooling – OpenAI, Anthropic ...

find to improve model and system performance. Build reliable ML infrastructure Develop and maintain data and machine learning workflows using technologies such as Apache Airflow. Help improve CI/CD, automated testing, monitoring and reproducibility across our ML stack. Engineer on AWS Build and improve cloud infrastructure using services … expect you to know everything: PyTorch, TensorFlow or JAX Hugging Face LLM APIs and open-source models RAG and vector databases Apache Airflow AWS S3 ECS/EKS Redshift Docker Kubernetes GitHub Actions Pulumi or Terraform SQL Model monitoring and observability Distributed processing Financial markets, economics or commodities ...

tooling, automated testing and reliable deployment processes. Startup Mindset: Proactive, resourceful and comfortable taking responsibility in a small, fast‐moving engineering team. Experience with Apache Airflow, Kubernetes, real‐time data platforms, AI/ML infrastructure, LLM agents, GPU infrastructure or financial‐market systems would be useful ...

validate transformations and trace data across platforms. Tooling/Delivery Methods Working knowledge of scheduling/orchestration tools such as Autosys and/or Apache Airflow (monitoring schedules, reruns, failure triage). Experience with CI/CD and release controls (Git, Harness, UrbanCode Deploy (UCD), Red Hat OpenShift ...

concepts, experience with LLMs, prompt engineering, or integrating AI capabilities into applications • Data Technologies – Experience with Kafka, PySpark, Parquet and Avro data formats, Apache Airflow for workflow management, or Streamlit for building interactive data applications • Advanced Data Management – Strong SQL design, query optimization, and database architecture expertise • Observability ...

collaborate with engineers, scientists and product teams to design and optimise data pipelines across our analytics platform. The ideal candidate has strong Python, Spark, Airflow and SQL skills, plus experience with Databricks or Snowflake on AWS, and familiarity with CI/CD and Terraform. Hybrid working within ...

times series data, they use TimescaleDB but other experience with similar platforms is fine You have experience with Orchestration platforms such as Dagster or Airflow You're collaborative with great communication skills What's in it for you: Salary to £120k + bonus X3 weeks per year work anywhere ...

support critical systems Core Skills and Technologies Experience running distributed data and analytics systems at scale using tools such as Spark, Kafka, Trino or Airflow Strong Linux skills and proficiency in Python for automation and integration Familiarity with infrastructure as code, using Terraform or Ansible Deep understanding ...

Arrow, plus an analytical engine, e.g. DuckDB, or a warehouse such as Snowflake. Orchestrated batch pipelines you've operated, not just written: Dagster or Airflow, S3-based data flows, and a habit of testing outputs for correctness rather than only for exceptions. Applied ML in production: embeddings and semantic ...

technologies CI/CD and DevOps practices Docker/containerisation, ECS, Fargate or Kubernetes Kafka/Kinesis, Flink or other real-time data technologies Airflow and data engineering pipelines Data visualisation and dashboarding Datadog or similar monitoring/logging platforms Automated testing, TDD, BDD and performance testing Low-latency ...

scalable data assets that can be used in BI and other user facing tools. Experience working with data pipeline orchestration tools like Prefect or Airflow and comfortable working with Python for data orchestration. Ability to communicate clearly with both technical and non-technical stakeholders, translating data into business value. ...

wait to be told what to analyze next. Nice to Have Experience in crypto, payments, banking, fraud, or compliance. Familiarity with dbt, Airflow, or similar data-stack tooling. Exposure to causal inference (difference-in-differences, propensity scoring, uplift modeling). Experience deploying models to production (batch or real-time ...

database querying languages Experience in large‐scale data processing, microservices, API design, Kafka, Redis, MemCached, Observability tools (Dynatrace, Splunk, Grafana), and Orchestration tools (Airflow, Temporal) Demonstrated experience leading effective use of enterprise‐authorized AI‐assist­ed software development tools within the work environment (e.g., for coding, code review, test acceleration ...

orchestration It would be nice if you could bring (desirables): Professional experience with PySpark, MySQL, Docker, Kubernetes Experience with best-practice pipeline orchestration software (Airflow, Dagster). Experience migrating code from prototype notebooks to .py, while maintaining logic and transparency of underlying code Owned a project across a whole ...