76 to 100 of 377 Apache Airflow Jobs in England

Data Underpinnings Vector Databases: Experience with high-dimensional data retrieval using Milvus, Weaviate, or Pinecone (essential for RAG architectures). Data Orchestration: Familiarity with Airflow, Prefect, or Dagster for managing complex data movement and model training pipelines. Search & Discovery: Advanced ElasticSearch/OpenSearch for semantic search and large‐scale ...

coupled with rigorous validation of code, data transformations and analytical conclusions. Desirable Skills and Experience Experience with Azure Data Lake, Azure Data Factory, dbt, Airflow, MLflow, Power BI, Microsoft Purview or comparable tools. Experience with search indexes, vector databases, graph databases, knowledge graphs, ontologies or master-data-management techniques. ...

such as SageMaker, Vertex AI or Azure MLData engineering and MLOpsExperience with data warehouses such as Snowflake, BigQuery or RedshiftWorkflow orchestration using tools like Airflow or DagsterExperience with MLOps tools such as MLflow, Weights and Biases or similarAwareness of data and model drift, and how to monitor and respond ...

Strong hands-on experience building and operating data platforms on GCP. Working experience with GCP data services such as BigQuery, Dataflow, Dataproc, Cloud Composer (Airflow), Pub/Sub, Cloud Storage, Bigtable, and Cloud SQL. Proficiency in Python and SQL, with experience in Spark or Apache Beam. Experience designing ...

Ansible for repeatable, automated provisioning of GPU clusters and AI platform environments; GitOps (ArgoCD) for continuous, declarative platform delivery. Pipeline orchestration: Kubeflow Pipelines, Apache Airflow, or Argo Workflows to orchestrate multi-stage training, fine-tuning, and inference pipelines. Cluster & workload scheduling: Slurm, Run:ai, and NVIDIA Base Command ...

Jira and Agile development environments Desirable Experience Exposure to any of the following would be beneficial: Databricks or Snowflake AWS or other cloud environments Airflow or AWS Glue SSIS/SSRS Financial markets or financial data FinTech, SaaS, credit or data analytics environments Wider development technologies such as Python ...

Solid understanding of data modelling and modern architectures Strong knowledge of data governance, security and compliance Experience with orchestration tools such as Airflow Familiarity with CI/CD pipelines, Git workflows and automated testing Exposure to real-time, IoT or environmental telemetry data Experience & Qualifications Bachelor's degree ...

marketing teams to align data solutions with business needs**Requirements*** Strong SQL skills and experience with ETL/ELT and pipeline orchestration tools (e.g., Airflow, dbt, Data Factory)* Experience with cloud data platforms (e.g., Azure, AWS, or GCP)* Proficiency in data modelling (dimensional, entity-relationship, event-driven)* Understanding ...

with data pipelines, ELT workflows, and contemporary cloud data architectures.Experience with current cloud data platforms such as Snowflake, Azure Data Lake, Azure SQL, dbt, Airflow, or similar tools.Ability to understand and work with SQL queries and data models.Knowledge of data governance, privacy, security, regulatory reporting, and responsible data ...

pipelines end to end. Modern data stack Experience with modern ELT, transformation and orchestration technologies. Our environment includes technologies such as dbt, Dagster/Airflow and Snowflake, although experience with comparable technologies is equally relevant. Cloud infrastructure Experience designing and implementing secure, scalable solutions on a major cloud platform. ...

data pipelines, ELT workflows, and contemporary cloud data architectures. Experience with current cloud data platforms such as Snowflake, Azure Data Lake, Azure SQL, dbt, Airflow, or similar tools. Ability to understand and work with SQL queries and data models. Knowledge of data governance, privacy, security, regulatory reporting, and responsible ...

systems Hands-on experience with dbt or Dataform, building modular, tested data models Experience orchestrating data pipelines with a modern orchestrator such as Dagster, Airflow, or Prefect Solid knowledge of cloud platforms (GCP and/or AWS), with the ability to design and deploy data solutions at scale Experience ...

maintaining scalable data platforms. Strong SQL and data modelling skills, with experience designing analytical datasets. Experience with modern data stack tools (e.g. dbt, Spark, Airflow, or similar orchestration and transformation frameworks). Experience with cloud‐based data platforms, ideally AWS. Understanding of medallion architecture or similar data layering approaches. ...

analyse large datasets and communicate insights effectively. Experience with LLMs, generative AI, and prompt engineering. Knowledge of MLOps tools and frameworks (MLflow, Kubeflow, Airflow). Familiarity with SQL, NoSQL, and distributed data technologies. Experience deploying AI models in production environments. Publications, patents, or open-source contributions in ML/ ...

technical roadmap for data engineering across Elliptic. Leverage and deploy AI and agentic systems to operate more effectively. Tech environment Scala | Spark | Databricks | AWS | Airflow | Kubernetes | Terraform | Functional Programming You will thrive here if you Enjoy end‐to‐end ownership, from writing the code to architecture design to coaching ...

ability to work with non-technical stakeholders NICE TO HAVE Hands-on experience with Google Cloud Platform, especially BigQuery Experience with orchestration tools (e.g. Airflow ) Knowledge of data governance and data quality frameworks Exposure to reverse ETL tools (e.g. Hightouch ) Familiarity with AI coding assistants and a habit ...

relational databases (e.g., PostgreSQL, DuckDB). Experience with the modern data stack, building data ingestion pipelines and working with ETL and orchestration tools (e.g., Airflow, Luigi, Argo, dbt), big data technologies (Spark, Kafka, Parquet), and web frameworks for model serving (e.g. Flask or FastAPI). Data Science: Familiarity ...

safe to challenge ideas, learn, and grow. Nice to have Hands-on familiarity with the modern data stack — for example, Spark, Flink, Kafka, Debezium, Airflow, dbt, or similar — and with cloud platforms (AWS/GCP or equivalent) and modern DevOps practices. Experience leading teams that built data platforms used ...

experience with BigQuery . Experience building and supporting production data pipelines. Experience developing transformation layers using dbt . Experience with orchestration tools such as Apache Airflow or Cloud Composer . Strong SQL and Python development skills. Experience designing scalable data models and data warehouse solutions. Strong stakeholder engagement … independently and taking ownership of delivery. Why Join? Be part of a growing Data Engineering team. Work with modern technologies including GCP, BigQuery, dbt, Airflow, Terraform and cloud-native services . Help shape a strategic cloud data platform. Join a business investing heavily in its data capability. Enjoy real ...

similar language. Proficiency in database technologies (SQL, NoSQL, time-series databases) and data modelling. Strong understanding of data pipeline orchestration tools (e.g., Apache Airflow, Kubernetes). You thrive when working as part of a team Comfortable in a fast-paced environment Have excellent written and verbal English skills ...

with reverse ETL and moving data between warehouses and operational systems Strong understanding of APIs Familiarity with modern data engineering tools such as dbt, Airflow, Fivetran or Census Excellent stakeholder management skills If you're an experienced Senior Data Engineer who enjoys solving complex integration challenges, working with modern ...

data modeling, and data integrationSkilled in data manipulation, processing, and orchestration frameworksHands-on experience with Data Factory, Databricks, SQL DB, Synapse, Stream Analytics, Glue, Airflow, Kinesis, Redshift, SonarQube, and PyTestProficiency with GitHub, GitHub Actions, and Azure DevOpsStrong communication and collaboration skills to align technical and business stakeholdersProven ability ...

pipelines Strong programming skills in Python and SQL or similar Experience with technologies such as Spark, Databricks, AWS, Azure, GCP and/or Airflow Strong focus on data quality, testing and reliability Solid software engineering fundamentals and high engineering standards Strong communication and collaborative skills Comfortable working with ambiguity ...

maintaining data pipelines at scale Experience with a modern data stack (e.g. Snowflake, dbt, Fivetran, Debezium or similar) Experience with orchestration tools such as Airflow, Dagster or Prefect Proficiency in Python for building production data workflows Familiarity with data warehousing concepts and modelling patterns Experience working with cloud platforms ...

Data Vault, dimensional modelling or complex entity resolution. Building data foundations for machine learning, anomaly detection or GenAI applications. Experience with Azure Data Factory, Airflow or similar orchestration technologies. Why consider it? This is an opportunity to take genuine technical ownership of an important data engineering capability, rather than ...