1 to 25 of 85 Dagster Jobs in the UK

clearance as a condition of employment. Nice to have Experience with distributed data processing or streaming tools (e.g. Spark, Kafka) Workflow orchestration (e.g. Airflow, Dagster or similar) Working with LLMs, vector databases, embeddings or RAG patterns MLOps practices: model versioning, deployment and monitoring Containers and infrastructure as code (e.g. ...

e.g., Apache Kafka, Kinesis). End-to-end exposure to data pipeline lifecycle development, including extensive experience with workflow orchestration tools (e.g., Apache Airflow, Dagster). Hands-on expertise with cloud data warehouses (e.g., Snowflake, BigQuery, Redshift) and data lake architectures (e.g., Databricks, Delta Lake, Apache Iceberg). Advanced ...

experience using Python and/or other modern programming languages. Experience developing and orchestrating data pipelines using technologies such as DBT, Apache Spark, Airflow, Dagster, Kafka, Flink, or equivalent tooling. Solid understanding of data lakehouse, warehouse, and medallion architecture patterns. Experience applying DevOps and DataOps best practices including ...

without an existing backlog, structure, or playbook Fluent English and strong communication skills Bonus Points For Experience with orchestration and transformation tools (Airflow, dbt, Dagster) Infrastructure as code and CI/CD (Terraform, Git-based workflows) Experience with data governance, data quality, or master data management Experience with Azure ...

looking for Advanced SQL, including writing complex queries, partitioning, clustering and cost-aware query optimisation . Experience of modern orchestration/ELT tooling, ideally Dagster and dbt. Strong Python for data engineering Solid understanding of data warehousing techniques; ETL/ELT processes and dimensional modelling. Experience designing data models ...

relational or analytical storage technologies. Examples include Kafka, Spark, PostgreSQL.* **Orchestration** Experience defining and running data workflows using modern orchestration frameworks. Examples include Airflow, Dagster or Prefect.* **Data Quality and Lineage** Familiarity with tools and techniques for data testing, documentation and lineage. Examples include Great Expectations or dbt. ...

quality ETL/ELT pipelines and automated data workflows. Experience integrating heterogeneous data sources, databases and APIs. Experience with workflow orchestration tooling (e.g. Airflow, Dagster or dbt) for scheduling and operating production data pipelines. Good understanding of data quality, metadata, lineage, testing and reproducibility. Ability to evaluate technical options ...

grade data pipelines end to end. Modern data stack Experience with modern ELT, transformation and orchestration technologies. Our environment includes technologies such as dbt, Dagster/Airflow and Snowflake, although experience with comparable technologies is equally relevant. Cloud infrastructure Experience designing and implementing secure, scalable solutions on a major ...

Senior Data Engineer. In this role, you will take hands-on ownership of architecting, building, and deploying modern data pipelines in Snowflake, dbt and Dagster You will lead the designing of scalable data models and machine learning models Key Responsibilities Pipeline & Architecture Ownership: Design and implement automated, modular transformation ...

batch and streaming data pipelines using SQL, Python, PySpark, and Delta Lake. Familiarity with dbt for data transformations and orchestration tools such as Airflow, Dagster, or Prefect. Experience designing and delivering secure, resilient API, webhook, and event‐driven integrations. Strong understanding of engineering practices, including version control, CI/ ...

Databases: Experience with high-dimensional data retrieval using Milvus, Weaviate, or Pinecone (essential for RAG architectures). Data Orchestration: Familiarity with Airflow, Prefect, or Dagster for managing complex data movement and model training pipelines. Search & Discovery: Advanced ElasticSearch/OpenSearch for semantic search and large-scale indexing. AI Frameworks ...

Databases: Experience with high-dimensional data retrieval using Milvus, Weaviate, or Pinecone (essential for RAG architectures). Data Orchestration: Familiarity with Airflow, Prefect, or Dagster for managing complex data movement and model training pipelines. Search & Discovery: Advanced ElasticSearch/OpenSearch for semantic search and large-scale indexing. AI Frameworks ...

/cloud services (ECS, EKS, Lambda, Kinesis, DynamoDB, EMR, Athena) Hands-on experience with orchestration and workflow tools (Docker, Kubernetes, Airflow, Prefect, Dagster) Familiar with CI/CD pipelines, infrastructure as code, and DBT Able to explain complex data concepts clearly to non-technical stakeholders Why This Role Matters ...

data pipelines at scale Experience with a modern data stack (e.g. Snowflake, dbt, Fivetran, Debezium or similar) Experience with orchestration tools such as Airflow, Dagster or Prefect Proficiency in Python for building production data workflows Familiarity with data warehousing concepts and modelling patterns Experience working with cloud platforms ...

next. Cloud: AWS, GCP, Azure Ingestion: Fivetran, Airbyte, ... Warehouse and lakehouse: Snowflake, Databricks, BigQuery, Microsoft Fabric Transformation: dbt Orchestration: Airflow, Dagster, Prefect, ... BI and semantic layers: Power BI, Looker, Tableau, ... Activation and CDPs: Hightouch, Census, Segment, ... Nice to have Building the data side of agentic ...

technical stakeholders to the right data solutions Familiarity with using AI tools to accelerate your workflow Nice to Have: Experience with orchestration tools like Dagster, Airflow, or similar Experience building real-time or near-real-time data pipelines Familiarity with machine learning workflows and feature engineering Experience embedded ...

knowledge of data governance and privacy obligations, including GDPR Experience ensuring analytics outputs are accurate, documented, and appropriately controlled Valuable Extras Analytics engineering practice: Dagster, version control, CI/CD, and automated testing Experience with cloud data platforms, such as Snowflake or BigQuery Group Personal Pension Plan – 8% employer ...

knowledge of data governance and privacy obligations, including GDPR Experience ensuring analytics outputs are accurate, documented, and appropriately controlled Valuable Extras Analytics engineering practice: Dagster, version control, CI/CD, and automated testing Experience with cloud data platforms, such as Snowflake or BigQuery Group Personal Pension Plan – 8% employer ...

Clear communication skills, with the ability to translate business requirements into technical solutions. Technology Environment The wider stack includes Snowflake, dbt, SQL, Python, Fivetran, Dagster, Metaplane and Tableau, alongside AWS, Terraform and GitHub. You don't need experience with every tool listed. Strong SQL, commercial dbt experience and relevant ...

agreed timelines Clear, concise communication and confidence building relationships with business stakeholders Nice to have ETL or ELT pipeline design, including orchestration with Dagster and transformation with dbt Snowflake Cortex, including Agents, Search and LLM integration Experience building AI powered applications and conversational interfaces Azure App Service, authentication ...

Postgres plus a warehouse or analytics store such as BigQuery or ClickHouse. Experience with streaming (Kafka, Pub/Sub or similar) and orchestration (Airflow, Dagster or similar). Data modelling for audit trails, lineage and multi-tenant isolation. Security-minded handling of sensitive data. Nice to have Data work ...

comes with tests, and you leave things tidier than you found them. Depth in at least one of SQL, Spark, Databricks/Delta Lake, Dagster/Airflow/Temporal or dbt. Bonus if you’ve shaped any of them into something a model can query, through a metrics layer ...

hands‐on experience in at least one data area and one platform area below. Data experience Orchestration: operating pipelines with tools such as Airflow, Dagster or Prefect, including dependencies, retries, backfills, reruns or schema changes. Storage and modelling: designing schemas and storage for relational databases, warehouses or lakehouses ...

comes with tests, and you leave things tidier than you found them. Depth in at least one of SQL, Spark, Databricks/Delta Lake, Dagster/Airflow/Temporal or dbt. Bonus if you've shaped any of them into something a model can query, through a metrics layer ...

Model and manage data across data warehouses and lakes and operational databases Build batch and streaming workflows, orchestrated with tools such as Airflow, Prefect, Dagster, or dbt Own data quality, validation, lineage, and observability ensuring the data feeding our models is accurate, timely, and trustworthy Engineer features and datasets … Technical Skills Python (essential) – pandas, NumPy, and the modern data/ML ecosystem SQL – advanced querying, modelling, and optimisation Data pipelines & orchestration – Airflow, Prefect, Dagster, or dbt Data warehouses/lakes – BigQuery, Snowflake, Redshift, or Databricks ML frameworks – PyTorch, TensorFlow, or scikit‐learn LLM & AI tooling – OpenAI, Anthropic; LangChain ...