1 to 25 of 61 Dagster Jobs in England

experience using Python and/or other modern programming languages. Experience developing and orchestrating data pipelines using technologies such as DBT, Apache Spark, Airflow, Dagster, Kafka, Flink, or equivalent tooling. Solid understanding of data lakehouse, warehouse, and medallion architecture patterns. Experience applying DevOps and DataOps best practices including ...

e.g., Apache Kafka, Kinesis). End-to-end exposure to data pipeline lifecycle development, including extensive experience with workflow orchestration tools (e.g., Apache Airflow, Dagster). Hands-on expertise with cloud data warehouses (e.g., Snowflake, BigQuery, Redshift) and data lake architectures (e.g., Databricks, Delta Lake, Apache Iceberg). Advanced ...

testing and data quality practice. A collaborative mindset and a genuine interest in building things well. Desirable Experience with orchestration tools such as Airflow, Dagster, Prefect or Azure Data Factory. Experience working within AWS, Azure, GCP or another enterprise cloud ecosystem. Experience with complex customer, transactional, digital or behavioural ...

looking for Advanced SQL, including writing complex queries, partitioning, clustering and cost-aware query optimisation . Experience of modern orchestration/ELT tooling, ideally Dagster and dbt. Strong Python for data engineering Solid understanding of data warehousing techniques; ETL/ELT processes and dimensional modelling. Experience designing data models ...

quality ETL/ELT pipelines and automated data workflows. Experience integrating heterogeneous data sources, databases and APIs. Experience with workflow orchestration tooling (e.g. Airflow, Dagster or dbt) for scheduling and operating production data pipelines. Good understanding of data quality, metadata, lineage, testing and reproducibility. Ability to evaluate technical options ...

relational or analytical storage technologies. Examples include Kafka, Spark, PostgreSQL.* **Orchestration** Experience defining and running data workflows using modern orchestration frameworks. Examples include Airflow, Dagster or Prefect.* **Data Quality and Lineage** Familiarity with tools and techniques for data testing, documentation and lineage. Examples include Great Expectations or dbt. ...

infrastructure and pipelines for reliability, monitoring and scale. Requirements: Production data platforms with high-volume ETL/ELT, distributed processing and orchestration (e.g. Airflow, Dagster, Prefect), as distinct from analytics pipelines. Python and SQL at a high level, plus Bash automation. Cloud infrastructure, with AWS preferred (S3, Lambda ...

.Data Platforms: Hands-on experience with Snowflake and data stack technologies like Apache Iceberg and Spark.Workflow Orchestration: Exposure to tools like Apache Airflow, Prefect, Dagster, or DBT.Data Services: Familiarity with AWS Glue, Lake Formation, EMR, EventBridge, Athena, and similar services.Metadata Tools: Experience with tools like Amundsen, Atlas, DataHub, OpenDataDiscovery ...

production pipelines end to end Excellent Python and SQL Solid grasp of data modelling, warehousing and lakehouse concepts Hands-on with orchestration (e.g. Airflow, Dagster) and distributed processing (e.g. Spark) Experience with cloud data platforms (AWS, Azure or GCP) and containerised deployment (Docker, Kubernetes) Good engineering practice: version control ...

language used for data processing Batch and event-driven processing. Experience with Kafka or an equivalent streaming system, a workflow orchestrator such as Airflow, Dagster or Prefect, and transformation tooling such as dbt or equivalent SQL-based workflows Cloud warehouse, lake or lakehouse experience , and integrating data from external ...

/cloud services (ECS, EKS, Lambda, Kinesis, DynamoDB, EMR, Athena) Hands-on experience with orchestration and workflow tools (Docker, Kubernetes, Airflow, Prefect, Dagster) Familiar with CI/CD pipelines, infrastructure as code, and DBT Able to explain complex data concepts clearly to non-technical stakeholders Why This Role Matters ...

Senior Data Engineer. In this role, you will take hands-on ownership of architecting, building, and deploying modern data pipelines in Snowflake, dbt and Dagster You will lead the designing of scalable data models and machine learning models Key Responsibilities Pipeline & Architecture Ownership: Design and implement automated, modular transformation ...

Senior Data Engineer. In this role, you will take hands-on ownership of architecting, building, and deploying modern data pipelines in Snowflake, dbt and Dagster You will lead the designing of scalable data models and machine learning models Key Responsibilities Pipeline & Architecture Ownership: Design and implement automated, modular transformation ...

data pipelines at scale Experience with a modern data stack (e.g. Snowflake, dbt, Fivetran, Debezium or similar) Experience with orchestration tools such as Airflow, Dagster or Prefect Proficiency in Python for building production data workflows Familiarity with data warehousing concepts and modelling patterns Experience working with cloud platforms ...

just training models in notebooks), including familiarity with serving patterns (e.g., REST APIs, batch inference, event-driven) Experience with orchestration tools (e.g., Airflow, Dagster) and cloud-native ML platforms (e.g., AWS SageMaker, GCP Vertex AI) Ability to influence decision-making regarding infrastructure and tooling, balancing 'build vs. buy' discussions ...

technical stakeholders to the right data solutions Familiarity with using AI tools to accelerate your workflow Nice to Have Experience with orchestration tools like Dagster, Airflow, or similar Experience building real-time or near-real-time data pipelines Familiarity with machine learning workflows and feature engineering Experience embedded ...

Postgres plus a warehouse or analytics store such as BigQuery or ClickHouse. Experience with streaming (Kafka, Pub/Sub or similar) and orchestration (Airflow, Dagster or similar). Data modelling for audit trails, lineage and multi-tenant isolation. Security-minded handling of sensitive data. Nice to have Data work ...

agreed timelines Clear, concise communication and confidence building relationships with business stakeholders Nice to have ETL or ELT pipeline design, including orchestration with Dagster and transformation with dbt Snowflake Cortex, including Agents, Search and LLM integration Experience building AI powered applications and conversational interfaces Azure App Service, authentication ...

code Grasps new business domains quickly, translating unfamiliar logic into clean, intuitive data models Bonus: ClickHouse or columnar databases; cloud-based data orchestration (Dagster, Step Functions); CI/CD for data pipelines and transformations; operating databases at scale in production Benefits Competitive salary and eligibility for equity Biannual bonus ...

e.g. AWS, GCP or Azure) Comfortable working in a Linux environment Nice to have: Advanced data engineering experience in Python, e.g. with libraries like Dagster, Prefect etc Experience optimising dataframe code, e.g. in Pandas or ideally Polars Experience of machine learning techniques and related libraries and frameworks e.g. scikit ...

unfamiliar logic into clean, intuitive data models without hand-holding Bonus Experience with ClickHouse or columnar databases Familiarity with cloud-based data orchestration tools (Dagster, Step Functions) CI/CD practices for data pipelines and transformations Experience operating databases at scale in a production environment Benefits Competitive salary ...

production data pipelines within regulated environments Experience implementing data validation and quality checks in production pipelines Experience with automation/orchestration tools, such as Dagster and cloud infrastructure on AWS Strong collaboration skills and attention to detail, with the curiosity to understand where data comes from ...

production data pipelines within regulated environments Experience implementing data validation and quality checks in production pipelines Experience with automation/orchestration tools, such as Dagster and cloud infrastructure on AWS Strong collaboration skills and attention to detail, with the curiosity to understand where data comes from ...

data products, focusing on reliable, scalable architectures. You'll work with SQL, Python, and cloud warehouses (BigQuery, Snowflake, Redshift) and orchestration tools (Airflow, Prefect, Dagster) while collaborating with analytics, data science, and product engineering to deliver impact across the organization. #J-18808-Ljbffr ...

data engineering experience with Spark, strong Python and SQL, extensive AWS services, and hands-on experience with orchestration tools (Docker, Kubernetes, Airflow, Prefect, Dagster). Hybrid work model in London. #J-18808-Ljbffr ...