1 to 25 of 78 Dagster Jobs in the UK

e.g., Apache Kafka, Kinesis). End-to-end exposure to data pipeline lifecycle development, including extensive experience with workflow orchestration tools (e.g., Apache Airflow, Dagster). Hands-on expertise with cloud data warehouses (e.g., Snowflake, BigQuery, Redshift) and data lake architectures (e.g., Databricks, Delta Lake, Apache Iceberg). Advanced ...

experience using Python and/or other modern programming languages. Experience developing and orchestrating data pipelines using technologies such as DBT, Apache Spark, Airflow, Dagster, Kafka, Flink, or equivalent tooling. Solid understanding of data lakehouse, warehouse, and medallion architecture patterns. Experience applying DevOps and DataOps best practices including ...

BigQuery, Redshift, EMR, or similar). Our stack is Databricks and Snowflake; strong equivalent experience elsewhere transfers. Experience with a workflow orchestrator (e.g. Airflow, Dagster, or Databricks Workflows). Proven system and platform design judgment: you own architectural decisions and can defend the trade-offs. Greenfield, scaling, and migration ...

relational or analytical storage technologies. Examples include Kafka, Spark, PostgreSQL.* **Orchestration** Experience defining and running data workflows using modern orchestration frameworks. Examples include Airflow, Dagster or Prefect.* **Data Quality and Lineage** Familiarity with tools and techniques for data testing, documentation and lineage. Examples include Great Expectations or dbt. ...

quality ETL/ELT pipelines and automated data workflows. Experience integrating heterogeneous data sources, databases and APIs. Experience with workflow orchestration tooling (e.g. Airflow, Dagster or dbt) for scheduling and operating production data pipelines. Good understanding of data quality, metadata, lineage, testing and reproducibility. Ability to evaluate technical options ...

testing and data quality practice. A collaborative mindset and a genuine interest in building things well. Desirable Experience with orchestration tools such as Airflow, Dagster, Prefect or Azure Data Factory. Experience working within AWS, Azure, GCP or another enterprise cloud ecosystem. Experience with complex customer, transactional, digital or behavioural ...

control and code review methodologies. Experience delivering within modern DataOps environments. Data Orchestration & Observability Experience with tools such as: Airflow Azure Data Factory (ADF) Dagster Similar data orchestration and monitoring platforms Business Intelligence Experience with: Power BI Tableau Looker Semantic modelling and metrics layer concepts Agile Delivery Scrum, Kanban ...

Building Extract Transform and Load (ETL) pipeline (streaming/batch) using MPP frameworks (Spark, Beam, or other) Experience in orchestrating complex pipelines using Airflow, Dagster or other Proficiency in one or more programming languages: Python, Java, GO or other Experience with AWS or GCP and their products ...

.Data Platforms: Hands-on experience with Snowflake and data stack technologies like Apache Iceberg and Spark.Workflow Orchestration: Exposure to tools like Apache Airflow, Prefect, Dagster, or DBT.Data Services: Familiarity with AWS Glue, Lake Formation, EMR, EventBridge, Athena, and similar services.Metadata Tools: Experience with tools like Amundsen, Atlas, DataHub, OpenDataDiscovery ...

language used for data processing Batch and event-driven processing. Experience with Kafka or an equivalent streaming system, a workflow orchestrator such as Airflow, Dagster or Prefect, and transformation tooling such as dbt or equivalent SQL-based workflows Cloud warehouse, lake or lakehouse experience , and integrating data from external ...

Transaction Monitoring Screening Risk Ratings Compliance Case Management Strong data governance, lineage and data quality experience Experience with dbt, Airflow, Azure Data Factory, Dagster or similar tooling CI/CD, Git and modern engineering practices Strong stakeholder engagement and communication skills #J-18808-Ljbffr ...

Transaction Monitoring Screening Risk Ratings Compliance Case Management Strong data governance, lineage and data quality experience Experience with dbt, Airflow, Azure Data Factory, Dagster or similar tooling CI/CD, Git and modern engineering practices Strong stakeholder engagement and communication skills Why Apply? Opportunity to help build a greenfield ...

Proven experience designing and deploying ML systems in production (5+ years in relevant roles) Proficiency in Python and SQL, and orchestration tools (Airflow, Kubeflow, Dagster, etc.) Experience with modern cloud platforms (preferably GCP or AWS), Kubernetes, and CI/CD workflows Understanding of ML model lifecycles: training, validation, deployment ...

data pipelines at scale Experience with a modern data stack (e.g. Snowflake, dbt, Fivetran, Debezium or similar) Experience with orchestration tools such as Airflow, Dagster or Prefect Proficiency in Python for building production data workflows Familiarity with data warehousing concepts and modelling patterns Experience working with cloud platforms ...

technical stakeholders to the right data solutions Familiarity with using AI tools to accelerate your workflow Nice to Have Experience with orchestration tools like Dagster, Airflow, or similar Experience building real-time or near-real-time data pipelines Familiarity with machine learning workflows and feature engineering Experience embedded ...

products. You will leverage SQL, Python, and cloud data warehouses (BigQuery, Snowflake, or Redshift) and orchestrate batch or streaming pipelines with Airflow, Prefect, or Dagster, partnering with analytics, data science, and product engineering. #J-18808-Ljbffr ...

clients. The Role You will: Own and develop the infrastructure supporting AI and ML services. Build and maintain orchestration pipelines using tools such as Dagster, Airflow or Prefect. Deploy, manage and optimise workloads in Kubernetes environments. Improve monitoring, observability and platform reliability. Implement infrastructure-as-code and CI/ ...

e.g. AWS, GCP or Azure) Comfortable working in a Linux environment Nice to have: Advanced data engineering experience in Python, e.g. with libraries like Dagster, Prefect etc Experience optimising dataframe code, e.g. in Pandas or ideally Polars Experience of machine learning techniques and related libraries and frameworks e.g. scikit ...

would be nice if you could bring (desirables): Professional experience with PySpark, MySQL, Docker, Kubernetes Experience with best-practice pipeline orchestration software (Airflow, Dagster). Experience migrating code from prototype notebooks to .py, while maintaining logic and transparency of underlying code Owned a project across a whole software product ...

data products, focusing on reliable, scalable architectures. You'll work with SQL, Python, and cloud warehouses (BigQuery, Snowflake, Redshift) and orchestration tools (Airflow, Prefect, Dagster) while collaborating with analytics, data science, and product engineering to deliver impact across the organization. #J-18808-Ljbffr ...

production-grade data pipelines Strong understanding of REST APIs, webhooks, OAuth, and event-driven architectures Familiarity with orchestration tools such as Airflow, Prefect, or Dagster Cloud platform experience with AWS, GCP, or Azure Hands-on experience with containerisation technologies like Docker and Kubernetes Knowledge of data integration patterns ...

take ownership of the data platform, optimising it for scalability to ensure successful client onboarding. You'll use modern tools (such as Airflow, Prefect, Dagster or AWS Step Functions) for ETL design and orchestration, work on transformation logic to clean, validate and enrich data (including handling missing values, standardising ...

code technologies such as Terraform, Bicep, or Pulumi. Knowledge of monitoring and observability tooling such as Prometheus and Grafana. Experience with orchestration platforms including Dagster, Airflow, Prefect, or similar. BENEFITS The successful Senior MLOps Engineer will receive the following benefits: Salary between £75,000 - £85,000 - depending on experience ...

Parquet/Arrow, plus an analytical engine, e.g. DuckDB, or a warehouse such as Snowflake. Orchestrated batch pipelines you've operated, not just written: Dagster or Airflow, S3-based data flows, and a habit of testing outputs for correctness rather than only for exceptions. Applied ML in production: embeddings ...

unfamiliar logic into clean, intuitive data models without hand-holding Bonus Experience with ClickHouse or columnar databases Familiarity with cloud-based data orchestration tools (Dagster, Step Functions) CI/CD practices for data pipelines and transformations Experience operating databases at scale in a production environment Benefits Competitive salary ...