1 to 25 of 32 Permanent Prefect Jobs in London

data quality practice. A collaborative mindset and a genuine interest in building things well. Desirable Experience with orchestration tools such as Airflow, Dagster, Prefect or Azure Data Factory. Experience working within AWS, Azure, GCP or another enterprise cloud ecosystem. Experience with complex customer, transactional, digital or behavioural data. Exposure ...

Cloud Platform ecosystem, like Cloud Run, Cloud Functions, Cloud Storage, Pub/Sub, and Cloud Scheduler Data orchestration tools such as Cloud Composer (Airflow), Prefect, or equivalent Looker/Data Studio and “Conventional analytics” (aka. BQ agenta) Building or maintaining a data warehouse in a regulated or healthcare environment GDPR ...

columnar data formats like Parquet. MLOps Exposure: Experience with tools and frameworks for training, tracking, deploying, and monitoring models in production (e.g., MLflow, Optuna, Prefect, or Airflow). Background in supply chain forecasting, enterprise AI, or retail analytics M.S. or Ph.D. in Computer Science, Artificial Intelligence, Statistics, Data Science ...

data transformation tools (such as dbt). Pipeline mastery: Practical experience building and operating batch or streaming pipelines using orchestration tools like Airflow, Prefect, or Dagster. Strong communicator: Skilled at writing clear technical specifications and explaining complex trade-offs to both technical and non-technical colleagues. Growth mindset: Open ...

pipelines for reliability, monitoring and scale. Requirements: Production data platforms with high-volume ETL/ELT, distributed processing and orchestration (e.g. Airflow, Dagster, Prefect), as distinct from analytics pipelines. Python and SQL at a high level, plus Bash automation. Cloud infrastructure, with AWS preferred (S3, Lambda, ECS, IAM); Snowflake ...

debug a feature pipeline with an engineer in the same afternoon. **Nice-to-have*** Experience with n8n or other workflow automation tools (Airflow, Prefect, Dagster).* Exposure to payments, fintech, SaaS, or other recurring-revenue businesses.* DBT experience for analytics engineering.* Experience integrating LLMs or GenAI into commercial workflows. ...

payments or fintech industry. Nice to Have Running multi‐tenant AI platforms or GPU/resource management at scale. Data pipelines and orchestration (Airflow, Prefect) and warehouses like Databricks, Snowflake, or BigQuery. Spanish proficiency. Tech Stack Languages — Python (AI/ML), Go and Kotlin (platform services). AI & LLMs ...

data processing Batch and event-driven processing. Experience with Kafka or an equivalent streaming system, a workflow orchestrator such as Airflow, Dagster or Prefect, and transformation tooling such as dbt or equivalent SQL-based workflows Cloud warehouse, lake or lakehouse experience , and integrating data from external APIs, databases and files ...

knowledge of AWS/cloud services (ECS, EKS, Lambda, Kinesis, DynamoDB, EMR, Athena) Hands-on experience with orchestration and workflow tools (Docker, Kubernetes, Airflow, Prefect, Dagster) Familiar with CI/CD pipelines, infrastructure as code, and DBT Able to explain complex data concepts clearly to non-technical stakeholders Why This ...

scale Experience with a modern data stack (e.g. Snowflake, dbt, Fivetran, Debezium or similar) Experience with orchestration tools such as Airflow, Dagster or Prefect Proficiency in Python for building production data workflows Familiarity with data warehousing concepts and modelling patterns Experience working with cloud platforms and infrastructure tooling Comfortable working ...

experience: modelling, semantic layers, agents & skills, testing, performance and cost tuning. Production experience with a pipeline orchestrator, ideally Dagster (or similar Airflow, Prefect), including dependent workloads, partial failures, and observability. Modern cloud infrastructure (Azure/AWS/GCP) and Kubernetes experience deploying and operating containerised workloads, plus infrastructure as code ...

platforms Experience briefing, designing and implementing ETL solutions end-to-end, from requirements gathering through to deployment Experience with data orchestration tools (e.g. Airflow, Prefect) and building scalable data pipelines Strong Git and collaborative development skills (e.g. GitHub, GitLab) Strong understanding of data fundamentals: MDM, data integrity best practices, auditing ...

.Data Platforms: Hands-on experience with Snowflake and data stack technologies like Apache Iceberg and Spark.Workflow Orchestration: Exposure to tools like Apache Airflow, Prefect, Dagster, or DBT.Data Services: Familiarity with AWS Glue, Lake Formation, EMR, EventBridge, Athena, and similar services.Metadata Tools: Experience with tools like Amundsen, Atlas, DataHub, OpenDataDiscovery ...

Azure) Comfortable working in a Linux environment Nice to have: Advanced data engineering experience in Python, e.g. with libraries like Dagster, Prefect etc Experience optimising dataframe code, e.g. in Pandas or ideally Polars Experience of machine learning techniques and related libraries and frameworks e.g. scikit-learn, Pytorch, Tensorflow etc Experience ...

workload, and strengthen data quality and retention practices. Projects could include moving large and non‐operational data out of PostgreSQL, improving pipeline orchestration with Prefect, and evolving our search and analytical capabilities with technologies such as OpenSearch and column databases. You’ll bring the experience and judgement to assess ...

building scalable data assets that can be used in BI and other user facing tools. Experience working with data pipeline orchestration tools like Prefect or Airflow and comfortable working with Python for data orchestration. Ability to communicate clearly with both technical and non-technical stakeholders, translating data into business value. ...

equivalent technologies. Experience implementing monitoring and observability solutions using tools such as Prometheus, Grafana, or similar. Familiarity with orchestration platforms including Dagster, Airflow, Prefect, or related technologies. Desirable experience includes: Model serving infrastructure for real-time or batch inference workloads. Multi-tenant platform environments. Vector databases such as Qdrant. Development ...

databases. Experience designing data models, schemas and transformation layers for complex datasets. Experience with directed acyclic graph–based orchestration tools such as Airflow, Dagster, Prefect or similar systems. Familiarity with data warehouses, object storage, queues, APIs and cloud infrastructure. Experience with data quality testing, lineage, observability, versioning or change‐data ...

data products, focusing on reliable, scalable architectures. You'll work with SQL, Python, and cloud warehouses (BigQuery, Snowflake, Redshift) and orchestration tools (Airflow, Prefect, Dagster) while collaborating with analytics, data science, and product engineering to deliver impact across the organization. #J-18808-Ljbffr ...

years of data engineering experience with Spark, strong Python and SQL, extensive AWS services, and hands-on experience with orchestration tools (Docker, Kubernetes, Airflow, Prefect, Dagster). Hybrid work model in London. #J-18808-Ljbffr ...

automationWorking knowledge of modern MLOps stacks including experiment tracking and artifact managementHands-on experience with orchestration tools (e.g., Kubeflow, Apache Airflow, Metaflow or Prefect)Demonstrated skills with Docker, Kubernetes and distributed deploymentsPractical knowledge of Infrastructure-as-Code (Terraform) and a major cloud provider (AWS, Azure or GCP)Familiarity with ...

programming skills and production experience in Python. Experience building and maintaining data or ML pipelines with an orchestration tool such as Dagster (or Airflow, Prefect, etc.). Hands-on experience with common ML libraries and frameworks, e.g. PyTorch, scikit-learn, and gradient-boosting libraries such as XGBoost or LightGBM. Clear ...

scale Strong software engineering skills Experience delivering & maintaining production systems Experience with workflow orchestration for parallelised jobs; Flyte preferred, but Airflow, Prefect or similar tools are also relevant Experience building reliable, observable data systems, including retry strategies, backfills, data integrity checks, lifecycle/state tracking and operational alerting Strong cloud ...

threading Profiling Python/C/C++ and performance optimisation Networking Nice to have: Data engineering experience in Python, e.g. with libraries like Dagster, Prefect etc Experience optimising dataframe code, e.g. in Pandas or ideally Polars Experience of machine learning techniques and related libraries and frameworks e.g. scikit-learn, Pytorch ...

they do. Our tech stack is built of strongly typed Python, TypeScript, Next.js, React.js, MariaDB and Redis. Our data platform is built using DBT, Prefect, Fivetran and Snowflake. Our production services run on AWS and Vercel. We automate infrastructure and configuration using Pulumi and GitHub Actions. Our observability stack ...