ELT processes, and Lakehouse concepts. Experience with data quality frameworks, data governance, and compliance requirements. Familiarity with version control (Git), CI/CD pipelines, and workflow orchestration tools (Airflow, Prefect). Soft Skills Strong analytical and problem-solving mindset with attention to detail. Good team player with effective communication and storytelling with data and insights. Consulting skills, including development of More ❯
Engineer , Data Manager, or similar leadership role. Strong proficiency in Python (or Scala/Java) and SQL . Hands-on experience with data orchestration tools (Airflow, dbt, Dagster, or Prefect). Solid understanding of cloud data platforms (AWS, GCP, or Azure) and data warehousing (Snowflake, BigQuery, Redshift). Experience with streaming technologies (Kafka, Kinesis, or similar). Strong knowledge of More ❯
Hands-on experience with Snowflake Production experience with dbt (mandatory) Strong SQL and Python programming skills Experience with Git-based workflows and DevOps practices Familiarity with orchestration tools (Airflow, Prefect) and ETL/ELT patterns Knowledge of cloud platforms (AWS, Azure) and security best practices More ❯
London, South East, England, United Kingdom Hybrid/Remote Options
Robert Half
for improved performance, cost efficiency, and scalability.* Troubleshoot and resolve data pipeline issues swiftly and effectively across the data platform.* Work with orchestration tools such as Airflow, ADF, or Prefect to schedule and automate workflows.* Keep abreast of industry trends and emerging technologies in data engineering, and continuously improve your skills and knowledge. Profile * Minimum 3 years' experience working as More ❯
years' experience as a Data Engineer Strong SQL and Python skills Proven experience building modern ETL/ELT pipelines (dbt experience ideal) Experience with data orchestration tools (Prefect preferred) Understanding of data modelling , especially event-driven architectures Knowledge of modern data engineering development practices Nice to have: Background in InsurTech/FinTech or regulated industries Experience with Docker , containerisation, and More ❯
data warehousing and transformation. Strong SQL skills and understanding of modern data architecture principles. Hands-on experience with Tableau for enterprise-grade dashboard development. Familiarity with orchestration tools (e.g., Prefect, Airflow), data quality frameworks, and metadata tools. Proficiency in Git, CI/CD, and scripting with Python. Excellent communication skills and ability to work collaboratively across technical and business teams. More ❯
complex business or security processes and build intelligent solutions Solid understanding of software security principles, secure coding practices, and security automation Experience with data orchestration frameworks (e.g., Airflow, Dagster, Prefect) to manage and scale automation pipelines Familiarity with infrastructure-as-code (Terraform), CI/CD systems (BuildKite or similar), and container orchestration platforms (Kubernetes, Docker) Strong problem-solving, communication, and More ❯
complex business or security processes and build intelligent solutions Solid understanding of software security principles, secure coding practices, and security automation Experience with data orchestration frameworks (e.g., Airflow, Dagster, Prefect) to manage and scale automation pipelines Familiarity with infrastructure-as-code (Terraform), CI/CD systems (BuildKite or similar), and container orchestration platforms (Kubernetes, Docker) Strong problem-solving, communication, and More ❯
complex business or security processes and build intelligent solutions Solid understanding of software security principles, secure coding practices, and security automation Experience with data orchestration frameworks (e.g., Airflow, Dagster, Prefect) to manage and scale automation pipelines Familiarity with infrastructure-as-code (Terraform), CI/CD systems (BuildKite or similar), and container orchestration platforms (Kubernetes, Docker) Strong problem-solving, communication, and More ❯
complex business or security processes and build intelligent solutions Solid understanding of software security principles, secure coding practices, and security automation Experience with data orchestration frameworks (e.g., Airflow, Dagster, Prefect) to manage and scale automation pipelines Familiarity with infrastructure-as-code (Terraform), CI/CD systems (BuildKite or similar), and container orchestration platforms (Kubernetes, Docker) Strong problem-solving, communication, and More ❯
complex business or security processes and build intelligent solutions Solid understanding of software security principles, secure coding practices, and security automation Experience with data orchestration frameworks (e.g., Airflow, Dagster, Prefect) to manage and scale automation pipelines Familiarity with infrastructure-as-code (Terraform), CI/CD systems (BuildKite or similar), and container orchestration platforms (Kubernetes, Docker) Strong problem-solving, communication, and More ❯
complex business or security processes and build intelligent solutions Solid understanding of software security principles, secure coding practices, and security automation Experience with data orchestration frameworks (e.g., Airflow, Dagster, Prefect) to manage and scale automation pipelines Familiarity with infrastructure-as-code (Terraform), CI/CD systems (BuildKite or similar), and container orchestration platforms (Kubernetes, Docker) Strong problem-solving, communication, and More ❯
complex business or security processes and build intelligent solutions Solid understanding of software security principles, secure coding practices, and security automation Experience with data orchestration frameworks (e.g., Airflow, Dagster, Prefect) to manage and scale automation pipelines Familiarity with infrastructure-as-code (Terraform), CI/CD systems (BuildKite or similar), and container orchestration platforms (Kubernetes, Docker) Strong problem-solving, communication, and More ❯
complex business or security processes and build intelligent solutions Solid understanding of software security principles, secure coding practices, and security automation Experience with data orchestration frameworks (e.g., Airflow, Dagster, Prefect) to manage and scale automation pipelines Familiarity with infrastructure-as-code (Terraform), CI/CD systems (BuildKite or similar), and container orchestration platforms (Kubernetes, Docker) Strong problem-solving, communication, and More ❯
complex business or security processes and build intelligent solutions Solid understanding of software security principles, secure coding practices, and security automation Experience with data orchestration frameworks (e.g., Airflow, Dagster, Prefect) to manage and scale automation pipelines Familiarity with infrastructure-as-code (Terraform), CI/CD systems (BuildKite or similar), and container orchestration platforms (Kubernetes, Docker) Strong problem-solving, communication, and More ❯
complex business or security processes and build intelligent solutions Solid understanding of software security principles, secure coding practices, and security automation Experience with data orchestration frameworks (e.g., Airflow, Dagster, Prefect) to manage and scale automation pipelines Familiarity with infrastructure-as-code (Terraform), CI/CD systems (BuildKite or similar), and container orchestration platforms (Kubernetes, Docker) Strong problem-solving, communication, and More ❯
complex business or security processes and build intelligent solutions Solid understanding of software security principles, secure coding practices, and security automation Experience with data orchestration frameworks (e.g., Airflow, Dagster, Prefect) to manage and scale automation pipelines Familiarity with infrastructure-as-code (Terraform), CI/CD systems (BuildKite or similar), and container orchestration platforms (Kubernetes, Docker) Strong problem-solving, communication, and More ❯
develop CI/CD pipelines for research processes and live trading apps develop robust monitoring solutions for infrastructure and deployed applications automate recurring jobs with tools like Airflow/Prefect Performance engineering: Be familiar with best practices for profiling, monitoring performance to assist with performance investigations Develop solutions that empower researchers and developers to understand the performance of their code More ❯
London, South East, England, United Kingdom Hybrid/Remote Options
Adecco
data workflows, optimize performance, and ensure reliability across cloud platforms-while guiding teams with strong technical leadership.Key Responsibilities* Pipeline Automation: Build and orchestrate data workflows using tools like Airflow, Prefect, or Dagster.* Containerization: Package and deploy data applications using Docker and Kubernetes (including EKS and AKS).* CI/CD for Data: Implement and maintain automated pipelines for data applications. … Terraform and Ansible to provision and manage data infrastructure.* Performance Optimization: Enhance data processing for speed, scalability, and reliability.What We're Looking For* Strong experience with orchestration tools (Airflow, Prefect, Dagster).* Expertise in Docker and Kubernetes.* Solid understanding of CI/CD principles and tooling.* Familiarity with open-source data technologies (Spark, Kafka, PostgreSQL).* Knowledge of Infrastructure as … application with you before presenting it to any potential employer.Please note we are on the client's supplier list for this position.KeywordsLead DataOps Engineer, DataOps, Data Pipeline Automation, Airflow, Prefect, Dagster, Docker, Kubernetes, EKS, AKS, CI/CD, Terraform, Ansible, Grafana, Prometheus, Spark, Kafka, PostgreSQL, Infrastructure as Code, Cloud Data Engineering, Hybrid Working, Security Clearance, Leadership, DevOps, Observability, Monitoring. More ❯
London, South East, England, United Kingdom Hybrid/Remote Options
Robert Half
needed). The Role Build and optimise scalable data pipelines for a new Data Lake Work with Python, SQL, Spark, AWS, Docker, CI/CD Orchestrate workflows using Airflow, Prefect, or Dagster Support ETL, API integrations, and high-quality data validation Translate business needs into technical solutions with full ownership What You Need STEM degree (Computer Science, Engineering, Maths, etc. … years in data engineering/analytics/software Strong Python + SQL 2+ years ETL, APIs, CI/CD, Docker, AWS Ideally but not essential experience with Airflow/Prefect/Dagster Background in Financial Services, FinTech, Insurance, PE/VC, or Banking Package & Benefits High performance bonus 25 days holiday Pension & private healthcare Join at inception - shape the team More ❯
Astronomer empowers data teams to bring mission-critical software, analytics, and AI to life and is the company behind Astro, the industry-leading unified DataOps platform powered by Apache Airflow. Astro accelerates building reliable data products that unlock insights, unleash More ❯
