with the ability to independently lead projects and solve complex problems. Nice to Have: Experience with data lake technologies such as Spark , Ray , or Iceberg . Familiarity with Kubernetes for container orchestration. Experience with dbt. Familiarity with Dagster for managing data pipelines. Background or experience in biotech or scientific More ❯
with the ability to independently lead projects and solve complex problems. Nice to Have: Experience with data lake technologies such as Spark , Ray , or Iceberg . Familiarity with Kubernetes for container orchestration. Experience with dbt. Familiarity with Dagster for managing data pipelines. Background or experience in biotech or scientific More ❯
with the ability to independently lead projects and solve complex problems. Nice to Have: Experience with data lake technologies such as Spark , Ray , or Iceberg . Familiarity with Kubernetes for container orchestration. Experience with dbt. Familiarity with Dagster for managing data pipelines. Background or experience in biotech or scientific More ❯
with the ability to independently lead projects and solve complex problems. Nice to Have: Experience with data lake technologies such as Spark , Ray , or Iceberg . Familiarity with Kubernetes for container orchestration. Experience with dbt. Familiarity with Dagster for managing data pipelines. Background or experience in biotech or scientific More ❯
with the ability to independently lead projects and solve complex problems. Nice to Have: Experience with data lake technologies such as Spark , Ray , or Iceberg . Familiarity with Kubernetes for container orchestration. Experience with dbt. Familiarity with Dagster for managing data pipelines. Background or experience in biotech or scientific More ❯
with the ability to independently lead projects and solve complex problems. Nice to Have: Experience with data lake technologies such as Spark , Ray , or Iceberg . Familiarity with Kubernetes for container orchestration. Experience with dbt. Familiarity with Dagster for managing data pipelines. Background or experience in biotech or scientific More ❯
with the ability to independently lead projects and solve complex problems. Nice to Have: Experience with data lake technologies such as Spark , Ray , or Iceberg . Familiarity with Kubernetes for container orchestration. Experience with dbt. Familiarity with Dagster for managing data pipelines. Background or experience in biotech or scientific More ❯
with the ability to independently lead projects and solve complex problems. Nice to Have: Experience with data lake technologies such as Spark , Ray , or Iceberg . Familiarity with Kubernetes for container orchestration. Experience with dbt. Familiarity with Dagster for managing data pipelines. Background or experience in biotech or scientific More ❯
with the ability to independently lead projects and solve complex problems. Nice to Have: Experience with data lake technologies such as Spark , Ray , or Iceberg . Familiarity with Kubernetes for container orchestration. Experience with dbt. Familiarity with Dagster for managing data pipelines. Background or experience in biotech or scientific More ❯
with the ability to independently lead projects and solve complex problems. Nice to Have: Experience with data lake technologies such as Spark , Ray , or Iceberg . Familiarity with Kubernetes for container orchestration. Experience with dbt. Familiarity with Dagster for managing data pipelines. Background or experience in biotech or scientific More ❯
London, England, United Kingdom Hybrid / WFH Options
Workato
data movement, databases (Oracle, SQL Server, PostgreSQL), and cloud analytics platforms (Snowflake, Databricks, BigQuery). Familiarity with emerging data technologies like Open Table Format, ApacheIceberg, and their impact on enterprise data strategies. Hands-on experience with data virtualization and analytics platforms (Denodo, Domo) to enable seamless self More ❯
and designing multi-step pipelines. About you: Proficient in Python Experience in building complex data transformation pipelines Experience with Databricks at scale, preferably with Iceberg Familiarity with Airflow or Dagster Experience with AWS and open-source technologies on top of DataWeave Desirable: Exposure to medical data, especially video/ More ❯
Data Engineer Language Skills (Python, SQL, Javascript) Experience of Azure, AWS or GCP cloud platforms and Data Lake/Warehousing Platforms such as Snowflake, Iceberg etc Experience of various ETL and Streaming Tools (fiveTran, Flink, Spark) Experience of a variety of data mining techniques (APIs, GraphQL, Website Scraping) Ability More ❯
on experience with multi-terabyte scale data processing. Familiarity with AWS; Kubernetes experience is a bonus. Knowledge of data lake technologies such as Parquet, Iceberg, AWS Glue etc. Strong Python software engineering skills. Pragmatic mindset — able to evaluate tradeoffs and find solutions that empower ML researchers to move quickly. More ❯
on experience with multi-terabyte scale data processing. Familiarity with AWS; Kubernetes experience is a bonus. Knowledge of data lake technologies such as Parquet, Iceberg, AWS Glue etc. Strong Python software engineering skills. Pragmatic mindset - able to evaluate tradeoffs find solutions that empower ML researchers to move quickly. Background More ❯
to-end engineering experience supported by excellent tooling and automation. Preferred Qualifications, Capabilities, and Skills: Good understanding of the Big Data stack (Spark/Iceberg). Ability to learn new technologies and patterns on the job and apply them effectively. Good understanding of established patterns, such as stability patterns More ❯
processes using infrastructure-as-code (Terraform) Build and maintain data pipelines using Airflow. Manage our tech stack including Python, Node.js, PostgreSQL, MongoDB, Kafka, and Apache Iceberg. Optimize infrastructure costs and develop strategies for efficient resource utilization. Provide critical support by monitoring services and resolving production issues. Contribute to the More ❯
processes using infrastructure-as-code (Terraform) Build and maintain data pipelines using Airflow. Manage our tech stack including Python, Node.js, PostgreSQL, MongoDB, Kafka, and Apache Iceberg. Optimize infrastructure costs and develop strategies for efficient resource utilization. Provide critical support by monitoring services and resolving production issues. Contribute to the More ❯
Experience working on and shipping live service games. Experience working on Spring Boot projects. Experience deploying software/services on Kubernetes. Experience working with Apache Spark and Iceberg. More ❯
Work closely with stakeholders to translate business needs into scalable solutions Tech environment includes Python, SQL, dbt, Databricks, BigQuery, Delta Lake, Spark, Kafka, Parquet, Iceberg (If you haven’t worked with every tool, that’s totally fine — my client values depth of thinking and engineering craft over buzzword familiarity. More ❯
Work closely with stakeholders to translate business needs into scalable solutions Tech environment includes Python, SQL, dbt, Databricks, BigQuery, Delta Lake, Spark, Kafka, Parquet, Iceberg (If you haven’t worked with every tool, that’s totally fine — my client values depth of thinking and engineering craft over buzzword familiarity. More ❯
analysis and automation. Proficiency in building and maintaining batch and streaming ETL/ELT pipelines at scale, employing tools such as Airflow, Fivetran, Kafka, Iceberg, Parquet, Spark, Glue for developing end-to-end data orchestration leveraging on AWS services to ingest, transform and process large volumes of structured and More ❯
London, England, United Kingdom Hybrid / WFH Options
Automata
analysis and automation. Proficiency in building and maintaining batch and streaming ETL/ELT pipelines at scale, employing tools such as Airflow, Fivetran, Kafka, Iceberg, Parquet, Spark, Glue for developing end-to-end data orchestration leveraging on AWS services to ingest, transform and process large volumes of structured and More ❯
using tools and techniques such as BDD, Data Reconciliation, Source Control, TDD, Jenkins. Documenting configurations, processes, and best practices. Knowledge of file formats JSON, Iceberg, Avro. Basic knowledge of AWS technologies like IAM roles, Lakeformation, Security Groups, CloudFormation, Redshift. Big Data/Data Warehouse testing experience. Experience in the More ❯
London, England, United Kingdom Hybrid / WFH Options
DoiT International
production experience with K8S. Hands-on, production experience with cloud (AWS, Azure, GCP). Experience with data engineering - streaming and batch processing, spark, tryno, iceberg, clickhouse, parquet. Be your truest self. Work on your terms. Make a difference. We are home to a global team of incredible talent who More ❯