on experience with multi-terabyte scale data processing. Familiarity with AWS; Kubernetes experience is a bonus. Knowledge of data lake technologies such as Parquet, Iceberg, AWS Glue etc. Strong Python software engineering skills. Pragmatic mindset — able to evaluate tradeoffs and find solutions that empower ML researchers to move quickly. More ❯
on experience with multi-terabyte scale data processing. Familiarity with AWS; Kubernetes experience is a bonus. Knowledge of data lake technologies such as Parquet, Iceberg, AWS Glue etc. Strong Python software engineering skills. Pragmatic mindset - able to evaluate tradeoffs find solutions that empower ML researchers to move quickly. Background More ❯
to-end engineering experience supported by excellent tooling and automation. Preferred Qualifications, Capabilities, and Skills: Good understanding of the Big Data stack (Spark/Iceberg). Ability to learn new technologies and patterns on the job and apply them effectively. Good understanding of established patterns, such as stability patterns More ❯
processes using infrastructure-as-code (Terraform) Build and maintain data pipelines using Airflow. Manage our tech stack including Python, Node.js, PostgreSQL, MongoDB, Kafka, and Apache Iceberg. Optimize infrastructure costs and develop strategies for efficient resource utilization. Provide critical support by monitoring services and resolving production issues. Contribute to the More ❯
processes using infrastructure-as-code (Terraform) Build and maintain data pipelines using Airflow. Manage our tech stack including Python, Node.js, PostgreSQL, MongoDB, Kafka, and Apache Iceberg. Optimize infrastructure costs and develop strategies for efficient resource utilization. Provide critical support by monitoring services and resolving production issues. Contribute to the More ❯
Experience working on and shipping live service games. Experience working on Spring Boot projects. Experience deploying software/services on Kubernetes. Experience working with Apache Spark and Iceberg. More ❯
Work closely with stakeholders to translate business needs into scalable solutions Tech environment includes Python, SQL, dbt, Databricks, BigQuery, Delta Lake, Spark, Kafka, Parquet, Iceberg (If you haven’t worked with every tool, that’s totally fine — my client values depth of thinking and engineering craft over buzzword familiarity. More ❯
Work closely with stakeholders to translate business needs into scalable solutions Tech environment includes Python, SQL, dbt, Databricks, BigQuery, Delta Lake, Spark, Kafka, Parquet, Iceberg (If you haven’t worked with every tool, that’s totally fine — my client values depth of thinking and engineering craft over buzzword familiarity. More ❯
analysis and automation. Proficiency in building and maintaining batch and streaming ETL/ELT pipelines at scale, employing tools such as Airflow, Fivetran, Kafka, Iceberg, Parquet, Spark, Glue for developing end-to-end data orchestration leveraging on AWS services to ingest, transform and process large volumes of structured and More ❯
using tools and techniques such as BDD, Data Reconciliation, Source Control, TDD, Jenkins. Documenting configurations, processes, and best practices. Knowledge of file formats JSON, Iceberg, Avro. Basic knowledge of AWS technologies like IAM roles, Lakeformation, Security Groups, CloudFormation, Redshift. Big Data/Data Warehouse testing experience. Experience in the More ❯
London, England, United Kingdom Hybrid / WFH Options
DoiT International
production experience with K8S. Hands-on, production experience with cloud (AWS, Azure, GCP). Experience with data engineering - streaming and batch processing, spark, tryno, iceberg, clickhouse, parquet. Be your truest self. Work on your terms. Make a difference. We are home to a global team of incredible talent who More ❯
of data technologies ideally with experience with relational databases, big data, data warehouses and marts, and stream processing technologies (e.g. Kafka, S3, Flink, Snowflake, Iceberg) Product-first mindset and a desire to design and build tools that solve real user problems Mastery of fundamental software engineering practices like good More ❯
/or Python. Experience with tools like Terraform to provision AWS cloud services. Knowledge of AWS Glue, AWS Athena, and AWS S3. Understanding of Apache Parquet and open table formats such as Delta, Iceberg, and Hudi. Experience with Test Driven Development using JUnit, Mojito, or similar tools. Extensive More ❯
Glasgow, Scotland, United Kingdom Hybrid / WFH Options
JR United Kingdom
big data technologies (e.g., Spark, Databricks, Delta Lake, BigQuery) Familiarity with eventing technologies (e.g., Event Hubs, Kafka) and file formats such as Parquet, Delta, Iceberg Want to learn more? Get in touch for an informal chat. #J-18808-Ljbffr More ❯
big data technologies (e.g., Spark, Databricks, Delta Lake, BigQuery) Familiarity with eventing technologies (e.g., Event Hubs, Kafka) and file formats such as Parquet, Delta, Iceberg Want to learn more? Get in touch for an informal chat. #J-18808-Ljbffr More ❯
big data technologies (e.g., Spark, Databricks, Delta Lake, BigQuery) Familiarity with eventing technologies (e.g., Event Hubs, Kafka) and file formats such as Parquet, Delta, Iceberg Interested in learning more? Get in touch for an informal chat. #J-18808-Ljbffr More ❯
big data technologies (e.g., Spark, Databricks, Delta Lake, BigQuery) Familiarity with eventing technologies (e.g., Event Hubs, Kafka) and file formats such as Parquet, Delta, Iceberg Interested in learning more? Get in touch for an informal chat. #J-18808-Ljbffr More ❯
big data technologies (e.g., Spark, Databricks, Delta Lake, BigQuery) Familiarity with eventing technologies (e.g., Event Hubs, Kafka) and file formats such as Parquet, Delta, Iceberg Interested in learning more? Get in touch for an informal chat. #J-18808-Ljbffr More ❯
Databricks/Delta Lake/BigQuery) Familiarity with eventing technologies (e.g., Event Hubs/Kafka) and file formats such as Parquet/Delta/Iceberg Want to learn more? Get in touch for an informal chat. #J-18808-Ljbffr More ❯
big data technologies (e.g., Spark, Databricks, Delta Lake, BigQuery) Familiarity with eventing technologies (e.g., Event Hubs, Kafka) and file formats such as Parquet, Delta, Iceberg If you're interested in learning more, get in touch for an informal chat. #J-18808-Ljbffr More ❯
Nottingham, England, United Kingdom Hybrid / WFH Options
JR United Kingdom
big data technologies (e.g., Spark, Databricks, Delta Lake, BigQuery) Familiarity with eventing technologies (e.g., Event Hubs, Kafka) and file formats such as Parquet, Delta, Iceberg Interested in learning more? Get in touch for an informal chat. #J-18808-Ljbffr More ❯
Sheffield, England, United Kingdom Hybrid / WFH Options
JR United Kingdom
big data technologies (e.g., Spark, Databricks, Delta Lake, BigQuery) Familiarity with eventing technologies (e.g., Event Hubs, Kafka) and file formats such as Parquet, Delta, Iceberg Interested in learning more? Get in touch for an informal chat. #J-18808-Ljbffr More ❯
Stoke-on-Trent, England, United Kingdom Hybrid / WFH Options
JR United Kingdom
big data technologies (e.g., Spark, Databricks, Delta Lake, BigQuery) Familiarity with eventing technologies (e.g., Event Hubs, Kafka) and file formats such as Parquet, Delta, Iceberg Want to learn more? Get in touch for an informal chat. #J-18808-Ljbffr More ❯
Your day-to-day -Wrangle and draw meaningful insights from massive amounts of unstructured textual data using the latest tools and technologies like Spark, Iceberg, Athena, AWS SageMaker -Apply unsupervised learning algorithms across billions of email interactions to identify emerging threat patterns -Work with state-of-the-art machine More ❯
the future of the AI Data Cloud. Join the Snowflake team. Snowflake is seeking an accomplished Principal Sales Specialist, Data Engineering Platform (Openflow and Iceberg/Polaris) to drive sales execution for our ingestion workloads for the EMEA markets. Reporting to the Managing Direction, Data Engineering Platform Sales, this More ❯