26 to 49 of 49 Apache Iceberg Jobs in London

Terraform, Kafka/MSK, Airflow, Glue or Spark, and building scalable data solutions Experience with modern data ecosystems including data lakes/lakehouse architectures, Iceberg or similar table formats, as well as batch and streaming processing Knowledge of data quality, governance, cataloguing and observability tools (e.g. Datadog), with ...

Terraform, Kafka/MSK, Airflow, Glue or Spark, and building scalable data solutions Experience with modern data ecosystems including data lakes/lakehouse architectures, Iceberg or similar table formats, as well as batch and streaming processing Knowledge of data quality, governance, cataloguing and observability tools (e.g. Datadog), with ...

Terraform, Kafka/MSK, Airflow, Glue or Spark, and building scalable data solutions Experience with modern data ecosystems including data lakes/lakehouse architectures, Iceberg or similar table formats, as well as batch and streaming processing Knowledge of data quality, governance, cataloguing and observability tools (e.g. Datadog), with ...

Data & Analytics, with proven experience as a Senior/Tech Lead setting architectural direction for other engineersDeep, hands-on knowledge of Snowflake, including ideally Iceberg/open table formats and RBAC.Advanced SQL, Python, and enterprise-grade dbt model development. Experience designing end-to-end data pipelines. Combining batch, file ...

e.g., Java, TypeScript, SQL) Experience building and consuming APIs and event‐driven services in a cloud environment Experience with data processing frameworks such as Apache Spark and working with large structured datasets Working knowledge of LLM‐based application development, including prompting, tool calling, retrieval, and evaluation Experience using approved … agent frameworks (e.g., Google ADK, LangGraph) and agent protocols such as MCP, A2A, or AG-UI Experience with Databricks, MLflow, and Delta Lake or Apache Iceberg Exposure to model serving on Kubernetes and to model monitoring (drift, latency, accuracy) Familiarity with CRM platforms such as Salesforce and their ...

data generation for code or language models. Training LLMs in distributed environments. Reinforcement learning, preference optimisation or reward modelling. Data tooling such as SQL, Apache Iceberg or DuckDB. LLM‐as‐a‐judge systems or automated code evaluation. Reward‐hacking detection, robustness evaluation or safety‐focused training. Open‐source ...

Matillion/Fivetran, Kafka/Kinesis/Event Hubs or similarExperience with Snowflake capabilities such as Snowpark, Streams & Tasks, Dynamic Tables, Data Sharing or Iceberg tables, CoCo, CoWork, Cortex Analyst and Search and Semantic ViewsExperience enabling ML/AI workloads through feature-ready datasets, governed access patterns and collaboration ...

Tool (DBT) with a demonstrated ability indeveloping data models, contracts, tests, validation, and transformations Experienceworking with modern data distributed file formats (i.e., Parquet, Delta,Iceberg, Hudi) Demonstratedexperience with building data ingestion pipelines from REST API data sources. Strongability to produce technical documentation that can be understood by bothtechnical ...

required, including 5 years' UK residency; successfully obtaining security clearance is a condition of employment These will help you succeed Strong working knowledge of Apache Iceberg and open table formats, with the ability to architect interoperable strategies that avoid vendor lock-in Practical experience designing or transitioning organisations ...

implementing computational improvements Making improvements and/or fixing bugs by submitting patches to open-source projects in our stack, such as Spark, Iceberg, Hadoop, Airflow, and many others Acting as an expert on Spark and Scala Mentoring engineers (including through pairing, design discussions, and knowledge-sharing sessions ...

self‐service analytics. Select and design for the right storage and exchange formats for each workload, including row‐based and columnar storage, Delta and Iceberg table formats, and file and message formats (e.g. Parquet, CSV, Excel, XML, JSON, Protobuf). Optimise platform performance and cost, including partitioning vs. liquid … designing metadata‐driven data ingestion or ELT frameworks. Experience working with a range of data formats, including row‐based and columnar storage, Delta and Iceberg table formats, and common file formats (e.g. CSV, XML, JSON). Experience applying data governance and security practices on a lakehouse platform, including access ...

services, a good understanding of cloud architecture, and proficiency with Terraform for provisioning and managing cloud infrastructure Experience building streaming data pipelines using Apache Flink, Apache Kafka, Spark Structured Streaming, or similar technologies Knowledge of modern data lakehouse architectures and ecosystem such as Kafka, Flink, Spark, Databricks, Snowflake … Airflow, Debezium, Delta/Iceberg, and comfort with Python/Java and SQL Experience with networking and security concepts within AWS, including VPCs, subnets, routing, security groups, IAM, etc. Nice to have Experience: Exposure to or interest in the cryptocurrency technology ecosystem Experience working with different blockchain technologies ...

cloud environments. You will enable real-time analytics and governance while promoting self-service insights for business users. Key skills include data lakehouse design (Iceberg), AWS/Azure data services, dbt, Airflow, and BI/dashboard experience. The role emphasizes automation, quality, and collaboration with architects. #J-18808-Ljbffr ...

value to real users. Nice to have Experience working with scientific or scholarly text. Familiarity with AWS (S3, Batch, Lambda, SageMaker) and Parquet or Iceberg data lake patterns. Experience running LLMs under real cost and latency budgets in production. Some exposure to agentic AI applications: tool use, multi‐step ...

latency software systems Experience with pipeline orchestration frameworks (e.g. Airflow, Dagster) Experience with streaming platforms (e.g. Kafka), data lake platforms (e.g. Delta Lake, Apache Iceberg), and relational databases Have a Bachelors or advanced degree in Computer Science, Mathematics, Statistics, Physics, Engineering, or equivalent work experience For more information ...

monitoring Solid understanding of cloud-native infrastructure and data engineering Someone comfortable debugging complex production issues and making architectural decisions independently Experience with Kafka, Iceberg, Athena or Parquet would be a bonus. You'll be the go-to engineer for MLOps within a lean technical team, with genuine ownership ...

markets experience Term structure modelling, stochastic calculus or Monte Carlo methods Interest rate curve bootstrapping Algorithmic execution experience Data lake or warehouse experience (Snowflake, Iceberg, Spark etc.) A product mindset - you think about who uses your systems and why. Clear communicator who can work cross-functionally with other quants ...

deep experience in Linux-based high-throughput environments, plus exposure to streaming platforms like Kafka and data lake technologies such as Delta Lake or Apache Iceberg. #J-18808-Ljbffr ...

Experience of working in an Agile delivery team Strong analytic skills related to working with unstructured datasets Python (PySpark, Pandas, PyArrow) Distributed data processing (Apache Spark) Data ETL (Apache Airflow, AWS Step Functions, Apache NiFi) Cloud services (AWS, Azure or GCP) Messaging/Streaming (Kafka … Other Cloud Queuing Native services) SQL and NoSQL Storage (HDFS, Iceberg, Elastic, S3, Data Lake) Containerisation and orchestration (Docker/Kubernetes/Openshift) Testing frameworks and best practices Use of Code repositories, familiarity with branching strategies, pull requests and merge processes Benefits £: Competitive salary Contributory Pension Scheme ...

Experience of working in an Agile delivery team Strong analytic skills related to working with unstructured datasets Python (PySpark, Pandas, PyArrow) Distributed data processing (Apache Spark) Data ETL (Apache Airflow, AWS Step Functions, Apache NiFi) Cloud services (AWS, Azure or GCP) Messaging/Streaming (Kafka … Other Cloud Queuing Native services) SQL and NoSQL Storage (HDFS, Iceberg, Elastic, S3, Data Lake) Containerisation and orchestration (Docker/Kubernetes/Openshift) Testing frameworks and best practices Use of Code repositories, familiarity with branching strategies, pull requests and merge processes We appreciate you may not be an expert ...

bandwidth. The goal: no software rewrites, predictable time to production, early integration, and fast iteration. You'll work with tools such as DBT, Python, Apache Airflow, Spark, Iceberg; AWS services like Lambda, Bedrock, and SageMaker, containers, and whatever adds the most value. You'll design tools that ...

working with some of them is desirable GCP Cloud: Compute Engine, Kubernetes Engine, IAM, VPC Networks, BigQuery, Dataproc, CloudStorage, Cloud Functions, CloudSQL, Memorystore PubSub, Iceberg Infrastructure & Technologies: Apache Airflow, Druid/Imply, Kafka, Spark, Postgres, Jenkins, Scylla, Redis, Docker, Prometheus, Github, Linux. Languages: Python, Scala, C++, JavaScript, Java. ...

engines and planning capabilities across large tables and diverse workloads. You will work with Java, Rust, Scala, or C++ and engage with Spark, DataFusion, Iceberg, Arrow, and Parquet in a high-performance environment that values #J-18808-Ljbffr ...

mindset focused on automation, reliability, and maintainable solutions.## **Preferred Qualifications*** Experience with Argo CD, Helm, or other GitOps tooling.* Familiarity with Airflow, Snowflake, or Apache Iceberg.* Experience with Apache Kafka or Amazon MSK.* Experience with hybrid cloud, enterprise networking, or multi-account AWS environments.* Experience with Ansible ...