1 to 25 of 27 Apache Iceberg Jobs in London

effectively with stakeholders and champion better ways of working. Nice to Have Experience with Docker, Java/Scala, Kafka, Pulsar, Azure Event Hub, Collibra, Apache Iceberg, Apache Arrow, PySpark or AI engineering technologies such as Snowflake AI, LangChain, LlamaIndex or Pinecone would be advantageous. This ...

customer requirements and translate them into practical technical solutions.Excellent analytical, problem-solving and communication skills.Desirable experienceData-lake or lakehouse technologies such as Delta Lake, Apache Iceberg, Apache Hudi, Databricks, Snowflake or AWS Lake Formation.Image or video visualisation, enhancement, rendering or tagging applications.GIS or geo-referenced imagery.Event-driven ...

translate them into practical technical solutions. Excellent analytical, problem-solving and communication skills. Desirable experienceData-lake or lakehouse technologies such as Delta Lake, Apache Iceberg, Apache Hudi, Databricks, Snowflake or AWS Lake Formation. Image or video visualisation, enhancement, rendering or tagging applications. GIS or geo-referenced imagery. ...

Modern data platforms: Databricks, Snowflake and cloud-native data services Data architecture: Lakehouse, medallion, data mesh and domain-oriented patterns Data processing: Apache Spark, SQL and distributed processing technologies Data storage: Delta Lake, Apache Iceberg and open table formats Data pipelines: Enterprise-scale ETL/ELT, batch … streaming patterns Transformation and quality: dbt, Great Expectations and equivalent tooling Orchestration: Apache Airflow and cloud-native orchestration services Event streaming: Kafka and cloud-native event-streaming technologies Data modelling: Dimensional, analytical and domain-oriented modelling Data governance: Catalogue, metadata, lineage, quality, security, privacy and access controls Data products ...

architecture standards and best practices Working closely with business and technical stakeholders Leading data transformation and cloud migration initiatives Desirable Databricks, Snowflake, dbt Kafka, Apache Iceberg, Delta Lake Data Mesh/Data Fabric AI & ML-enabled data platforms ...

Experience designing data products across source-aligned, aggregate and consumer-aligned patterns. Proven ability to architect and scale modern cloud data platforms using Snowflake, Apache Iceberg, Amazon Redshift, and S3.Hands-on expertise in developing robust data transformation pipelines and orchestrating complex workflows using dbt and Apache Airflow. ...

main language. Understanding how Spark runs on the JVM. Backend services and APIs. Change data capture or database replication. Open table formats such as Iceberg, Delta or Hudi. AWS data services, including Glue, Athena, DMS, MWAA or RDS. Cloud security and networking, including IAM, VPCs and secrets management. ...

Expopsure to data governance frameworks, metadata management, and regulatory complianc It would be a bonus if you have: Familiarity with emerging technologies such as Apache Iceberg, Delta Lake, real-time/streaming architectures, or data observability tooling Experience leading architecture workshops with both technical and non-technical stakeholders ...

warehousing architecturesExpopsure to data governance frameworks, metadata management, and regulatory compliancIt would be a bonus if you have:Familiarity with emerging technologies such as Apache Iceberg, Delta Lake, real-time/streaming architectures, or data observability toolingExperience leading architecture workshops with both technical and non-technical stakeholdersA background ...

e.g. sampling methods), attention to detail, and a proactive empirical mindsetFamiliarity with data modelling & modern data tech stack tools such as Airflow, dbt, Kafka, Iceberg, Trino, Superset etc.Familiarity with some data visualization tools (Tableau, QlikSense, Power BI, or similar) to communicate quality insightsFamiliarity with at least one version control ...

have, not requiredDo not rule yourself out if you do not have these. They are a bonus, not a checklist. Experience with Delta Lake, Iceberg, Spark or Databricks, PalantirExperience optimising LLM serving with tools such as vLLM, TGI or TensorRT LLMSearch and ranking experience. For example Elasticsearch or rerankersBackground ...

this role, you will be the driving force behind building, scaling, and optimizing the critical data pipelines on top of our Lakehouse (Trino + Iceberg) infrastructure following a strict Medallion Architecture (Bronze, Silver, Gold layers). You will build robust data pipelines that serve as the foundational bedrock … working on? Technical Leadership & Pipeline EngineeringLakehouse Architecture: Design, build, and optimize robust end-to-end data pipelines on top of our Trino + Iceberg Lakehouse stack. Medallion Implementation: Own the extraction, cleaning, and modeling layers to progressively refine raw event logs (Bronze) into verified subledgers (Silver) and structured Gold ...

Data & Analytics, with proven experience as a Senior/Tech Lead setting architectural direction for other engineersDeep, hands-on knowledge of Snowflake, including ideally Iceberg/open table formats and RBAC.Advanced SQL, Python, and enterprise-grade dbt model development. Experience designing end-to-end data pipelines. Combining batch, file ...

within a shared target architecture. Depth in environments such as Azure AI, AWS Bedrock, GCP Vertex AI, OpenAI, Anthropic, Mistral, Kubernetes, Kafka, Databricks, Snowflake, Apache Iceberg, Palantir Foundry, Terraform, MCP, AI gateways or OpenTelemetry. These are examples, not a checklist. Nice to Have AI agent platform engineering: reusable ...

PySpark, and dbt Build and optimize batch and streaming data pipelines with strong performance, fault tolerance, and observability Develop and operate workflow orchestration (e.g., Apache Airflow) to schedule, monitor, and manage data movement and transformations Model and transform data for analytics using SQL and dbt to support business intelligence … with modern data warehousing/lakehouse technologies (e.g., Redshift, BigQuery, Snowflake; and engines such as Spark, Flink, or Trino; and table formats such as Iceberg, Hudi, or similar) Strong SQL skills and experience with SQL-based transformation tooling (e.g., dbt) Experience designing and operating orchestration pipelines using Airflow ...

PySpark, and dbt Build and optimize batch and streaming data pipelines with strong performance, fault tolerance, and observability Develop and operate workflow orchestration (e.g., Apache Airflow) to schedule, monitor, and manage data movement and transformations Model and transform data for analytics using SQL and dbt to support business intelligence … with modern data warehousing/lakehouse technologies (e.g., Redshift, BigQuery, Snowflake; and engines such as Spark, Flink, or Trino; and table formats such as Iceberg, Hudi, or similar) Strong SQL skills and experience with SQL-based transformation tooling (e.g., dbt) Experience designing and operating orchestration pipelines using Airflow ...

PySpark, and dbt Build and optimize batch and streaming data pipelines with strong performance, fault tolerance, and observability Develop and operate workflow orchestration (e.g., Apache Airflow) to schedule, monitor, and manage data movement and transformations Model and transform data for analytics using SQL and dbt to support business intelligence … with modern data warehousing/lakehouse technologies (e.g., Redshift, BigQuery, Snowflake and engines such as Spark, Flink, or Trino and table formats such as Iceberg, Hudi, or similar) Strong SQL skills and experience with SQL-based transformation tooling (e.g., dbt) Experience designing and operating orchestration pipelines using Airflow ...

replication tool, and Waggle Dance, a federation service that allows access to data lake tables across multiple catalogs. Recently, we added Hive support to Apache Iceberg, a high-performance open table format for large analytic datasets that underpins many modern Lakehouse architectures. We guarantee you'll learn ...

throughput, low-latency software systemsExperience with pipeline orchestration frameworks (e.g. Airflow, Dagster)Experience with streaming platforms (e.g. Kafka), data lake platforms (e.g. Delta Lake, Apache Iceberg), and relational databasesHave a Bachelors or advanced degree in Computer Science, Mathematics, Statistics, Physics, Engineering, or equivalent work experienceFor more information about ...

face detection, quality assessment, tracking)Improve training performance across single and multi-node setups using PyTorch and RayBuild evaluation and experimentation systems (Parquet/Iceberg) for model output analysisOwn model versioning, lifecycle management, and promotion to productionOptimise inference pipelines using Triton; build model ensembles and define request protocolsRequirementsProven ...

familiar withFX or financial markets experienceTerm structure modelling, stochastic calculus or Monte Carlo methodsInterest rate curve bootstrappingAlgorithmic execution experienceData lake or warehouse experience (Snowflake, Iceberg, Spark etc.)A product mindset - you think about who uses your systems and why.Clear communicator who can work cross-functionally with other quants, analysts ...

familiar withFX or financial markets experienceTerm structure modelling, stochastic calculus or Monte Carlo methodsInterest rate curve bootstrappingAlgorithmic execution experienceData lake or warehouse experience (Snowflake, Iceberg, Spark etc.)A product mindset - you think about who uses your systems and why. Clear communicator who can work cross-functionally with other quants ...

data at scale on AWS using technologies such as Lambda, containerised services, and, where appropriate, Glue, EMR Serverless, and lakehouse table formats such as Apache IcebergEmbedding observability, automated testing, monitoring, and quality controls so that issues are identified before they affect customersTaking ownership of production outcomes and helping … value strong teams and shared ownership over individual heroics and thrive in an Agile environmentNICE TO HAVESBeneficial, but not what we're screening on: Apache Spark/PySpark and experience optimising large-scale distributed processingBig data experience — designing and operating systems for datasets and workloads well beyond single-machine ...

pipelines and streaming architectures. Establish best practices for monitoring, reliability, and scale across our data ecosystem (using Python, dbt, Airflow, Kafka, and Trino/Iceberg). Build the Unified Customer View: Productionize PoCs into scalable Airflow/dbt data workflows to lay the groundwork for our Customer Data Platform … Advanced proficiency in Python and proven experience architecting, deploying, and maintaining Big Data and streaming/batch pipelines (e.g., Kafka Streams, Event Streaming, Trino, Iceberg). dbt & Airflow Proficiency: Hands-on experience using dbt for scalable data transformations and Airflow for workflow orchestration. Data Architecture Mastery: A strong background ...

bandwidth. The goal: no software rewrites, predictable time to production, early integration, and fast iteration. You'll work with tools such as DBT, Python, Apache Airflow, Spark, Iceberg; AWS services like Lambda, Bedrock, and SageMaker, containers, and whatever adds the most value. You'll design tools that ...