76 to 90 of 90 Apache Iceberg Jobs in the UK

Experience of working in an Agile delivery team Strong analytic skills related to working with unstructured datasets Python (PySpark, Pandas, PyArrow) Distributed data processing (Apache Spark) Data ETL (Apache Airflow, AWS Step Functions, Apache NiFi) Cloud services (AWS, Azure or GCP) Messaging/Streaming (Kafka … Other Cloud Queuing Native services) SQL and NoSQL Storage (HDFS, Iceberg, Elastic, S3, Data Lake) Containerisation and orchestration (Docker/Kubernetes/Openshift) Testing frameworks and best practices Use of Code repositories, familiarity with branching strategies, pull requests and merge processes Benefits £: Competitive salary Contributory Pension Scheme ...

centric data fabric, unifying multi-stream clinical data such as EDC, telemetry, and real-world datasets. You will build a modern Data Lake using Apache Iceberg, Snowflake, and Kafka, develop backend services in Java or Scala on AWS, and apply AI-assisted tooling for schema mapping, testing ...

Kingdom. The role demands strong expertise in Solution Architecture, Cloud-Native Architecture Design, and hands-on experience with Snowflake and Data Lakehouse architectures, including Apache Iceberg. Responsibilities involve creating Architecture Design Documents, managing S3-based Data Lakes, and ensuring platform performance tuning and observability. Ideal candidates should have substantial ...

such as CML, SparkMLlib, and Python ML libraries (scikit‐learn, XGBoost), including modeldeployment. Design and develop highly scalable, Real time systemsusing Hadoop ecosystem components(Iceberg, Spark, Ozone, Trino, Hive,Ranger, Kafka, Flink and Nifi) Build robust data ingestion and transformationframeworks using Java, Spark, Python, and shell scripting for ingestingmulti ...

Experience of working in an Agile delivery team Strong analytic skills related to working with unstructured datasets Python (PySpark, Pandas, PyArrow) Distributed data processing (Apache Spark) Data ETL (Apache Airflow, AWS Step Functions, Apache NiFi) Cloud services (AWS, Azure or GCP) Messaging/Streaming (Kafka … Other Cloud Queuing Native services) SQL and NoSQL Storage (HDFS, Iceberg, Elastic, S3, Data Lake) Containerisation and orchestration (Docker/Kubernetes/Openshift) Testing frameworks and best practices Use of Code repositories, familiarity with branching strategies, pull requests and merge processes We appreciate you may not be an expert ...

Experience of working in an Agile delivery team Strong analytic skills related to working with unstructured datasets Python (PySpark, Pandas, PyArrow) Distributed data processing (Apache Spark) Data ETL (Apache Airflow, AWS Step Functions, Apache NiFi) Cloud services (AWS, Azure or GCP) Messaging/Streaming (Kafka … Other Cloud Queuing Native services) SQL and NoSQL Storage (HDFS, Iceberg, Elastic, S3, Data Lake) Containerisation and orchestration (Docker/Kubernetes/Openshift) Testing frameworks and best practices Use of Code repositories, familiarity with branching strategies, pull requests and merge processes We appreciate you may not be an expert ...

data environments, Starburst helps enterprises power AI and analytics without the cost and complexity of traditional data consolidation. With open standards including Trino and Apache Iceberg, Starburst enables trusted access to complete enterprise context while helping organizations avoid vendor lock-in. Leading global enterprises trust Starburst to fuel ...

pipelines and streaming architectures. Establish best practices for monitoring, reliability, and scale across our data ecosystem (using Python, dbt, Airflow, Kafka, and Trino/Iceberg). Build the Unified Customer View: Productionize PoCs into scalable Airflow/dbt data workflows to lay the groundwork for our Customer Data Platform … Advanced proficiency in Python and proven experience architecting, deploying, and maintaining Big Data and streaming/batch pipelines (e.g., Kafka Streams, Event Streaming, Trino, Iceberg). dbt & Airflow Proficiency: Hands‐on experience using dbt for scalable data transformations and Airflow for workflow orchestration. Data Architecture Mastery: A strong background ...

team of highly technically skilledengineerswhoaredesigningCreditsafe’snewConsumer Bureauwithdata quality,lineage, andauditabilityas primary goals whilstmaintaininghigh throughputand scalability.Thedata platformwill be builtusingAWSinfrastructure such as S3and S3-tables(Iceberg), AuroraServerless, DynamoDB, Lambda,Fargate, ECS, Airflow,Sparkand Kinesis.Theplatformisexpectedto manage overone hundred millionobjectswith a high volume of daily updateshandlingaddition, deletion,andcorrectionof our dataand indexesinanauditable manner.The platform … Amigo's refinement sessions all the way to post-deployment observability. Design and build metadata driven,eventbaseddistributeddata processing platformusing technologies such asPython,Spark,Airflow, Iceberg,DynamoDB,AWS Glue, S3. Playan activerole in the design, development, and deployment ofour business-criticalsystem. Building and scalingCreditsafe’sdata pipelinescombining batch and streaming approachesandtargetingsub ...

engines and planning capabilities across large tables and diverse workloads. You will work with Java, Rust, Scala, or C++ and engage with Spark, DataFusion, Iceberg, Arrow, and Parquet in a high-performance environment that values #J-18808-Ljbffr ...

Spark/Scala data processing (also some Flink), Kubernetes and AWS native tooling. What You'll Do Write highly fault-tolerant Java code within Apache Spark to produce platform products used by our customers to query our event pipelines/ingestion for insight into active threat trends and related … between backend/cloud development and data platform engineering roles 5+ years of experience building data platform product(s) or features with (one of) Apache Spark, Flink or Iceberg, or with comparable tools in GCP 5+ years of experience programming with Java, Scala or Kotlin. Proven experience utilizing ...

mindset focused on automation, reliability, and maintainable solutions.## **Preferred Qualifications*** Experience with Argo CD, Helm, or other GitOps tooling.* Familiarity with Airflow, Snowflake, or Apache Iceberg.* Experience with Apache Kafka or Amazon MSK.* Experience with hybrid cloud, enterprise networking, or multi-account AWS environments.* Experience with Ansible ...

DevOps knowledge, including Linux/Unix, OpenShift/Kubernetes, and Ansible. Experience building and supporting reactive, event-sourced systems. Hands-on experience with Apache Spark and Apache Flink. Strong understanding of Apache HTTP, Envoy, and REST API design. Experience working with Protobuf, Avro, and Parquet data formats. … Knowledge of Iceberg, Databricks, and Hadoop ecosystems. Skills that will help Follows TDD, ATDD and Clean Coding practices and used to setting up or working in an environment using CI/CD using tools such as Jenkins with build tools like Gradle Experience of Scrum/Agile methods Experienced ...

cases and capabilities across the value chain — from getting data onto AWS and modernising it onto purpose-built engines, to unifying it in an Iceberg-native lakehouse, governing it for AI, and consuming it through AI-powered analytics — developing and executing go-to-market and enablement plans that lead … EMEA territory:1. Identify industry-specific and horizontal data use cases and solution offerings across the six pillars — database migration and modernisation, Iceberg-native lakehouse, vector and agent-memory workloads, data governance, and AI-powered analytics — in the EMEA region;2. Actively engage with mature and emerging Data & Analytics ...

Expedia Group is seeking a Senior Software Engineer to join the Data Lake and Catalog team in London. You will design, build, and operate scalable data lake services, focusing on reliability, observability, and end-to ...