51 to 75 of 91 Apache Iceberg Jobs

services, a good understanding of cloud architecture, and proficiency with Terraform for provisioning and managing cloud infrastructure Experience building streaming data pipelines using Apache Flink, Apache Kafka, Spark Structured Streaming, or similar technologies Knowledge of modern data lakehouse architectures and ecosystem such as Kafka, Flink, Spark, Databricks, Snowflake … Airflow, Debezium, Delta/Iceberg, and comfort with Python/Java and SQL Experience with networking and security concepts within AWS, including VPCs, subnets, routing, security groups, IAM, etc. Nice to have Experience: Exposure to or interest in the cryptocurrency technology ecosystem Experience working with different blockchain technologies ...

replication tool, and Waggle Dance, a federation service that allows access to data lake tables across multiple catalogs. Recently, we added Hive support to Apache Iceberg, a high-performance open table format for large analytic datasets that underpins many modern Lakehouse architectures. We guarantee you’ll learn ...

replication tool, and Waggle Dance, a federation service that allows access to data lake tables across multiple catalogs. Recently, we added Hive support to Apache Iceberg, a high-performance open table format for large analytic datasets that underpins many modern Lakehouse architectures. We guarantee you’ll learn ...

cloud environments. You will enable real-time analytics and governance while promoting self-service insights for business users. Key skills include data lakehouse design (Iceberg), AWS/Azure data services, dbt, Airflow, and BI/dashboard experience. The role emphasizes automation, quality, and collaboration with architects. #J-18808-Ljbffr ...

implementing computational improvements Making improvements and/or fixing bugs by submitting patches to open-source projects in our stack, such as Spark, Iceberg, Hadoop, Airflow, and many others Acting as an expert on Spark and Scala Mentoring engineers (including through pairing, design discussions, and knowledge-sharing sessions ...

services on Kubernetes in at least one major cloud (AWS, GCP, or Azure). Experience with data warehousing or analytics technologies such as Snowflake, Iceberg, Trino, or Postgres. Experience designing, deploying, and operating ML models in production, including familiarity with MLOps tooling such as MLflow. Demonstrated ability to work ...

agentic systems. Experience designing external‐facing APIs, including versioning, backwards compatibility, and OpenAPI/REST standards. Familiarity with data infrastructure such as Airflow, Spark, Iceberg, or Kafka. Experience with energy markets. Modo Energy is an equal opportunity employer. Employment decisions are made on the basis of qualifications, merit ...

agentic systems. Experience designing external-facing APIs, including versioning, backwards compatibility, and OpenAPI/REST standards. Familiarity with data infrastructure such as Airflow, Spark, Iceberg, or Kafka. Experience with energy markets. Modo Energy is an equal opportunity employer. Employment decisions are made on the basis of qualifications, merit ...

Data Engineering Lead – AWS & AI - Manchester An established financial services organisation is looking for a hands-on Data Engineering Lead to take ownership of its existing data estate and deliver a modern platform supporting Business ...

Data Engineering Lead AWS & AI - Manchester An established financial services organisation is looking for a hands-on Data Engineering Lead to take ownership of its existing data estate and deliver a modern platform supporting Business ...

Skill Requirements:*** Strong experience in **Solution Architecture** and **Cloud-Native Architecture Design*** Hands-on expertise in **Snowflake Architecture*** Experience with **Data Lakehouse architectures**, including **Apache Iceberg*** Proficiency in designing and managing **S3-based Data Lakes*** Experience with **Apache Airflow/MWAA** for workflow orchestration* Strong knowledge … scale data processing* Ability to create **Architecture Design Documents (ADDs)*** Deep understanding of **cloud-native design patterns*** Experience in **performance tuning** across:* Snowflake* Airflow* Iceberg* Focus on **platform reliability, scalability, and observability*** Experience designing and operating **data platforms** in production environments #J-18808-Ljbffr ...

required, including 5 years' UK residency; successfully obtaining security clearance is a condition of employment These will help you succeed Strong working knowledge of Apache Iceberg and open table formats, with the ability to architect interoperable strategies that avoid vendor lock-in Practical experience designing or transitioning organisations ...

streaming workloads, managing schemas as they evolve, and guaranteeing that consumers only read complete data. We build on open standards like OpenLineage and Apache Iceberg, and define the internal standards for naming, storage and metadata that the rest of Spotify builds against. As the platform takes ...

DevOps practices. Experience with monitoring and logging tools likeCloudWatch, Dynatrace, Datadog,Prometheus, Grafana, or ELK stack. Proficiency in data warehousing solutions like Snowflake and Iceberg and relational databases such as RDS or Oracle. Experience in Infrastructure as code (IAC) such as Terraform or equivalent Leverages enterprise-authorized AI coding ...

queues, event-driven architectures or data pipelines. Kubernetes: deploying, running or debugging workloads, including K3s, RKE2 or OpenShift. Data at scale: Spark, Trino, Parquet, Iceberg or similar columnar and distributed query tooling. GPU workloads and serving models on-premise. Cloud experience, ideally GCP, and cloud migration experience. Mentoring engineers ...

queues, event-driven architectures or data pipelines. Kubernetes: deploying, running or debugging workloads, including K3s, RKE2 or OpenShift. Data at scale: Spark, Trino, Parquet, Iceberg or similar columnar and distributed query tooling. GPU workloads and serving models on-premise. Cloud experience, ideally GCP, and cloud migration experience. Mentoring engineers ...

queues, event-driven architectures or data pipelines. Kubernetes: deploying, running or debugging workloads, including K3s, RKE2 or OpenShift. Data at scale: Spark, Trino, Parquet, Iceberg or similar columnar and distributed query tooling. GPU workloads and serving models on-premise. Cloud experience, ideally GCP, and cloud migration experience. Mentoring engineers ...

value to real users. Nice to have Experience working with scientific or scholarly text. Familiarity with AWS (S3, Batch, Lambda, SageMaker) and Parquet or Iceberg data lake patterns. Experience running LLMs under real cost and latency budgets in production. Some exposure to agentic AI applications: tool use, multi‐step ...

throughput, low-latency software systemsExperience with pipeline orchestration frameworks (e.g. Airflow, Dagster)Experience with streaming platforms (e.g. Kafka), data lake platforms (e.g. Delta Lake, Apache Iceberg), and relational databasesHave a Bachelors or advanced degree in Computer Science, Mathematics, Statistics, Physics, Engineering, or equivalent work experienceFor more information about ...

Skills: Experience with agent frameworks and protocols (Google ADK, LangGraph, MCP, A2A, AG-UI) Experience with Databricks, MLflow, Spark, and Delta Lake or Apache Iceberg Experience serving models on Kubernetes, including GPU workloads and fine-tuned small language models Experience with optimization or pricing models and exposing them ...

data environments, Starburst helps enterprises power AI and analytics without the cost and complexity of traditional data consolidation. With open standards including Trino and Apache Iceberg, Starburst enables trusted access to complete enterprise context while helping organizations avoid vendor lock-in. Leading global enterprises trust Starburst to fuel ...

data environments, Starburst helps enterprises power AI and analytics without the cost and complexity of traditional data consolidation. With open standards including Trino and Apache Iceberg, Starburst enables trusted access to complete enterprise context while helping organizations avoid vendor lock-in. Leading global enterprises trust Starburst to fuel ...

Rate: £650 - £750 Location: Hybrid – 2/3 days per week on-site Contract: 6 months | Outside IR35 Key Skills: Data Lakehouse PySpark DBT Iceberg AWS Strong data engineering and pipeline development experience You’ll be responsible for designing, building and optimising scalable data pipelines and lakehouse solutions within ...

align solutions with strategic goals. The role requires 5+ years in Python/Java, strong AWS knowledge, and hands-on experience with Snowflake, Iceberg, Spark, and DBT. The position offers a hybrid work arrangement and involvement in enterprise-grade data projects. #J-18808-Ljbffr ...

About us At Trainline, our purpose is to empower greener travel choices, connecting people and places. Trainline enables millions of travellers to find and book the best value tickets across carriers, fares, and journey options ...