1 to 25 of 36 Permanent Apache Iceberg Jobs

Onsite- 5 days) Type of Employment- Permanent Job Overview: We are seeking an experienced AWS Data Engineer with strong expertise in ETL pipelines, Redshift, Iceberg, Athena, and S3 to support large-scale data processing and analytics initiatives in the telecom domain . The candidate will work closely with data … optimize schemas, tables, distribution keys, sort keys , and workload management. Implement Redshift Spectrum to query data in S3 using external tables. 3. Data Lake & Iceberg Implement and maintain Apache Iceberg tables on AWS for schema evolution and ACID operations. Build Iceberg-based ingestion and transformation pipelines ...

Position type: Permanent (Onsite) Location: Ipswich, UK Job Description: We are seeking an experienced AWS Data Engineer with strong expertise in ETL pipelines, Redshift, Iceberg, Athena, and S3 to support large-scale data processing and analytics initiatives in the telecom domain . The candidate will work closely with data … optimize schemas, tables, distribution keys, sort keys , and workload management. Implement Redshift Spectrum to query data in S3 using external tables. 3. Data Lake & Iceberg Implement and maintain Apache Iceberg tables on AWS for schema evolution and ACID operations. Build Iceberg-based ingestion and transformation pipelines ...

Russell — and we want you to shape it with us. Your role will involve: • Designing and developing scalable, testable data pipelines using Python and Apache Spark • Orchestrating data workflows with AWS tools like Glue, EMR Serverless, Lambda, and S3 • Applying modern software engineering practices: version control, CI/… modular design, and automated testing • Contributing to the development of a lakehouse architecture using Apache Iceberg • Collaborating with business teams to translate requirements into data-driven solutions • Building observability into data flows and implementing basic quality checks • Participating in code reviews, pair programming, and architecture discussions • Continuously learning ...

Russell — and we want you to shape it with us. Your role will involve: • Designing and developing scalable, testable data pipelines using Python and Apache Spark • Orchestrating data workflows with AWS tools like Glue, EMR Serverless, Lambda, and S3 • Applying modern software engineering practices: version control, CI/… modular design, and automated testing • Contributing to the development of a lakehouse architecture using Apache Iceberg • Collaborating with business teams to translate requirements into data-driven solutions • Building observability into data flows and implementing basic quality checks • Participating in code reviews, pair programming, and architecture discussions • Continuously learning ...

NumPy, Flask, Streamlit, SHAP, Lime, PyIceberg, Matplotlib, Seaborn, Plotly) SQL, Graph stores (e.g. Neo4J) Containers (e.g. Docker) Data bases/Data Lakes (e.g. PostgreSQL, Apache Iceberg, Databricks, Azure Fabric etc.) AWS (CDK/Python, Bedrock, Sagemaker) or similar cloud toolset experience ...

services (Redshift, Lambda, S3, Glue, Lake Formation). Expertise in ETL/ELT development, data warehouse design (Kimball, Lakehouse). Proficiency in Python, Apache Spark, SQL, and CI/CD pipelines . Knowledge of Agile methodologies and ability to manage projects effectively. Excellent stakeholder management and communication skills. Hands … experience with DBT, Apache Iceberg, Docker (nice to have). What you’ll get: Salary up to £60,000 Strong career progression and development Hybrid working ( 2 days a week in Leeds - non-negotiable ) Annual Leave: 25 days + Bank Holidays 5% Discretionary bonus Pension: Employee contribution ...

many different sectors. What We’re Looking For 5 + years, hands-on experience in AWS data engineering technologies, including Glue, PySpark, Athena, Iceberg, Databricks, Lake Formation, and other standard data engineering tools. Strong experience engineering in a front-office/capital markets environment. Previous experience in implementing best ...

many different sectors. What We’re Looking For 5 + years, hands-on experience in AWS data engineering technologies, including Glue, PySpark, Athena, Iceberg, Databricks, Lake Formation, and other standard data engineering tools. Strong experience engineering in a front-office/capital markets environment. Previous experience in implementing best ...

many different sectors. What We’re Looking For 10 + years, hands-on experience in AWS data engineering technologies, including Glue, PySpark, Athena, Iceberg, Databricks, Lake Formation, and other standard data engineering tools. Strong experience engineering in a front-office/capital markets environment. Previous experience in implementing best ...

many different sectors. What We’re Looking For 10 + years, hands-on experience in AWS data engineering technologies, including Glue, PySpark, Athena, Iceberg, Databricks, Lake Formation, and other standard data engineering tools. Strong experience engineering in a front-office/capital markets environment. Previous experience in implementing best ...

Azure. Develop and maintain Parquet-based data layers; contribute to evolving the data lake architecture and metadata management. Implement dataset versioning and management using Apache Iceberg. Collaborate closely with trading and quant teams to translate data requirements into robust, production-grade pipelines. Implement monitoring, validation, and automated error-handling … optimisation and concurrency. Proven experience designing, building, and tuning tick-data pipelines for high-volume environments. Hands-on experience with Parquet storage; experience with Apache Iceberg or similar table formats is a plus. Practical experience with containerisation (Docker) and orchestration platforms (Kubernetes). Strong background in profiling, debugging ...

Azure. Develop and maintain Parquet-based data layers; contribute to evolving the data lake architecture and metadata management. Implement dataset versioning and management using Apache Iceberg. Collaborate closely with trading and quant teams to translate data requirements into robust, production-grade pipelines. Implement monitoring, validation, and automated error-handling … optimisation and concurrency. Proven experience designing, building, and tuning tick-data pipelines for high-volume environments. Hands-on experience with Parquet storage; experience with Apache Iceberg or similar table formats is a plus. Practical experience with containerisation (Docker) and orchestration platforms (Kubernetes). Strong background in profiling, debugging ...

data stack (Glue, OpenSearch, RDS) Expert skills within SQL with experience in both transactional RDBMS systems and distributed systems Hands-on with Lakehouse technologies (Apache Iceberg, S3 Tables, StarRocks) Strong grasp of data governance, schema design, and quality frameworks Comfortable leading infrastructure decisions and collaborating across distributed teams ...

understanding of data governance and regulatory compliance frameworks such asPCI-DSS, GDPR, SOX, and CPRA. Bonus: Experience with Lakehouse technologies such as Delta Lake, Apache Iceberg, or Hudi. Daily Duties: Design, develop, and maintain scalable data pipelines that support enterprise reporting, regulatory compliance, and analytics. Implement data validation ...

experience in data modeling tools (Erwin, ER/Studio, PowerDesigner, or similar Strong knowledge of relational, dimensional, and lakehouse modeling techniques. Experience with Parquet, Iceberg, and cloud-native data storage formats. Strong understanding of Finance & Capital Markets data structures (trades, positions, risk, reference/master data 58 years ...

required Do not rule yourself out if you do not have these. They are a bonus, not a checklist. Experience with Delta Lake, Iceberg, Spark or Databricks, Palantir Experience optimising LLM serving with tools such as vLLM, TGI or TensorRT LLM Search and ranking experience. For example Elasticsearch ...

required Do not rule yourself out if you do not have these. They are a bonus, not a checklist. Experience with Delta Lake, Iceberg, Spark or Databricks, Palantir Experience optimising LLM serving with tools such as vLLM, TGI or TensorRT LLM Search and ranking experience. For example Elasticsearch ...

required Do not rule yourself out if you do not have these. They are a bonus, not a checklist. Experience with Delta Lake, Iceberg, Spark or Databricks, Palantir Experience optimising LLM serving with tools such as vLLM, TGI or TensorRT LLM Search and ranking experience. For example Elasticsearch ...

Onsite- 5 days) Type of Employment- Permanent Job Overview: We are seeking an experienced AWS Data Engineer with strong expertise in ETL pipelines, Redshift, Iceberg, Athena, and S3 to support large-scale data processing and analytics initiatives in the telecom domain click apply for full job details ...

Kubernetes (EKS). Ask anything Important skills: Must have: 1. Bash Scripting 2.Real time with Spark structured streaming Nice to have : Familiarity with IceBerg GCP/BQexposure ...

event routing and filtering Schema Management: Avro, Protobuf Schema Registry: compatibility, subject strategy, evolution AWS Data Stack: S3, Glue, Athena, Redshift Step Functions, Lambda Iceberg-ready lakehouse patterns Streaming Pipelines: Kinesis → S3 → Glue Glue Streaming DLQ patterns Minimum years of experience: > 10 years ...

event routing and filtering Schema Management: Avro, Protobuf Schema Registry: compatibility, subject strategy, evolution AWS Data Stack: S3, Glue, Athena, Redshift Step Functions, Lambda Iceberg-ready lakehouse patterns Streaming Pipelines: Kinesis → S3 → Glue Glue Streaming DLQ patterns Minimum years of experience: > 10 years ...

Strong communication skills and experience working cross-functionally Required Skills Experience consuming data from event streams Familiarity with modern table formats (e.g. Iceberg or similar) Exposure to BI and dashboarding tools Experience improving data observability and monitoring Comfort working in Linux or Unix environments Preferred Skills High ownership ...

Strong communication skills and experience working cross-functionally Required Skills Experience consuming data from event streams Familiarity with modern table formats (e.g. Iceberg or similar) Exposure to BI and dashboarding tools Experience improving data observability and monitoring Comfort working in Linux or Unix environments Preferred Skills High ownership ...

experience in the following: Cloud Data Architecture: Strong knowledge of modern, cloud-based data architecture and tooling (eg, S3, Glue, Redshift, Athena, Lake Formation, Iceberg/Delta). AWS Platform Build: Demonstrable experience designing and building modern data platforms in AWS. ETL/Orchestration Expertise: Expertise in ETL/… design and data orchestration, specifically with Apache Airflow. SQL Mastery: Strong SQL skills with significant experience in query tuning and performance optimisation. Programming Proficiency: Proficiency in Python and Bash (for data processing, Scripting, and automation). Data Streaming: Familiarity with streaming data technologies (eg, Kinesis, Kafka, or similar). ...