1 to 25 of 38 Apache Iceberg Jobs

Onsite- 5 days) Type of Employment- Permanent Job Overview: We are seeking an experienced AWS Data Engineer with strong expertise in ETL pipelines, Redshift, Iceberg, Athena, and S3 to support large-scale data processing and analytics initiatives in the telecom domain . The candidate will work closely with data … optimize schemas, tables, distribution keys, sort keys , and workload management. Implement Redshift Spectrum to query data in S3 using external tables. 3. Data Lake & Iceberg Implement and maintain Apache Iceberg tables on AWS for schema evolution and ACID operations. Build Iceberg-based ingestion and transformation pipelines ...

Position type: Permanent (Onsite) Location: Ipswich, UK Job Description: We are seeking an experienced AWS Data Engineer with strong expertise in ETL pipelines, Redshift, Iceberg, Athena, and S3 to support large-scale data processing and analytics initiatives in the telecom domain . The candidate will work closely with data … optimize schemas, tables, distribution keys, sort keys , and workload management. Implement Redshift Spectrum to query data in S3 using external tables. 3. Data Lake & Iceberg Implement and maintain Apache Iceberg tables on AWS for schema evolution and ACID operations. Build Iceberg-based ingestion and transformation pipelines ...

Russell — and we want you to shape it with us. Your role will involve: • Designing and developing scalable, testable data pipelines using Python and Apache Spark • Orchestrating data workflows with AWS tools like Glue, EMR Serverless, Lambda, and S3 • Applying modern software engineering practices: version control, CI/… modular design, and automated testing • Contributing to the development of a lakehouse architecture using Apache Iceberg • Collaborating with business teams to translate requirements into data-driven solutions • Building observability into data flows and implementing basic quality checks • Participating in code reviews, pair programming, and architecture discussions • Continuously learning ...

Russell — and we want you to shape it with us. Your role will involve: • Designing and developing scalable, testable data pipelines using Python and Apache Spark • Orchestrating data workflows with AWS tools like Glue, EMR Serverless, Lambda, and S3 • Applying modern software engineering practices: version control, CI/… modular design, and automated testing • Contributing to the development of a lakehouse architecture using Apache Iceberg • Collaborating with business teams to translate requirements into data-driven solutions • Building observability into data flows and implementing basic quality checks • Participating in code reviews, pair programming, and architecture discussions • Continuously learning ...

NumPy, Flask, Streamlit, SHAP, Lime, PyIceberg, Matplotlib, Seaborn, Plotly) SQL, Graph stores (e.g. Neo4J) Containers (e.g. Docker) Data bases/Data Lakes (e.g. PostgreSQL, Apache Iceberg, Databricks, Azure Fabric etc.) AWS (CDK/Python, Bedrock, Sagemaker) or similar cloud toolset experience ...

services (Redshift, Lambda, S3, Glue, Lake Formation). Expertise in ETL/ELT development, data warehouse design (Kimball, Lakehouse). Proficiency in Python, Apache Spark, SQL, and CI/CD pipelines . Knowledge of Agile methodologies and ability to manage projects effectively. Excellent stakeholder management and communication skills. Hands … experience with DBT, Apache Iceberg, Docker (nice to have). What you’ll get: Salary up to £60,000 Strong career progression and development Hybrid working ( 2 days a week in Leeds - non-negotiable ) Annual Leave: 25 days + Bank Holidays 5% Discretionary bonus Pension: Employee contribution ...

many different sectors. What We’re Looking For 5 + years, hands-on experience in AWS data engineering technologies, including Glue, PySpark, Athena, Iceberg, Databricks, Lake Formation, and other standard data engineering tools. Strong experience engineering in a front-office/capital markets environment. Previous experience in implementing best ...

many different sectors. What We’re Looking For 5 + years, hands-on experience in AWS data engineering technologies, including Glue, PySpark, Athena, Iceberg, Databricks, Lake Formation, and other standard data engineering tools. Strong experience engineering in a front-office/capital markets environment. Previous experience in implementing best ...

many different sectors. What We’re Looking For 10 + years, hands-on experience in AWS data engineering technologies, including Glue, PySpark, Athena, Iceberg, Databricks, Lake Formation, and other standard data engineering tools. Strong experience engineering in a front-office/capital markets environment. Previous experience in implementing best ...

many different sectors. What We’re Looking For 10 + years, hands-on experience in AWS data engineering technologies, including Glue, PySpark, Athena, Iceberg, Databricks, Lake Formation, and other standard data engineering tools. Strong experience engineering in a front-office/capital markets environment. Previous experience in implementing best ...

Azure. Develop and maintain Parquet-based data layers; contribute to evolving the data lake architecture and metadata management. Implement dataset versioning and management using Apache Iceberg. Collaborate closely with trading and quant teams to translate data requirements into robust, production-grade pipelines. Implement monitoring, validation, and automated error-handling … optimisation and concurrency. Proven experience designing, building, and tuning tick-data pipelines for high-volume environments. Hands-on experience with Parquet storage; experience with Apache Iceberg or similar table formats is a plus. Practical experience with containerisation (Docker) and orchestration platforms (Kubernetes). Strong background in profiling, debugging ...

Azure. Develop and maintain Parquet-based data layers; contribute to evolving the data lake architecture and metadata management. Implement dataset versioning and management using Apache Iceberg. Collaborate closely with trading and quant teams to translate data requirements into robust, production-grade pipelines. Implement monitoring, validation, and automated error-handling … optimisation and concurrency. Proven experience designing, building, and tuning tick-data pipelines for high-volume environments. Hands-on experience with Parquet storage; experience with Apache Iceberg or similar table formats is a plus. Practical experience with containerisation (Docker) and orchestration platforms (Kubernetes). Strong background in profiling, debugging ...

team to make sure our data function not only delivers but also drives strategic decision-making. Our Tech Stack: Python, Docker, Dagster, dbt, Fivetran, Apache Iceberg, Snowflake, S3, Glue, ECS, and Omni. We’re constantly evolving our stack and welcome input from engineering leaders … Hands-on experience with dbt, modern data warehouses, and AWS. Demonstrated ability to mentor and develop engineers. Desirable Experience: Experience with Snowflake. Experience with Apache Iceberg. Experience with infrastructure-as-code (Terraform preferred). Experience embedding observability and monitoring in data systems. Previous experience building and leading data teams ...

data stack (Glue, OpenSearch, RDS) Expert skills within SQL with experience in both transactional RDBMS systems and distributed systems Hands-on with Lakehouse technologies (Apache Iceberg, S3 Tables, StarRocks) Strong grasp of data governance, schema design, and quality frameworks Comfortable leading infrastructure decisions and collaborating across distributed teams ...

understanding of data governance and regulatory compliance frameworks such asPCI-DSS, GDPR, SOX, and CPRA. Bonus: Experience with Lakehouse technologies such as Delta Lake, Apache Iceberg, or Hudi. Daily Duties: Design, develop, and maintain scalable data pipelines that support enterprise reporting, regulatory compliance, and analytics. Implement data validation ...

using AWS Glue (PySpark/Python) Working with structured/semi-structured telecom datasets (CDRs, network logs, subscriber data) Data Lake Technologies: Expertise in Apache Iceberg table format Schema evolution, partitioning, compaction & metadata management Query performance tuning with Athena & Redshift Spectrum Redshift Expertise: Data modeling, distribution styles, sort … Responsibilities Provide L2/L3 support for AWS-based data platforms in the telecom domain. Maintain and enhance ETL pipelines built on Glue + Iceberg + Athena + Redshift. Monitor production jobs, fix failures, optimize queries, and ensure SLA adherence. Develop automation for operational workflows using Python. Collaborate with ...

experience in data modeling tools (Erwin, ER/Studio, PowerDesigner, or similar Strong knowledge of relational, dimensional, and lakehouse modeling techniques. Experience with Parquet, Iceberg, and cloud-native data storage formats. Strong understanding of Finance & Capital Markets data structures (trades, positions, risk, reference/master data 58 years ...

required Do not rule yourself out if you do not have these. They are a bonus, not a checklist. Experience with Delta Lake, Iceberg, Spark or Databricks, Palantir Experience optimising LLM serving with tools such as vLLM, TGI or TensorRT LLM Search and ranking experience. For example Elasticsearch ...

required Do not rule yourself out if you do not have these. They are a bonus, not a checklist. Experience with Delta Lake, Iceberg, Spark or Databricks, Palantir Experience optimising LLM serving with tools such as vLLM, TGI or TensorRT LLM Search and ranking experience. For example Elasticsearch ...

required Do not rule yourself out if you do not have these. They are a bonus, not a checklist. Experience with Delta Lake, Iceberg, Spark or Databricks, Palantir Experience optimising LLM serving with tools such as vLLM, TGI or TensorRT LLM Search and ranking experience. For example Elasticsearch ...

Onsite- 5 days) Type of Employment- Permanent Job Overview: We are seeking an experienced AWS Data Engineer with strong expertise in ETL pipelines, Redshift, Iceberg, Athena, and S3 to support large-scale data processing and analytics initiatives in the telecom domain click apply for full job details ...

event routing and filtering Schema Management: Avro, Protobuf Schema Registry: compatibility, subject strategy, evolution AWS Data Stack: S3, Glue, Athena, Redshift Step Functions, Lambda Iceberg-ready lakehouse patterns Streaming Pipelines: Kinesis → S3 → Glue Glue Streaming DLQ patterns Minimum years of experience: > 10 years ...

event routing and filtering Schema Management: Avro, Protobuf Schema Registry: compatibility, subject strategy, evolution AWS Data Stack: S3, Glue, Athena, Redshift Step Functions, Lambda Iceberg-ready lakehouse patterns Streaming Pipelines: Kinesis → S3 → Glue Glue Streaming DLQ patterns Minimum years of experience: > 10 years ...

event routing and filtering Schema Management: Avro, Protobuf Schema Registry: compatibility, subject strategy, evolution AWS Data Stack: S3, Glue, Athena, Redshift Step Functions, Lambda Iceberg-ready lakehouse patterns Streaming Pipelines: Kinesis S3 Glue Glue Streaming DLQ patterns If you are interested in this position and would like to learn ...

Strong communication skills and experience working cross-functionally Required Skills Experience consuming data from event streams Familiarity with modern table formats (e.g. Iceberg or similar) Exposure to BI and dashboarding tools Experience improving data observability and monitoring Comfort working in Linux or Unix environments Preferred Skills High ownership ...