1 to 25 of 29 Remote/Hybrid Permanent Apache Iceberg Jobs

Software Engineer – Query Engines

Location
Greater London, England, United Kingdom
computations are represented, how query plans are optimized, how operators execute, and how data is read and written efficiently. We build on and extend Apache Spark, Apache DataFusion, and Apache Iceberg, bringing advances in open-source engines and table formats into the demanding environments our customers … want to advance the capabilities of modern data systems and apply that work to consequential problems. Technologies We Use Java, Scala, Rust, and Python Apache Spark, Apache DataFusion, Apache Comet, and Velox for data processing and query execution Apache Iceberg for table management and catalog ...

Software Engineer, New Grad - Query Engines

Location
Greater London, England, United Kingdom
computations are represented, how query plans are optimised, how operators execute, and how data is read and written efficiently. We build on and extend Apache Spark, Apache DataFusion, and Apache Iceberg, bringing advances in open-source engines and table formats into the demanding environments our customers … learn how modern data systems work from the inside and apply that knowledge to consequential problems. Technologies We Use Java, Scala, Rust, and Python Apache Spark, Apache DataFusion, Apache Comet, and Velox for data processing and query execution Apache Iceberg for table management and catalog ...

Data Engineer (contract)

Location
United Kingdom
teams to deliver high-performance data solutions. This role requires deep knowledge of data engineering best practices and strong hands-on experience with AWS, Apache Spark, data warehousing (Redshift/Apache Iceberg), database modelling, and SQL. You will provide technical leadership across the full data engineering lifecycle … working in central government and highly regulated environments. Key Responsibilities Design, build, and maintain robust, scalable, and secure data pipelines using AWS services and Apache Spark. Develop and optimise data models for reporting and analytics in Redshift and other DWH platforms. Collaborate with Data Scientists, Analysts, and Business Stakeholders ...

Sr. Snowflake Architect

Location
City of Edinburgh, Scotland, United Kingdom
analytical and operational domains.Establish ELT/ETL and streaming architectures on Snowflake Snowpipe, Snowpipe Streaming, Dynamic Tables, Streams and Tasks, external tables, and Apache Iceberg tables along with orchestration standards (dbt, Airflow, Snowflake Tasks) and CI/CD/DataOps best practices.Implement security-by-design in Snowflake: RBAC … Azure, or GCP) including Snowflake deployment, networking (Private Link), and cloud storage integration (S3, ADLS, GCS); familiarity with lakehouse stacks (Databricks, Delta Lake, Apache Iceberg) is a plus.Mastery of Snowflake security and governance: RBAC, tag-based governance, dynamic data masking, row access policies, column-level security, Tri-Secret ...

Data Engineer

Location
Redditch, England, United Kingdom
pipelines using Python and PySpark Building and optimising data workflows using AWS Glue, Athena, Redshift , and related AWS services Supporting modern lakehouse architectures, including Apache Iceberg Working with relational and NoSQL databases to ensure data is reliable and fit for purpose Collaborating with analysts, engineers, product owners … Athena, Amazon Redshift , and cloud‐based data platforms Strong SQL skills and experience working with both relational and NoSQL databases Experience supporting or developing Apache Iceberg or modern lakehouse solutions Experience working in Agile delivery environments Familiarity with Linux and Git‐based version control Desirable experience Experience with ...

Senior Full Stack Software Engineer

Location
Brentford, England, United Kingdom
Learning or computer vision products into customer-facing software. GIS applications or geo-referenced imagery. Data-lake or lakehouse technologies such as Delta Lake, Apache Iceberg, Apache Hudi, Databricks, Snowflake or AWS Lake Formation. Event-driven architectures using Kafka, RabbitMQ, Redis Streams or AWS services. Cloud platforms ...

Senior Full Stack Software Engineer

Hiring Organisation
MFK Recruitment
Location
London, England, United Kingdom
Machine Learning or computer vision products into customer-facing software.GIS applications or geo-referenced imagery.Data-lake or lakehouse technologies such as Delta Lake, Apache Iceberg, Apache Hudi, Databricks, Snowflake or AWS Lake Formation.Event-driven architectures using Kafka, RabbitMQ, Redis Streams or AWS services.Cloud platforms and containerisation technologies ...

Data Engineer, Portugal London

Location
Greater London, England, United Kingdom
environments. Technical Skills: Experience building and operating ETL/ELT pipelines. Hands-on experience with modern data stacks — our tech includes Python, SQL, Snowflake, Apache Iceberg, AWS S3, PostgresDB, Airflow, dbt, and Apache Spark, deployed via AWS, Docker, and Terraform (experience with some of these or similar ...

Senior Data Engineer, Fabric & Analytics (Remote)

Location
Sheffield, England, United Kingdom
support comprehensive data analysis and reporting. Implement and uphold best practices in data governance, security, and compliance, using tools like Azure Purview, UnityCatalog, and Apache Atlas tomaintaindata integrity and trust. Ensure data quality and integrity through meticulous attention to detail and rigorous QA processes, continually refining and optimising data … large-scale data systems with high performance and low latency, such as managing 1 billion+ records or terabyte-sized databases. Knowledge of Delta TablesorApache Iceberg:Understanding and experience with Delta Tables or Apache Iceberg for managing large-scale data lakes efficiently. Knowledge of Data Governance Tools:Experience ...

Senior Data Engineer, Fabric & Analytics (Remote)

Location
Greater London, England, United Kingdom
support comprehensive data analysis and reporting. Implement and uphold best practices in data governance, security, and compliance, using tools like Azure Purview, UnityCatalog, and Apache Atlas tomaintaindata integrity and trust. Ensure data quality and integrity through meticulous attention to detail and rigorous QA processes, continually refining and optimising data … large-scale data systems with high performance and low latency, such as managing 1 billion+ records or terabyte-sized databases. Knowledge of Delta TablesorApache Iceberg:Understanding and experience with Delta Tables or Apache Iceberg for managing large-scale data lakes efficiently. Knowledge of Data Governance Tools:Experience ...

Principal Data Engineer

Location
Greater London, England, United Kingdom
Experience designing data products across source-aligned, aggregate and consumer-aligned patterns. Proven ability to architect and scale modern cloud data platforms using Snowflake, Apache Iceberg, Amazon Redshift, and S3. Hands-on expertise in developing robust data transformation pipelines and orchestrating complex workflows using dbt and Apache ...

Lead Data Engineer - Perm up to £100k

Hiring Organisation
Lorien Resourcing Limited
Location
Manchester, North West, United Kingdom
Employment Type
Permanent Contract, Hybrid
Redshift and EMR Advanced Python, PySpark and SQL Production experience with Kafka or Amazon MSK, including Spark Structured Streaming Hands-on experience with Apache Iceberg or Delta Lake Strong data modelling and lakehouse design (bronze, silver, gold) Terraform, CI/CD and pipeline orchestration A track record … pairing and design sessions Confidence communicating with technical and non-technical stakeholders, and challenging constructively Desirable: SQL Server or on-prem to cloud migration, Apache Flink, regulated industry experience If you'd rather shape a platform than inherit a finished one, and want a seat at the table while ...

Senior Platform Engineer

Location
Greater London, England, United Kingdom
main language. Understanding how Spark runs on the JVM. Backend services and APIs. Change data capture or database replication. Open table formats such as Iceberg, Delta or Hudi. AWS data services, including Glue, Athena, DMS, MWAA or RDS. Cloud security and networking, including IAM, VPCs and secrets management. ...

Senior Data Engineer

Location
Stevenage, England, United Kingdom
Airflow, Azure, Microsoft SSIS), streaming (Spark, Kafka). Strong Python and SQL; C++, C#, or MATLAB a plus. Databases: PostgreSQL, SQL Server, Oracle, Snowflake, Iceberg, TimescaleDB. Data Platform Capabilities: Low-Code No-Code Data Science solutions (Dataiku, Knime), Data Virtualisation (Dataiku, Ab Initio, Starburst), Data Visualisation (Power ...

AWS Architect

Location
Greater London, England, United Kingdom
Amazon S3 AWS Lake Formation Amazon Athena Experience building Lakehouse architectures and modern data platforms. Experience working with data formats such as Parquet and Iceberg tables. Strong experience with Terraform for infrastructure automation. #J-18808-Ljbffr ...

Senior Software Engineer II (Data & AI Platform)

Location
Greater London, England, United Kingdom
Terraform, Kafka/MSK, Airflow, Glue or Spark, and building scalable data solutions Experience with modern data ecosystems including data lakes/lakehouse architectures, Iceberg or similar table formats, as well as batch and streaming processing Knowledge of data quality, governance, cataloguing and observability tools (e.g. Datadog), with ...

Senior Data Scientist

Location
Greater London, England, United Kingdom
MLflow, VS Code or JupyterHub for model development, experimentation and deployment.* Work with technologies to access, manage and analyse data at scale (e.g., Apache Iceberg and Trino).**Delivery Excellence*** Participate fully in Agile ceremonies including sprint planning, backlog refinement, demonstrations and retrospectives.* Produce high-quality technical documentation … machine-learning models, including monitoring, drift detection and performance management.* Awareness of modern lakehouse or enterprise data platforms and technologies such as Trino and Apache Iceberg.* Experience in NLP, large language models, Generative AI, deep learning, graph analytics.* Responsible AI or retrieval-augmented generation.* Delivery of data products ...

Senior Data Scientist

Location
City of Westminster, England, United Kingdom
MLflow, VS Code or JupyterHub for model development, experimentation and deployment. Work with technologies to access, manage and analyse data at scale (e.g., Apache Iceberg and Trino). Delivery Excellence Participate fully in Agile ceremonies including sprint planning, backlog refinement, demonstrations and retrospectives. Produce high-quality technical documentation … machine-learning models, including monitoring, drift detection and performance management. Awareness of modern lakehouse or enterprise data platforms and technologies such as Trino and Apache Iceberg. Experience in NLP, large language models, Generative AI, deep learning, graph analytics. Responsible AI or retrieval-augmented generation. Delivery of data products ...

Data Engineer, Data Platform

Location
Greater London, England, United Kingdom
platforms. You will leverage cloud-native technologies and AWS tools such as AWS S3, EKS, Glue, Airflow, Trino, and Parquet, while preparing to adopt Apache Iceberg for even greater performance and flexibility. You’ll tackle high-performance data workloads, ensuring seamless execution of massive queries, including 600+ billion ...

Senior Data Engineer

Location
United Kingdom
Dagster or Airflow, and AWS Pragmatic data modelling and a real care for data quality Startup or scale-up background, comfortable with ambiguity Iceberg, CDC or streaming is a bonus What's on offer £85,000 - £100,000, fully remote within the UK Private health insurance, income protection ...

Senior Data Engineer

Location
Greater London, England, United Kingdom
Tool (DBT) with a demonstrated ability indeveloping data models, contracts, tests, validation, and transformations Experienceworking with modern data distributed file formats (i.e., Parquet, Delta,Iceberg, Hudi) Demonstratedexperience with building data ingestion pipelines from REST API data sources. Strongability to produce technical documentation that can be understood by bothtechnical ...

Lead Data Engineer - AWS / AI / SQL

Hiring Organisation
Adria Solutions Ltd
Location
Manchester, United Kingdom
Employment Type
Permanent
Salary
£70000 - £90000/annum
Lead Data Engineer Manchester | Hybrid | Circa £85K An established organisation is looking for a hands-on Lead Data Engineer to take ownership of its existing data estate and lead the development of a modern, scalable ...

Lead Data Engineer - AWS / AI / SQL

Hiring Organisation
Adria Solutions
Location
Manchester, North West, United Kingdom
Employment Type
Permanent Contract, Hybrid
Salary
£90,000
Lead Data EngineerManchester | Hybrid | Circa £85K An established organisation is looking for a hands-on Lead Data Engineer to take ownership of its existing data estate and lead the development of a modern, scalable data ...

Data Architect (Azure Databricks, Specialty Insurance)

Location
Greater London, England, United Kingdom
self‐service analytics. Select and design for the right storage and exchange formats for each workload, including row‐based and columnar storage, Delta and Iceberg table formats, and file and message formats (e.g. Parquet, CSV, Excel, XML, JSON, Protobuf). Optimise platform performance and cost, including partitioning vs. liquid … designing metadata‐driven data ingestion or ELT frameworks. Experience working with a range of data formats, including row‐based and columnar storage, Delta and Iceberg table formats, and common file formats (e.g. CSV, XML, JSON). Experience applying data governance and security practices on a lakehouse platform, including access ...

Data Engineering Lead – AWS & AI - Manchester

Hiring Organisation
Adria Solutions Ltd
Location
Manchester, United Kingdom
Employment Type
Permanent
Salary
£70000 - £95000/annum
Data Engineering Lead – AWS & AI - Manchester An established financial services organisation is looking for a hands-on Data Engineering Lead to take ownership of its existing data estate and deliver a modern platform supporting Business ...