24 of 24 Permanent Apache Iceberg Jobs

role-this is your chance to engineer solutions that truly matter. Key Responsibilities: Design, develop, and optimize scalable data pipelines using technologies such as Apache Spark, Apache Iceberg, Trino, OpenSearch, AWS EMR, NiFi, and Kubernetes containers. Ingest and move structured and unstructured data using approved methods into … of working with diverse data types and formats, including structured, semi-structured, and unstructured data. Familiarity with data ingestion tools and platforms such as Apache NiFi, Spark, and related open-source technologies. Demonstrated ability to collaborate across teams, including data scientists, software engineers, data stewards, and mission partners. Knowledge More ❯

design, implementation, testing, and support of next-generation features related to Dremio's Query Planner and Reflections technologies Work with open source projects like Apache Calcite and Apache Iceberg Use modular design patterns to deliver an architecture that's elegant, simple, extensible and maintainable Solve complex technical … distributed query engines. Hands on experience in query processing or optimization, distributed systems, concurrency control, data replication, code generation, networking, storage systems, heap management, Apache Arrow, SQL Operators, caching techniques, and disk spilling Hands on experience with multi-threaded and asynchronous programming models More ❯

grasp of data governance/data management concepts, including metadata management, master data management and data quality. Ideally, have experience with Data Lakehouse toolset (Iceberg) What you'll get in return Hybrid working (4 days per month in London HQ + as and when required) Access to market leading More ❯

Scala Starburst and Athena Kafka and Kinesis DataHub ML Flow and Airflow Docker and Terraform Kafka, Spark, Kafka Streams and KSQL DBT AWS, S3, Iceberg, Parquet, Glue and EMR for our Data Lake Elasticsearch and DynamoDB More information: Enjoy fantastic perks like private healthcare & dental insurance, a generous work More ❯

working with hierarchical reference data models. Proven expertise in handling high-throughput, real-time market data streams. Familiarity with distributed computing frameworks such as Apache Spark. Operational experience supporting real-time systems. Equal Opportunity Workplace We are proud to be an equal opportunity workplace. We do not discriminate based More ❯

data. What you offer Experience with AWS cloud. Experience programming, debugging, and running production systems in Python. Exposure to open-source technologies such as Iceberg, Trino, and Airflow. Passionate about the use and adoption of these capabilities, focused on user experience and ensuring our business sees real value from More ❯

to-end engineering experience supported by excellent tooling and automation. Preferred Qualifications, Capabilities, and Skills: Good understanding of the Big Data stack (Spark/Iceberg). Ability to learn new technologies and patterns on the job and apply them effectively. Good understanding of established patterns, such as stability patterns More ❯

analysis and automation. Proficiency in building and maintaining batch and streaming ETL/ELT pipelines at scale, employing tools such as Airflow, Fivetran, Kafka, Iceberg, Parquet, Spark, Glue for developing end-to-end data orchestration leveraging on AWS services to ingest, transform and process large volumes of structured and More ❯

delivering customer proposals aligned with Analytics Solutions. Experience with one or more relevant tools (Sqoop, Flume, Kafka, Oozie, Hue, Zookeeper, HCatalog, Solr, Avro, Parquet, Iceberg, Hudi). Experience developing software and data engineering code in one or more programming languages (Java, Python, PySpark, Node, etc). AWS and other More ❯

for each project including ETL mappings, code use guide, code location and access instructions. Design and optimize Data Pipelines using tools such as Spark, Apache Iceberg, Trino, OpenSearch, EMR cloud services, NiFi and Kubernetes containers Ensure the pedigree and provenance of the data is maintained such that the More ❯

processes using infrastructure-as-code (Terraform) Build and maintain data pipelines using Airflow. Manage our tech stack including Python, Node.js, PostgreSQL, MongoDB, Kafka, and Apache Iceberg. Optimize infrastructure costs and develop strategies for efficient resource utilization. Provide critical support by monitoring services and resolving production issues. Contribute to the More ❯

using tools and techniques such as BDD, Data Reconciliation, Source Control, TDD, Jenkins. Documenting configurations, processes, and best practices. Knowledge of file formats JSON, Iceberg, Avro. Basic knowledge of AWS technologies like IAM roles, Lakeformation, Security Groups, CloudFormation, Redshift. Big Data/Data Warehouse testing experience. Experience in the More ❯

a highly skilled Software Engineer with expertise in SQL and either Python or Scala, who is experienced in building large-scale data pipelines using Apache Spark and designing robust data architectures on AWS. The ideal candidate will have hands-on experience in data lake architectures, open table formats (Delta … Lake/Iceberg), and modern data platforms. If you are a problem solver, a data infrastructure enthusiast, and someone who thrives in fast-paced environments, we'd love to hear from you! Job Description Core Responsibilities Collaborate with project stakeholders and cross-functional teams (including Frontend Service Engineers) to … learning applications, and real-time analytics. We process tens of billions of ad events daily, leveraging a modern data stack that includes Databricks, AWS, Apache Spark, ClickHouse, Snowflake, and Google Looker. Disclaimer: This information has been designed to indicate the general nature and level of work performed by employees More ❯

Preferred Qualifications: Experience with software development. Experience with geospatial data. Experience building data-streaming processes. Experience using PostGIS. Experience with any of the following: Apache-Hive, Trino, Presto, Starburst, OpenMetadata, Apache-SuperSet, Terraform, dbt, Tableau, Fivetran, Airflow. Experience implementing resilient, scalable, and supportable systems in AWS. Experience using More ❯

Salary range: £70,000-£80,000 + 10% bonus + benefits Purpose: Build and maintain large, scalable Data Lakes, processes and pipelines Tech: Python, Iceberg/Kafka, Spark/Glue, CI/CD Industry: Financial services/securities trading Immersum continue to support a leading SaaS securities trading platform … Infra tooling using Terraform, Ansible and Jenkins whilst automating everything with Python Tech (experience in any listed is advantageous) Python Cloud: AWS Lake house: Apache Spark or AWS Glue Cloud Native storage: Iceberg, RDS, RedShift, Kafka IaC: Terraform, Ansible CI/CD: Jenkins, Gitlab Other platforms such as More ❯

Salary range: £70,000-£80,000 + 10% bonus + benefits Purpose: Build and maintain large, scalable Data Lakes, processes and pipelines Tech: Python, Iceberg/Kafka, Spark/Glue, CI/CD Industry: Financial services/securities trading Immersum continue to support a leading SaaS securities trading platform … Infra tooling using Terraform, Ansible and Jenkins whilst automating everything with Python Tech (experience in any listed is advantageous) Python Cloud: AWS Lake house: Apache Spark or AWS Glue Cloud Native storage: Iceberg, RDS, RedShift, Kafka IaC: Terraform, Ansible CI/CD: Jenkins, Gitlab Other platforms such as More ❯

AWS Data Engineer S3 Data Centre of Excellence AWS Data Engineer S3 Data Centre of Excellence Country: United Kingdom Interested in part-time, job-share or flexible working? We want to talk to you! Join our community. We have an More ❯

scripts using Python and GitHub pipelines to enhance operational efficiency. Document workflows and ensure alignment with business objectives. Maintain expertise in technologies like Immuta, Apache Ranger, Collibra, Snowflake, PostgreSQL, Redshift, Hive, Iceberg, dbt, AWS Lambda, AWS Glue, and Power BI. Provide insights and recommendations for platform improvements and … Proficiency in Python and YAML scripting for automation and configuration. Experience working with Kubernetes (k8s). Knowledge of data and analytical products like Immuta, Apache Ranger, Collibra, Snowflake, PostgreSQL, Redshift, Hive, Iceberg, dbt, AWS Lambda, AWS Glue, and Power BI. Familiarity with cloud environments such as AWS. Knowledge More ❯

Proficiency in Python and YAML scripting for automation and configuration. Experience working with Kubernetes (k8s). Knowledge of data and analytical products like Immuta, Apache Ranger, Collibra, Snowflake, PostgreSQL, Redshift, Hive, Iceberg, dbt, AWS Lambda, AWS Glue, and Power BI. Familiarity with cloud environments such as AWS. Knowledge More ❯

Staff Software Engineer - Data Department: Engineering Employment Type: Full Time Location: Cardiff Description is the UK's first comparison platform for car insurance. We've been helping customers since 2002 by empowering them to make better decisions around insurance and More ❯

with poly required to start Desired Experience Experience with the Sponsor's data environment and on-premises compute structure. Experience with Glue, Hive, and Iceberg or similar technologies. Experience with Terraform. Experience with DevSecOps solutions and tools. Experience with Data Quality and Data Governance concepts and experience. Experience maintaining … supporting, and improving the ETL process using Apache NiFi or similar tools. Experience with Apache Spark. Equal Opportunity Employer/Veterans/Disabled Accommodations: If you are a qualified individual with a disability or a disabled veteran, you may request a reasonable accommodation if you are unable or More ❯

data migration efforts. 5. (Mandatory) Demonstrated experience with database architecture, performance design methodologies, and system-tuning recommendations. Preference for familiarity with Glue, Hive, and Iceberg or similar 6. (Mandatory) Demonstrated experience with Python, Bash, and Terraform 7. (Mandatory) Demonstrated experience with DevSecOps solutions and tools 8. (Mandatory) Demonstrated experience … Governance concepts and experience. 11. (Desired) Demonstrated experience maintaining, supporting, and improving the ETL process through the implementation and standardization of data flows with Apache Nifi and other ETL tools. 12. (Desired) Demonstrated experience with Apache Spark B4CORP Company Information B4Corp is a small defense contracting company that More ❯

the needs of our growing business. Responsibilities: You will be responsible for supporting the migration of DBT models from Redshift and Snowflake to Trino (Iceberg) while ensuring compatibility, performance, and best practices in our evolving infrastructure. Your role will require a deep understanding of SQL dialects, data modeling principles … a plus. Competencies (Attributes Necessary for Success in this Role): Strong proficiency in SQL, with experience in Multiple SQL Syntaxes (Redshift, Snowflake, and Trino (Iceberg). Deep understanding of dbt, including model development, macros, and Jinja templating. Experience with BI tools, such as Tableau, and a working knowledge of … a must. Experience working in startup environments is a plus. Strong proficiency in SQL, with experience in Multiple SQL Syntaxes (Redshift, Snowflake, and Trino (Iceberg). Deep understanding of dbt, including model development, macros, and Jinja templating. Experience with BI tools, such as Tableau, and a working knowledge of More ❯

knowledge of warehousing and ETLs. Extensive knowledge of popular database providers such as SQL Server, PostgreSQL, Teradata and others. • Proficiency in technologies in the Apache Hadoop ecosystem, especially Hive, Impala and Ranger • Experience working with open file and table formats such Parquet, AVRO, ORC, Iceberg and Delta Lake More ❯