21 of 21 Apache Iceberg Jobs

JumpStart, prompt engineering, RAG architectures, vector databases, and 2) **Data Platforms****:** AWS Glue, SageMaker Unified Studio, Databricks on AWS, Snowflake on AWS, Redshift, EMR, Apache Iceberg, Delta Lake.• Position AWS solutions effectively against other cloud platforms' offerings in both generative AI (Azure OpenAI, Vertex AI) and data platforms … governance, data quality.* Proficiency in Python, SQL, and Spark with hands-on experience in:* Generative AI: LangChain, vector databases, embedding models.* Data Engineering: PySpark, Apache Iceberg/Delta Lake, orchestration tools.* A proven ability to articulate both visionary AI possibilities and practical data platform requirements to diverse audiences.#LI ...

InnovationResearch, evaluate, and recommend new technologies and tools that enhance Arriva's data and BI capabilitiesStay current with emerging trends including open table formats (Apache Iceberg, Delta Lake), data observability, real-time/streaming architectures, and cloud-native solutionsConduct proofs-of-concept and technical assessments to validate … understanding of modern data engineering patterns including ETL/ELT, data lake architectures, and lakehouse approachesUnderstanding of emerging technologies such as open table formats (Apache Iceberg, Delta Lake), data observability tools, and real-time/streaming architecturesExperience with data governance frameworks, data quality tools, metadata management, and ensuring ...

evaluate, and recommend new technologies and tools that enhance Arriva's data and BI capabilities* Stay current with emerging trends including open table formats (Apache Iceberg, Delta Lake), data observability, real-time/streaming architectures, and cloud-native solutions* Conduct proofs-of-concept and technical assessments to validate … modern data engineering patterns including ETL/ELT, data lake architectures, and lakehouse approaches* Understanding of emerging technologies such as open table formats (Apache Iceberg, Delta Lake), data observability tools, and real-time/streaming architectures* Experience with data governance frameworks, data quality tools, metadata management, and ensuring ...

evaluate, and recommend new technologies and tools that enhance Arriva's data and BI capabilities* Stay current with emerging trends including open table formats (Apache Iceberg, Delta Lake), data observability, real-time/streaming architectures, and cloud-native solutions* Conduct proofs-of-concept and technical assessments to validate … modern data engineering patterns including ETL/ELT, data lake architectures, and lakehouse approaches* Understanding of emerging technologies such as open table formats (Apache Iceberg, Delta Lake), data observability tools, and real-time/streaming architectures* Experience with data governance frameworks, data quality tools, metadata management, and ensuring ...

root‐cause analysis. Experience building or supporting production data pipelines in a collaborative engineering environment. Experience working with distributed data processing frameworks such as Apache Spark . Working knowledge of common data formats such as JSON , Avro and Parquet . Technology Environment The role will involve working with … should bring relevant experience and the ability to work across comparable technologies. Examples of technologies in scope include: Data processing and logic: ANSI SQL, Apache Spark, Kafka Platforms and storage: Snowflake, Apache Iceberg, Databricks, Hadoop ecosystem technologies, Sybase IQ Engineering and deployment: CI/CD tooling, containerised ...

with a backend or software engineering background Desirable: Experience with Snowflake Orchestration tools such as Dagster or Airflow dbt for data transformation and modelling Apache Iceberg or modern table formats Background in high-growth, consumer-facing products SALARY AND BENEFITS Base salary up to £100,000 Competitive benefits ...

Code tools such as AWS CDK, CloudFormation, Terraform, or AWS CLI Experience with real-time stream processing and Complex Event Processing (CEP) using Apache Flink or similar technologies is a plus Experience with automated testing frameworks, specifically Playwright for end-to-end and UI testing Strong focus on scalability … monitoring and analytics tools such as Grafana, InfluxDB, or similar Experience with cloud data stores and query engines such as Amazon Redshift, Athena, DynamoDB, Apache Iceberg and S3-based data lakes a plus Understanding of data modeling and transformation using AWS services such as Glue, Lambda, Redshift, Athena ...

Skill Requirements:*** Strong experience in **Solution Architecture** and **Cloud-Native Architecture Design*** Hands-on expertise in **Snowflake Architecture*** Experience with **Data Lakehouse architectures**, including **Apache Iceberg*** Proficiency in designing and managing **S3-based Data Lakes*** Experience with **Apache Airflow/MWAA** for workflow orchestration* Strong knowledge … scale data processing* Ability to create **Architecture Design Documents (ADDs)*** Deep understanding of **cloud-native design patterns*** Experience in **performance tuning** across:* Snowflake* Airflow* Iceberg* Focus on **platform reliability, scalability, and observability*** Experience designing and operating **data platforms** in production environments #J-18808-Ljbffr ...

Agile (Scrum) delivery environments Strong communication and stakeholder-management skills Proactive, collaborative, and delivery-focused mindset Desirable Skills Knowledge of open table formats (Iceberg, Delta) Experience with additional AWS services (CloudWatch, SNS, Athena, DynamoDB, EMR, Kinesis) Data modelling Job scheduling and orchestration tools Data virtualisation tools (e.g. Denodo ...

data pipelines in AWS using services such as S3, Glue or EKS Hands on experience with modern analytics stacks such as Snowflake, dbt and Iceberg Familiarity with orchestration tools such as Dagster or Airflow Strong understanding of data modelling concepts and analytics best practice A software engineering mindset, including ...

data pipelines in AWS using services such as S3, Glue or EKS Hands on experience with modern analytics stacks such as Snowflake, dbt and Iceberg Familiarity with orchestration tools such as Dagster or Airflow Strong understanding of data modelling concepts and analytics best practice A software engineering mindset, including ...

/analytics, ML/GenAI (RAG, LLMOps) . Strong familiarity with Snowflake, Databricks, Azure/AWS/GCP data services, Kafka, Delta/Iceberg, Power BI/Tableau , and MLOps tooling. Demonstrated account growth and commercial ownership (SoW creation, pricing, margin, change orders). Executive presence: can influence ...

data platforms (Databricks, Snowflake, Synapse, BigQuery) Ensure robust data engineering practices (Airflow, dbt, Prefect) and governance (Collibra, Purview) Drive scalable lakehouse architectures (Delta Lake, Iceberg, Hudi) Cloud, Infrastructure & MLOps Champion multi-cloud engineering across Azure, AWS, and GCP Lead MLOps strategy (MLflow, Kubeflow, SageMaker, Vertex AI) and CI/ ...

Experience with data governance frameworks, metadata management, and regulatory compliance It would be a bonus if you have: Familiarity with emerging technologies such as Apache Iceberg, Delta Lake, or data observability tooling Experience leading architecture workshops with both technical and non-technical stakeholders A background providing technical mentorship ...

site business environment Strong proficiency in Python and SQL, with hands-on experience in Spark/PySpark and modern table formats (e.g. Delta Lake, Iceberg, Hudi) Expertise in AWS data services (e.g. S3, Glue, Redshift, Lambda, Step Functions, Kinesis) and infrastructure-as-code (Terraform or CDK) Experience with workflow ...

Kingdom. The role demands strong expertise in Solution Architecture, Cloud-Native Architecture Design, and hands-on experience with Snowflake and Data Lakehouse architectures, including Apache Iceberg. Responsibilities involve creating Architecture Design Documents, managing S3-based Data Lakes, and ensuring platform performance tuning and observability. Ideal candidates should have substantial ...

BigQuery, GCS, Dataflow) Deep expertise in: Databricks (Lakehouse architecture, Delta Lake, Unity Catalog, Delta Sharing, Pyspark, Spark SQL etc.) Snowflake (data warehousing & data sharing, Iceberg, ) Strong understanding of: Data architecture patterns, industry benchmarking, Data integration patterns (ETL/ELT, streaming, CDC, APIs) Data governance & metadata management Architecture ...

modelling, medallion architecture) Leading the design of cloud data platforms (AWS) – including services like Redshift, S3, Glue, Athena Evaluating and introducing new technologies (e.g. Iceberg, Delta Lake, observability tools) Driving data governance, standards, and best practice Acting as a trusted advisor to both technical and business stakeholders What ...

execute. You have deep experience designing, building, and operating production data pipelines, including owning reliability, monitoring, and failure recovery for orchestration systems such as Apache Airflow (or equivalent). You are fluent in modern data and cloud ecosystems, with hands-on experience (or the ability to ramp quickly … technologies such as Iceberg, Spark, PostgreSQL, and cloud-native infrastructure on AWS (or equivalent). You write clean, readable, and testable Python code, applying sound abstractions, naming, and review discipline to build systems that scale beyond the first version. You bring strong analytical and problem-solving skills ...

Architect and deliver a cloud-native data platform across ingestion, modelling, warehousing and BI Lead adoption of DBT, ELT patterns, AWS DMS, Redshift, Athena, Iceberg Build high-quality models (Kimball, Data Vault, OBT) and set the standards for testing, documentation and governance Shape BI strategy and ensure consistent, trusted … partner with Principal Engineers, Product and Analytics to deliver high-value data products Evolve the AWS-native platform: Redshift, Athena, S3 lake patterns, Iceberg Modernize DataOps: CI/CD for dbt and infra, automated testing, GitHub Actions Build high-reliability ingestion pipelines with strong SLAs and schema-change resilience ...

with cloud platforms such as AWS, Azure, or Google Cloud (e.g., S3, Redshift, Snowflake, or Databricks). Familiarity with event streaming platforms such as Apache Kafka, Kinesis, or similar. Strong attention to detail with a commitment to data accuracy and quality assurance. Ability to thrive in a fast-paced …/CD pipelines and infrastructure-as-code tools (e.g., Terraform, GitHub Actions). Familiarity with lakehouse architecture principles and tools like Delta Lake or Apache Iceberg. Experience building robust logging, error handling, and monitoring for data pipelines. Understanding of data governance, privacy, and compliance principles. Strong documentation and communication ...