4 of 4 Permanent Apache Hudi Jobs in the UK

keys, sort keys , and workload management. Implement Redshift Spectrum to query data in S3 using external tables. 3. Data Lake & Iceberg Implement and maintain Apache Iceberg tables on AWS for schema evolution and ACID operations. Build Iceberg-based ingestion and transformation pipelines using Glue, EMR, or Spark. Ensure high … monitoring. Required Skills 8 10 years ' experience in data engineering. Strong hands-on experience with: AWS S3, Athena, Glue, Redshift, EMR/Spark Apache Iceberg Python/SQL Experience in telecom data pipelines and handling large-scale structured/semi-structured data. Strong problem-solving, optimization, and debugging skills. ...

uplift practices and delivery maturity. Required Technical Skills & Expertise AWS Data & Analytics (Hands-On Expertise) AWS Glue (ETL/ELT development, PySpark, Glue Catalog) Apache Spark AWS Redshift (modelling, optimisation) AWS Lambda AWS RDS AWS Step Functions Experience with Open Table Formats: Delta Lake Apache Hudi Strong ...

Terraform and GitHub Actions CI/CD implementation skills. Experience with multi-account AWS architectures and governance. Nice-to-Have: Iceberg/Delta/Hudi, DMS, EMR tuning. Data quality & lineage tools (Datahub). Exposure to regulated industries (Banking/Finance). ...

Excellence Lead the evolution of the enterprise data landscape using modern platforms such as Databricks, Snowflake, Azure Synapse, and BigQuery. Oversee data engineering with Apache Airflow, dbt, and Prefect, ensuring performance, governance, and alignment with enterprise metadata standards (Collibra, Alation, Microsoft Purview). Drive adoption of Delta Lake, Iceberg … Hudi to support scalable data lakehouse architectures. Ensure high‐quality, compliant, and reliable data foundations for ML and analytics workloads. Cloud, Infrastructure & MLOps Champion multi‐cloud architecture across Azure, AWS, and GCP. Ensure resilient, secure, and cost‐efficient deployments using Docker, Kubernetes (AKS/EKS/GKE), and Terraform ...