14 of 14 Apache Jobs in the South East

while collaborating with the Data & Analytics Lead, technical colleagues and business stakeholders. Experience in the following areas would also be useful: Materialised Lake Views Apache Spark or Python. Power BI semantic modelling. APIs and cloud business system integration. Data governance, lineage and quality frameworks. Professional services experience. What ...

skills in Python, and experience with Java or Scala. Data Engineering at Scale: Solid ELT/ETL design experience and hands-on work with Apache Spark. Bonus points if you have: Multi-Cloud Exposure: Experience with Azure alongside GCP. Container Orchestration: Experience packaging and deploying workloads with Helm ...

architecture standards and best practices Working closely with business and technical stakeholders Leading data transformation and cloud migration initiatives Desirable Databricks, Snowflake, dbt Kafka, Apache Iceberg, Delta Lake Data Mesh/Data Fabric AI & ML-enabled data platforms ...

Skills & Experience Strong commercial experience in data engineering within cloud-based environments. Advanced Python and SQL development skills. Hands-on experience with Azure Databricks, Apache Spark and PySpark. Good understanding of Delta Lake, lakehouse architecture and modern data modelling. Experience building and managing production data pipelines, including batch ...

Familiarity with monitoring tools such as CloudWatch, Prometheus or Grafana. Useful additional experience Relevant AWS certifications. Exposure to data-processing environments using Databricks or Apache Spark. The person we are looking for You enjoy making systems easier to run and reducing the need for manual intervention. You investigate problems ...

Data Engineer 6 Month Contract Location: London (Onsite 4 days a week) Rate: £700 - £800 per day (INSIDE IR35) Skills: Python, Azure, Kubernetes, Apache Airflow, Apache Spark, DV Clearance We are looking to recruit a DV Cleared Data Engineer for a leading technology consultancy. This will be joining … Work across Azure environments on the low side Support data engineering workloads within Kubernetes environments on the high side Develop and maintain workflows using Apache Airflow Work with Apache Spark for large-scale data processing Support the ongoing development and improvement of established data pipelines Help ensure data ...

Snowpark Iceberg Tables Secure Data Sharing Snowflake Cortex Data Integration Fivetran SnapLogic Enterprise SaaS integration patterns Data Transformation & Orchestration dbt Cloud/dbt Core Apache Airflow (MWAA) CI/CD pipelines Event-driven orchestration Data Governance & Observability Collibra Data Catalogue Data Lineage Data Governance Frameworks Acceldata Data Observability Data ...

role). Cloud Infrastructure & Data Pipelines: Build with AWS and Azure services to construct ETL workflows and dynamic systems using orchestration platforms like Apache Airflow. Production Operations: Support production applications, maintain SLAs, and create continuous evaluation metrics for ML solution performance. Qualifications & Skills 2-5 years' hands-on software ...

knowledge sharing and mentoring across the team. What we are looking for Practical experience managing Databricks in a production environment. A strong understanding of Apache Spark, Delta Lake and Unity Catalog. Experience working with one or more major cloud providers, including storage, networking and identity management. Confidence using Terraform ...

kubernetes) Experience with Key value stores Experience in git and basic Unix Commands You might also have (Desirable) Experience with large data processing technologies (Apache Spark) Other helpful information Hybrid Working: At EMBL-EBI we are pleased to offer hybrid working options for all our employees. A dedicated desk ...

cloud data AWS Aurora PostgreSQL AWS Database Migration Service (DMS) Snowflake Azure Google Cloud Platform (GCP) Data integration and engineering Informatica Talend Apache Airflow Python for data processing and automation Data warehouse design and optimisation APIs and event-driven architecture REST APIs GraphQL Apache Kafka Event-driven architectures ...

other genome browser Experience working in a multi-person team delivering production services Experience with large-scale biological datasets Experience working with the Apache Parquet file format Experience implementing AI/ML approaches for quality control and/or genome annotation Behaviours we value in our team: You bring ...

business. Key Responsibilities: Design, build and enhance a secure, scalable and reliable Databricks platformDevelop and maintain batch and real-time data pipelines using Databricks, Apache Spark and Delta LakeContribute to platform architecture, engineering standards and best practices across the data estateWork with Product, Analytics and Data Science teams … contribute to continuous improvement across the engineering teamJob Requirements: Significant experience building and supporting scalable, production-grade data platformsStrong hands-on expertise with Databricks, Apache Spark, Delta Lake, Python and SQLGood understanding of cloud-based data engineering, ideally within Microsoft Azure environmentsExperience with Infrastructure as Code tools such ...

Data Engineer/Architect - Apache Airflow - Astronomer Location: London - 3 days a week onsite (non-negotiable) Duration: 6 Months Rate: £700-£800 per day Inside IR35 The Role We're supporting a highly reputable client in the Financial Services industry who are seeking an experienced Data Engineer to support … them through a period of significant change across their Technology Delivery function. The team are in the early stages of adopting Apache Airflow and Astronomer as their core orchestration platform/tooling for their DBT projects. To move forward with the process and maximise the tooling, the team require ...