2 of 2 Distributed Computing Jobs in South London

pipelines for both batch and real-time data processing. Apply solid understanding of data engineering concepts, including ETL/ELT processes, data transformation, and distributed computing frameworks. Implement best practises in containerisation technologies, CI/CD practises, and cloud-native application development. Utilise version control systems such ...

exposure to the Microsoft Fabric ecosystem (OneLake, Data Factory, Fabric Lakehouse).Big Data Processing: Deep technical expertise with Apache Spark/Azure Databricks for distributed compute, memory optimization, and Delta Lake formats. Data Engineering Languages: Advanced proficiency in SQL (T-SQL, SparkSQL) and Python (PySpark).Security & Compliance: Direct experience … data infrastructure as code using Terraform, Bicep, and managing deployments via Git/Azure DevOps. Soft SkillsExceptional analytical and root-cause debugging skills for distributed Spark jobs and multi-stage pipelines. Ability to collaborate with compliance and security teams to validate data privacy baselines. Strong communication skills to bridge ...