6 of 6 Apache Spark Jobs in Surrey

with the Lead Data Engineer and partnering with data science, analytics and product teams. Key responsibilities * Build and maintain scalable data pipelines using Databricks, Apache Spark and Delta Lake * Develop and optimise batch and streaming workflows for reliability and performance * Contribute to lakehouse architecture and medallion pattern implementation … optimisation opportunities * Translate business requirements into technical solutions * Contribute to documentation and continuous improvement Required experience * Strong experience delivering production grade Databricks pipelines * Strong Spark, Delta Lake and Databricks Workflows experience * Python and SQL proficiency * Experience on Azure data platforms such as Azure Data Factory, ADLS Gen2, Azure DevOps ...

workflows. Key Requirements Passion for building scalable, resilient cloud data platforms. Strong experience with Databricks or Snowflake on AWS. Proven Python skills, including Spark and Airflow expertise. Advanced SQL skills and end-to-end data modelling experience. Experience building batch and real-time data integrations. Hands-on CICD skills ...

platforms (AWS, Azure, or GCP).Distributed Systems Mastery: Deep technical capability in structuring data platforms, data pipelines, and high-throughput distributed systems (like Spark or Kafka) optimized for performance and scalability.Cross-Functional Technical Leadership: Exceptional ability to bridge the gap between deep technical engineering and strategic business needs, guiding ...

expectations, and feedback to team members, stakeholders, and customers. Foster collaboration and teamworkDesired:Advanced analytics degree.Experience with big data technologies (e.g. Hadoop, Hive, and Spark) is a plus.Why Join Us At BP, we support our people to grow in a diverse and exciting environment. We believe that our team ...

Azure, AWS or GCP* Define lakehouse standards, data modelling approaches and governance frameworks* Lead design of batch and streaming platforms using Delta Lake, Spark and related technologies* Establish security, access control, data quality and compliance standards* Optimise performance, scalability and cloud cost management* Lead and mentor a small team … improvementRequired experience* Strong experience as a senior or lead data engineer building and scaling production data platforms* Deep hands on expertise in Databricks including Spark, Delta Lake and Databricks Workflows* Python and SQL proficiency* Infrastructure as code experience using Terraform, Bicep or CloudFormation* Cloud data platform experience, ideally Azure ...

Azure, AWS or GCP * Define lakehouse standards, data modelling approaches and governance frameworks * Lead design of batch and streaming platforms using Delta Lake, Spark and related technologies * Establish security, access control, data quality and compliance standards * Optimise performance, scalability and cloud cost management * Lead and mentor a small team … Required experience * Strong experience as a senior or lead data engineer building and scaling production data platforms * Deep hands on expertise in Databricks including Spark, Delta Lake and Databricks Workflows * Python and SQL proficiency * Infrastructure as code experience using Terraform, Bicep or CloudFormation * Cloud data platform experience, ideally Azure ...