21 of 21 PySpark Jobs in the South East

understanding of data strategy and management, including data governance, data quality, security and compliance Advanced SQL skills, with hands-on experience using Python and PySpark Broad, practical experience across Azure data services and supporting Azure infrastructure Experience with Infrastructure-as-Code (Terraform or Bicep) and CI/CD pipelines ...

services such as Synapse, Data Factory, Databricks or Fabric Strong SQL skills with experience building optimised, complex queries Proficiency in Python and/or PySpark for data transformation and automation Strong experience with Power BI and modern data modelling techniques Demonstrable experience leading data engineering initiatives or projects Excellent ...

services such as Synapse, Data Factory, Databricks or Fabric Strong SQL skills with experience building optimised, complex queries Proficiency in Python and/or PySpark for data transformation and automation Strong experience with Power BI and modern data modelling techniques Demonstrable experience leading data engineering initiatives or projects Excellent ...

Strong proficiency in SQL , including Spark SQL and MS SQL Server , for querying, data manipulation, and performance optimization. Hands-on experience with Python and PySpark for data processing, transformation, and automation tasks. Proven ability to design, build, and maintain scalable data pipelines for batch and streaming data using modern ...

have: Hands-on experience creating data pipelines using Azure services such as Synapse, Data Factory or Databricks Strong understanding of SQL and Python/PySpark Experience with Power BI and data modelling Commercial experience with Microsoft Fabric would be advantageous Some of the package/role details include: Salary ...

have: Hands-on experience creating data pipelines using Azure services such as Synapse, Data Factory or Databricks Strong understanding of SQL and Python/PySpark Experience with Power BI and data modelling Commercial experience with Microsoft Fabric would be advantageous Some of the package/role details include: Salary ...

with various databases e.g. MS SQL, Azure Cosmos DB. Skilled at optimizing large and more complicated SQL statements. Proficiency in Python and experience with PySpark Experience of REST API standards. Proficiency in writing clean, readable, and secure code for Azure backend services, i.e. functions, utility functions etc. Experience ...

enquire about other roles, please contact Nick Mandella at Harnham. KEYWORDS Python, SQL, AWS, GCP, Azure, Cloud, Databricks, Docker, Kubernetes, CI/CD, Terraform, Pyspark, Spark, Kafka, machine learning, statistics, Data Science, Data Scientist, Big Data, Artificial Intelligence, private equity, finance. ...

DevOps and broader software engineering standards. Cloud experience (any platform). Previous mentoring experience. Nice to have: Snowflake or Databricks Spark, PySpark, Hadoop or similar big data tooling BI exposure (PowerBI, Tableau, etc.) Interview Process The process is fully structured, transparent, and efficient: Video call - high-level overview ...

Essential Strong experience with Azure Databricks Infrastructure as Code using Terraform DevOps practices and CI/CD pipelines Git and peer review workflows Python, PySpark, and SQL Cloud operations within Azure To apply for this role please submit your CV or contact Dillon Blackburn on or at . Tenth ...

analysis using tools like Power BI and Excel or similar. Knowledge and capability in a wide variety of additional analytical tools (e.g. SQL, Python, PySpark, Databricks and Azure). Able to support the development and technical progression of others within the team. Several years of relevant business or technical ...

optimize Lakehouse architectures using Delta Lake principles Ingest, transform, and curate data from multiple sources (APIs, databases, files, streaming) Develop scalable data transformations using PySpark and Spark SQL Implement data models optimized for analytics and reporting (e.g. star schemas) Monitor, troubleshoot, and optimize performance and cost of data workloads … practices Required Skills & Experience Strong experience with Microsoft Fabric (Lakehouse, Pipelines, Notebooks, Dataflows, OneLake) Hands-on experience with Databricks in production environments Proficiency in PySpark and SQL Solid understanding of data engineering concepts (ETL/ELT, orchestration, partitioning) Experience working with Delta Lake Familiarity with cloud platforms (Azure preferred ...

Experience 5+ years in data engineering, including experience in a senior or lead position Strong expertise across AWS Hands-on experience with Databricks, PySpark, and Lakehouse patterns Strong Python and SQL engineering ability Experience implementing scalable ETL/ELT pipelines Background in data modelling and building analytics-ready datasets ...

Role and Responsibility Design, build and maintain scalable data pipelines and products within a modern Databricks Lakehouse environment using Delta Lake, Spark, Python (PySpark) and SQL. Manage and optimise the cloud-based lakehouse, leveraging Unity Catalogue, Databricks Workflows and best practices for performance, quality, lineage and compliance. Work closely ...

heavily in the public sector - we're looking for Azure Data Engineers for some immediate contract roles. Key skills: Azure Data Engineering Fabric and PySpark Databricks Data Pipelines Medallion architecture Data warehouses Given the urgency of the roles and projects, we do need everyone to be available either immediately ...

Design, build and maintain scalable and secure data pipelines on the Azure platform. Develop and deploy data ingestion processes using Azure Data Factory, Databricks (PySpark), and Azure Synapse Analytics. Optimise ETL/ELT processes to improve performance, reliability and efficiency. Integrate multiple data sources including Azure Data Lake (Gen2 … Required Skills & Experience - Azure Data Engineer: Proven commercial experience as a Data Engineer delivering enterprise-scale solutions in Azure Azure Data Factory Azure Databricks (PySpark) SQL & Python Understanding of CI/CD in a data environment, ideally with tools like Azure DevOps. Experience working within consultancy or client-facing ...

with key AWS data services including Lake Formation, Glue, Athena, S3, IAM, and KMS . Extensive experience with Terraform , GitHub Actions , and Spark/PySpark is essential. Key Responsibilities: Architect end to end AWS data platforms (S3, Lake Formation, Glue, Athena, EMR/Spark). Design multi account data … governance, security, and access models. Develop scalable ETL/ELT pipelines using Python & PySpark . Implement IaC using Terraform and CI/CD using GitHub Actions . Optimize data pipelines for performance, quality, and cost. Collaborate with security, data science, and analytics teams. Must-Have Skills: 10+ years experience ...

with key AWS data services including Lake Formation, Glue, Athena, S3, IAM, and KMS . Extensive experience with Terraform , GitHub Actions , and Spark/PySpark is essential. Key Responsibilities: Architect end to end AWS data platforms (S3, Lake Formation, Glue, Athena, EMR/Spark). Design multi account data … governance, security, and access models. Develop scalable ETL/ELT pipelines using Python & PySpark . Implement IaC using Terraform and CI/CD using GitHub Actions . Optimize data pipelines for performance, quality, and cost. Collaborate with security, data science, and analytics teams. Must-Have Skills: 10 years experience ...

tagging, monitoring and platform operations. * Architect and design end-to-end data platform solutions focused on scalability, security and reliability. * Lead delivery using Databricks, PySpark, Spark SQL and Azure Data Factory. * Drive DevOps best practices including CI/CD, automated testing and infrastructure as code. * Collaborate closely with data … followed. Your Skills and Experience You will bring: * Strong commercial experience in Azure and Databricks, with deep knowledge of Lakehouse architecture. * Proficiency in PySpark, Python, SQL and Spark SQL. * Strong DevOps engineering skills including CI/CD, automated testing and deployment pipelines. * Experience with monitoring, logging and alerting solutions. ...

Lakehouse solutions across cloud environments. Responsibilities: Design, build, and maintain data pipelines using Databricks (Spark, Delta Lake). Develop batch and streaming workloads using PySpark/Spark SQL. Implement data ingestion, transformation, and orchestration pipelines. Optimise Databricks workloads for performance, reliability, and cost. Integrate Databricks with cloud-native services … across Azure, AWS, or GCP. Skills: Strong hands-on expertise with Databricks in production environments. Advanced PySpark and Spark SQL development experience. Experience building lakehouse architectures using Delta Lake. Strong cloud experience in Azure (preferred), AWS, or GCP. Experience with orchestration tools (ADF, Airflow, dbt, or similar). Certifications ...

building common functionality for distributed product teams. Early work will focus on: Abstracting existing codebases into shared Python libraries Developing data-driven solutions using PySpark and DataFrames Building and extending Python-based microservices using Azure Functions Creating scalable, reusable services that solve common challenges across teams Beyond the initial … essential, as youll be working closely with engineers and technical leads across five time zones . Essential skills Strong commercial experience with Python PySpark and data-frame based processing Solid SQL capability Experience working with Azure infrastructure Good understanding of containers, microservices, and functional design patterns Comfortable working ...