14 of 14 PySpark Jobs in Manchester

with Distributed computing frameworks knowledge: Hive/Hadoop, Apache Spark, Kafka, Airflow Working with programming languages Python , Java, SQL. Working on building ETL (Extraction Transformation and Loading) solution using PySpark Experience in SQL/NoSQL database design Deep understanding in software architecture, object oriented design principles, and data structures Extensive experience in developing microservices using Java, Python Good experience More ❯

Enhanced Maternity & Paternity Charity Volunteer Days Cycle to work scheme And More.. DATA ENGINEER – ESSTENTIAL SKILLS Proven experience building data pipelines using Databricks . Strong understanding of Apache Spark (PySpark or Scala) and Structured Streaming . Experience working with Kafka (MSK) and handling real-time data . Good knowledge of Delta Lake/Delta Live Tables and the Medallion More ❯

data services – Databricks, ADF, ADLS, Power BI. Proficiency in SQL and data profiling for test design and validation. Hands-on experience with test automation frameworks such as Python/PySpark, Great Expectations, Pytest, or dbt tests. Practical understanding of CI/CD integration (Azure DevOps, GitHub Actions, or similar). Strong problem-solving skills and the ability to work More ❯

assurance, modeling, and delivery to downstream systems. Essential Technical Requirements 5+ years proven experience in a Data Engineering role, building production-ready pipelines. Deep hands-on expertise with Databricks (PySpark/SparkSQL) environments (2+ years desirable). Expert proficiency in Python and SQL development. Experience using Databricks. Experience with cloud-based data platforms (Azure, AWS, or GCP). Prior More ❯

assurance, modeling, and delivery to downstream systems. Essential Technical Requirements 5+ years proven experience in a Data Engineering role, building production-ready pipelines. Deep hands-on expertise with Databricks (PySpark/SparkSQL) environments (2+ years desirable). Expert proficiency in Python and SQL development. Experience using Databricks. Experience with cloud-based data platforms (Azure, AWS, or GCP). Prior More ❯

assurance, modeling, and delivery to downstream systems. Essential Technical Requirements 5+ years proven experience in a Data Engineering role, building production-ready pipelines. Deep hands-on expertise with Databricks (PySpark/SparkSQL) environments (2+ years desirable). Expert proficiency in Python and SQL development. Experience using Databricks. Experience with cloud-based data platforms (Azure, AWS, or GCP). Prior More ❯

assurance, modeling, and delivery to downstream systems. Essential Technical Requirements 5+ years proven experience in a Data Engineering role, building production-ready pipelines. Deep hands-on expertise with Databricks (PySpark/SparkSQL) environments (2+ years desirable). Expert proficiency in Python and SQL development. Experience using Databricks. Experience with cloud-based data platforms (Azure, AWS, or GCP). Prior More ❯

assurance, modeling, and delivery to downstream systems. Essential Technical Requirements 5+ years proven experience in a Data Engineering role, building production-ready pipelines. Deep hands-on expertise with Databricks (PySpark/SparkSQL) environments (2+ years desirable). Expert proficiency in Python and SQL development. Experience using Databricks. Experience with cloud-based data platforms (Azure, AWS, or GCP). Prior More ❯

within a cross-functional environment. Experience in line management or team leadership, with a track record of developing and supporting engineers. AWS (Lambda, Glue, ECS, S3, etc.) Python and PySpark (data pipelines, APIs, automation) TypeScript and React (frontend development) Excellent communication and stakeholder management skills. Demonstrated expertise in technical design and architecture of distributed systems. Familiarity with fraud detection More ❯

within a cross-functional environment. Experience in line management or team leadership, with a track record of developing and supporting engineers. AWS?(Lambda, Glue, ECS, S3, etc.) Python and PySpark?(data pipelines, APIs, automation) TypeScript and React?(frontend development) Excellent communication and stakeholder management skills. Demonstrated expertise in technical design and architecture of distributed systems. Familiarity with fraud detection More ❯

Engineer Location: Manchester (hybrid) Type: Permanent Must have skills Working with AWS cloud services to manage and process data. Using AWS Glue for building and running data pipelines. Writing PySpark code in Python, along with SQL, to handle big data processing. Creating and running SQL queries to analyze and manage data More ❯

Engineer Location: Manchester (hybrid) Type: Permanent Must have skills Working with AWS cloud services to manage and process data. Using AWS Glue for building and running data pipelines. Writing PySpark code in Python, along with SQL, to handle big data processing. Creating and running SQL queries to analyze and manage data More ❯

source of truth. Develop and optimise CI/CD pipelines in Azure DevOps to automate deployment of workspaces, Unity Catalog, networking, and security. Work with Databricks (Spark/Scala, PySpark) to support ingestion frameworks, data processing, and platform-level libraries. Implement secure connectivity (VNET injection, Private Link, firewall, DNS, RBAC). Monitor, troubleshoot, and optimise platform reliability and performance. … processes, and standards for wider engineering adoption. Must Have: Proven expertise with Microsoft Azure (networking, security, storage, compute). Strong proficiency in Databricks with hands-on Scala (Spark) and PySpark . Deep experience with Terraform for Azure resource deployment and governance. Hands-on with Azure DevOps pipelines (YAML, agents, service connections). Understanding of Azure Active Directory/Entra More ❯

source of truth. Develop and optimise CI/CD pipelines in Azure DevOps to automate deployment of workspaces, Unity Catalog, networking, and security. Work with Databricks (Spark/Scala, PySpark) to support ingestion frameworks, data processing, and platform-level libraries. Implement secure connectivity (VNET injection, Private Link, firewall, DNS, RBAC). Monitor, troubleshoot, and optimise platform reliability and performance. … processes, and standards for wider engineering adoption. Must Have: Proven expertise with Microsoft Azure (networking, security, storage, compute). Strong proficiency in Databricks with hands-on Scala (Spark) and PySpark . Deep experience with Terraform for Azure resource deployment and governance. Hands-on with Azure DevOps pipelines (YAML, agents, service connections). Understanding of Azure Active Directory/Entra More ❯