22 of 22 PySpark Jobs in the North of England

ideally in enterprise-scale environments. Deep expertise in Azure Databricks, Azure Data Factory, Azure Synapse and Azure Data Lake. Hands-on experience with Spark (PySpark/SparkSQL), ETL/ELT design and data pipeline orchestration. Strong data modelling skills (Dimensional, ODS, Data Vault) and experience with data warehousing concepts. ...

scale data platforms,lakehousesand data warehouses. Data modelling and dimensional modelling techniques such as Kimball. ETL/ELT development and large-scale data integration. PySpark, Spark SQL and modern transformation techniques. Advanced SQL and relational databases including SQL Server, PostgreSQL and Oracle. Integrating structured, semi-structured and unstructured data ...

assisted software development tools. Exposure to agentic AI, workflow orchestration or retrieval-augmented generation. Key responsibilities Design and develop scalable data pipelines using Python, PySpark, Apache Kafka and Apache NiFi. Integrate vendor platforms and enterprise systems through REST APIs and secure authentication mechanisms. Build data processing and storage solutions ...

such as Claude Code or equivalent tools, to improve engineering productivity while maintaining high standards of quality, governance and security. Desirable experience Regulated environment. PySpark for large-scale data transformations. Migrating legacy data estates to lakehouse or metadata-driven architectures. Customer Data Platforms or event-driven systems. Data-quality ...

accelerate development across code, infrastructure, testing and documentation while maintaining appropriate quality and security controls. Desirable experience Financial services or another regulated environment. PySpark for large-scale batch or streaming transformations. Migrating SQL Server and SSIS estates to lakehouse or metadata-driven architectures. Customer Data Platforms or event-driven ...

platform or data warehouse migration (cutover, reconciliation, parallel running, decommissioning) Deep, hands-on AWS data experience: Glue, Lambda, S3, Redshift and EMR Advanced Python, PySpark and SQL Production experience with Kafka or Amazon MSK, including Spark Structured Streaming Hands-on experience with Apache Iceberg or Delta Lake Strong data ...

platform or data warehouse migration (cutover, reconciliation, parallel running, decommissioning) Deep, hands-on AWS data experience: Glue, Lambda, S3, Redshift and EMR Advanced Python, PySpark and SQL Production experience with Kafka or Amazon MSK, including Spark Structured Streaming Hands-on experience with Apache Iceberg or Delta Lake Strong data ...

Data Engineer Key skills: * Microsoft Fabric * Azure Cloud Technologies * Python * SQL Server/SQL * Data Warehousing & Data Modelling * Data Architecture * Data Pipelines & Integration Solutions * PySpark * ETL/ELT Development * Data Governance & Security * Source Control (Git) * Technical Leadership & Team Management You will be a Lead Data Engineer Developer with good ...

making. Key Responsibilities: Lead the design and development of curated, analytics-ready dimensional and semantic data models. Build, maintain and optimise complex SQL and PySpark transformation pipelines within Databricks. Define and maintain KPI definitions, conformed dimensions and consistent business logic across data domains. Ensure data quality, integrity, testing, observability ...

best practices Skills & Experience: Understanding of data processing using Apache Spark and Apache Airflow for task orchestration Use of Python, SQL, and familiarity with PySpark Understanding of EMR and reviewing output logs Use of Jupyter notebooks and/or Amazon Athena to query and validate data Data analysis ...

Platforms: Google Cloud Platform, Amazon Web Services, Microsoft Azure, and Databricks. Experience in one or more of the listed Languages such as: Python, R, PySpark Experience in one or more of the following Agentic & Gen AI Platforms: such as Microsoft Copilot Studio, Adept AI, UiPath, OpenAI GPT-5 Agents ...

Machine Learning – Understanding of AI/ML concepts, experience with LLMs, prompt engineering, or integrating AI capabilities into applications • Data Technologies – Experience with Kafka, PySpark, Parquet and Avro data formats, Apache Airflow for workflow management, or Streamlit for building interactive data applications • Advanced Data Management – Strong SQL design, query ...

providing a longer-term opportunity and potential for permanent employment. What youll be doing Building and maintaining data pipelines within Azure, using Databricks, PySpark and Azure Data Factory. Developing SQL and Spark transformations to ingest, cleanse and prepare data from multiple sources. Applying data modelling and lakehouse principles … right technical foundations, a practical mindset and the motivation to learn. Whats on offer? 12 month contract The opportunity to work with Azure Databricks, PySpark and modern data platform technologies. Exposure to a significant transformation programme and a broad range of business data. The chance to learn from experienced ...

performance optimisation. Hands-on delivery experience within hospitality reporting and commercial reporting environments. Experience working with Databricks is useful. Ability to interpret Python/PySpark is beneficial. If you are interested in this Power BI Developer role please apply with your most recent CV or reach out on jordan ...

bonus and an extensive benefits package. What you'll be doing Design, build and maintain scalable data pipelines and data products using Databricks , PySpark, Spark SQL and Delta Lake. Develop and optimise ETL/ELT processes to ingest, transform and curate data from internal and external systems. Shape enterprise … help mature a growing data function. What you need to bring “Senior” level experience in data engineering, with strong hands-on Databricks skills (PySpark, Spark SQL, Delta Lake, etc) Advanced SQL and Python coding skills. A track record of building and optimising enterprise-scale data pipelines and ETL/ ...

Azure Data Engineer - 1/2 days onsite 18 month FTC Summary: Join a team building a modern Azure-based data platform. This hands-on engineering role involves designing and developing scalable, automated data pipelines ...

Data Engineer (Databricks) Leeds (Hybrid) 6 Month Contract £(Apply online only)/day (Inside IR35) Data Engineer needed with strong hands-on Databricks, PySpark, AWS, SQL + Python experience. 6 Month Contract based in Leeds (Hybrid). Start ASAP in October 2026. Hybrid Working Model - primarily working from home … month. A chance to work with a leading global IT transformation on a large-scale Government project: Data Engineer with strong Databricks, AWS, Python, PySpark + SQL skills. Hands-on Data Engineer to run + maintain production data pipelines on Databricks and AWS. Keeping scheduled data processing reliable, resolve ...

2026. A chance to work with a leading global IT transformation on a large-scale Government project: Data Engineer with strong Databricks, AWS, Python, PySpark and SQL skills. Hands-on Data Engineer to run + maintain production data pipelines on Databricks and AWS. Keeping scheduled data processing reliable, resolve … understanding of role-based access, KMS encryption + diagnosing data access failures. Owning incidents from investigation through recovery and closure. Diagnosing issues in Python, PySpark, SQL, Databricks jobs and AWS integrations + providing clear progress updates and timely escalation. Working knowledge of Git, code reviews, CI/ ...

personal technical development. To ensure data dictionaries and models are robust and maintained on a regular basis. Technical Skill Expert-level Databricks experience Advanced PySpark development Delta Lake architecture and optimisation Databricks SQL Data warehouse Dimensional modelling Data Lake Databricks Genie Python development Key Skills, Experience and Knowledge Extensive … within enterprise-scale data environments. Extensive hands-on experience delivering solutions using Databricks. Extensive experience designing and implementing Lakehouse architectures. Strong commercial experience using PySpark, Python, SQL and Delta Lake. Experience optimising Databricks workloads for performance, scalability and cost efficiency Previous experience delivering solutions within insurance, financial services ...

Strong knowledge of migrating and transforming data from legacy and on-premise systems to cloud platforms. Expertise in automating ETL/ELT pipelines using PySpark and Spark SQL across Azure-based data environments. You and your role As a Data Engineer, you will design build and enhance scalable …/ELT pipelines using Python and PySpark, ensuring data solutions are reliable, efficient and aligned with engineering best practice. You'll work closely with stakeholders to understand complex business requirements, transforming data into trusted, high-quality datasets that enable valuable insights, reporting and decision-making. You will manage ...

Strong knowledge of migrating and transforming data from legacy and on-premise systems to cloud platforms. Expertise in automating ETL/ELT pipelines using PySpark and Spark SQL across Azure-based data environments. You and your role As a Data Engineer, you will design build and enhance scalable …/ELT pipelines using Python and PySpark, ensuring data solutions are reliable, efficient and aligned with engineering best practice. You'll work closely with stakeholders to understand complex business requirements, transforming data into trusted, high-quality datasets that enable valuable insights, reporting and decision-making. You will manage ...

Strong knowledge of migrating and transforming data from legacy and on-premise systems to cloud platforms. Expertise in automating ETL/ELT pipelines using PySpark and Spark SQL across Azure-based data environments. You and your role As a Data Engineer, you will design build and enhance scalable …/ELT pipelines using Python and PySpark, ensuring data solutions are reliable, efficient and aligned with engineering best practice. You'll work closely with stakeholders to understand complex business requirements, transforming data into trusted, high-quality datasets that enable valuable insights, reporting and decision-making. You will manage ...