1 to 25 of 27 PySpark Jobs in the North of England

Engineer role. They aren't looking for someone who wants to move away from development into pure management. You'll work across: Python/PySpark Microsoft Fabric Spark and notebook engineering Lakehouse/Medallion architecture Delta SQL Azure Git/CI/CD Automated testing Data quality and observability … Reusable engineering patterns They're particularly interested in people coming from Databricks/PySpark environments, so extensive Fabric experience isn't essential. At Senior level, you'll also have the opportunity to influence architecture, technical design, coding standards, CI/CD, testing and engineering best practice, while supporting other ...

Data Engineer - Python/PySpark/Microsoft Fabric £55,000 - £65,000 + Benefits St Helens - 1-2 days per week This is an opportunity to join an organisation at a particularly interesting point in the development of its data capability. The business has invested heavily in modernising … procedures and drag-and-drop development. The team is deliberately moving towards a more code-led engineering approach, with a strong emphasis on Python, PySpark and notebook-based development. What you'll be working with The environment includes: Python/PySpark Microsoft Fabric Spark and notebook development Lakehouse ...

bonus and an extensive benefits package. What you'll be doing Design, build and maintain scalable data pipelines and data products using Databricks , PySpark, Spark SQL and Delta Lake. Develop and optimise ETL/ELT processes to ingest, transform and curate data from internal and external systems. Shape enterprise … help mature a growing data function. What you need to bring “Senior” level experience in data engineering, with strong hands-on Databricks skills (PySpark, Spark SQL, Delta Lake, etc) Advanced SQL and Python coding skills. A track record of building and optimising enterprise-scale data pipelines and ETL/ ...

Kubernetes Experience in creating production grade coding and SOLID programming principles, including test-driven development (TDD) approaches Experience in programming languages (e.g. Python, PySpark, R, SAS, SQL) Experience in source-control software, e.g., GitHub Proficient at communicating results in a concise manner both verbally and written Experience in data ...

champion best-practice methods; upskill the team through coaching and code reviews. Query and engineer large datasets (e.g., Python/R/SQL/PySpark) on modern platforms (e.g., Azure Databricks). Communicate pricing proposals and insights clearly and persuasively to stakeholders at all levels. Collaborate with Modelling, Optimisation … problem solving capabilities with the ability to apply market standard pricing techniques to non-standard products. Proficiency in programming languages such as R, Python, PySpark, SAS, or SQL. Good knowledge of Azure Databricks and modern data technologies. Technical expertise in predictive modelling techniques e.g. GLMs. Experience with WTW Radar ...

knowledge of Matillion’s Data Productivity Cloud/Matillion ETL, or primary competitive data integration platforms. Big Data & Analytics Ecosystems: Experience with Spark/PySpark, modern Data Lakes/Lakehouses, and MPP database architectures (Teradata, Vertica, Netezza). Education: Bachelor’s or Master’s degree in Computer Science, Data ...

OneLake, pipelines and notebooks, and how these can be used to build scalable enterprise data solutions. Strong experience with SQL, Python and/or PySpark and modern data engineering approaches. Strong understanding of data architecture and data modelling, including designing reusable and scalable data structures. Experience implementing modern software ...

contract. Contract Details: Inside IR35 Initial 6-month engagement Sheffield-based, 3 days per week on-site Key Requirements: Strong hands-on experience with PySpark and Databricks Deep understanding of Spark performance optimisation, including lazy evaluation, column pruning, groupBy operations, and window functions Proven ability to troubleshoot and resolve ...

hands-on data engineering role within a large-scale cloud data programme, responsible for building, maintaining, and troubleshooting data pipelines using Apache Spark, PySpark, Apache Airflow, and a broad suite of AWS services. You will apply strong analytical and engineering skills to deliver trusted, well-governed data assets … data programme - designing, building, and maintaining data pipelines that process large volumes of data across a modern AWS-native stack. Using Apache Spark and PySpark for distributed data processing, Apache Airflow for orchestration, and a range of AWS services for storage, compute, and analytics, you will help deliver reliable ...

actually trust and use. Essential Skills & Experience: Hands-on experience with Databricks and Microsoft Fabric Experience building and running production-grade pipelines Strong PySpark and SQL skills Experience with Delta Lake and Lakehouse architectures Good grasp of data modelling, orchestration and workflow automation Comfortable working with structured and semi ...

analytics strategy AI & Automation Agentic AI patterns and intelligent automation solutions Document intelligence and AI-driven business use cases SQL Python and/or PySpark Data pipelines and transformation frameworks CI/CD concepts and deployment automation Azure DevOps and/or GitHub Infrastructure as Code awareness Environment promotion ...

delivery. What You'll Be Doing Design, develop and maintain data pipelines and data engineering solutions using Databrick sBuild scalable data processing solutions using PySpark and SQL Work across Databricks Lakehouse environments, supporting implementation, optimisation and ongoing improvement Develop and optimise batch and data processing workflows Contribute to technical … transformation journeys What We're Looking For Strong commercial experience in Data Engineering Hands-on experience with Databricks and/or Apache Spark Strong PySpark and SQL development skills Experience designing, building and supporting data pipelines Experience working with Azure or AWS Good understanding of modern data platforms ...

Data Engineer Key skills: * Python * SQL Server/SQL * Data Warehousing & Data Modelling * Data Architecture * Data Pipelines & Integration Solutions * Microsoft Fabric * Azure Cloud Technologies * PySpark * ETL/ELT Development * Data Governance & Security * Source Control (Git) * Technical Leadership & Team Management You will be a Lead Data Engineer Developer with good ...

modern data platform. Develop and maintain Medallion Architecture data layers, ensuring data is available for reporting and analytics. Create high-performance data transformations using PySpark, Databricks and Fabric Notebooks. Implement data governance and access controls through Databricks Unity Catalog. Support CI/CD deployment processes using Git integration … within a Data Engineering environment, delivering reliable and scalable data solutions. Strong hands-on experience with Microsoft Fabric and Databricks. Strong programming capability across PySpark and T-SQL. Experience working with Delta Lake, Lakehouse architectures and modern cloud data platforms. Understanding of data quality frameworks, testing approaches and governance ...

/ETL workflows within a Fabric Lakehouse environment Work with OneLake, Lakehouse architecture, and modern data engineering best practices Utilise PySpark, SQL and Databricks for complex data transformation and engineering workloads Create robust data ingestion, orchestration and monitoring processes Support the development of enterprise-wide analytics solutions and Power … with Microsoft Fabric, including Data Factory, Dataflows, Notebooks and Lakehouse solutions Experience building and supporting production-grade ELT/ETL pipelines Strong skills in PySpark and SQL Experience working with OneLake, Delta formats or modern Lakehouse architectures Exposure to Databricks and distributed data processing environments Understanding of data modelling ...

Engineer (AWS) - Manchester (Data Engineer, Senior Data Engineer, ETL, AWS, AWS Glue, AWS Lambda, AWS S3, AWS Redshift, Amazon EMR, Python, SQL, Pipeline Orchestration, PySpark, data warehousing, data lakehouse, data architecture, Agile, Senior Data Engineer, Data Engineer) Our client is a global innovator and world leader with … with AWS services such as AWS Glue, AWS Lambda, AWS S3, AWS Redshift, and Amazon EMR. Proven expertise in Python, SQL, Pipeline Orchestration and PySpark is also expected, as is experience with data lakehouse architectures and data warehousing solutions. Agile experience is also key, as are excellent communication skills ...

visible role where you'll combine strong technical skills with commercial thinking to drive meaningful product improvements. Your Skills & Experience Strong SQL skills Python (PySpark experience is beneficial) Experience in Product Analytics Strong knowledge of A/B testing and experimentation Excellent statistical analysis skills, with exposure to modelling ...

visible role where you'll combine strong technical skills with commercial thinking to drive meaningful product improvements. Your Skills & Experience Strong SQL skills Python (PySpark experience is beneficial) Experience in Product Analytics Strong knowledge of A/B testing and experimentation Excellent statistical analysis skills, with exposure to modelling ...

functional teams to deliver high-quality technical solutions. Support the ongoing maintenance and optimisation of platform services. Essential Skills & Experience Hands-on experience with PySpark for data transformation and processing. Experience working with AWS data services , including: AWS Glue Amazon S3 Experience using Terraform for Infrastructure as Code. Experience ...

solutions Requirements: Enterprise data warehouse architecture and design Microsoft Fabric, Azure Data Platform technologies and modern Lakehouse architectures Databricks, Unity Catalog and Delta Lake PySpark, Databricks, SQL Medallion Architecture implementation Data modelling approaches including Star Schema and Data Vault Data governance, security and access management Why This Opportunity Greenfield ...

experience with Axiom (plus) and regulatory reporting flows (plus) Excellent communication; proven stakeholder engagement Ability to learn/adapt to new technologies Skilled PySpark, HDFS, SQL, Airflow, data modeling, and ETL processing. Familiarity with GitLab and Jupyter Notebook is preferred, and knowledge of Java will be considered an added ...

performance optimisation. Hands-on delivery experience within hospitality reporting and commercial reporting environments. Experience working with Databricks is useful. Ability to interpret Python/PySpark is beneficial. If you are interested in this Power BI Developer role please apply with your most recent CV or reach out on jordan ...

Data Engineer - Manchester (Hybrid) - Microsoft Fabric - Databricks - PySpark - Up to £55,000 We're partnering with a highly successful, private equity-backed organisation that is investing heavily in its data and technology capabilities. With a business-wide data transformation programme underway, this is a rare opportunity to join … Designing and building scalable ETL/ELT data pipelines. Microsoft Fabric, including Data Factory, Dataflow Gen2 and Fabric Notebooks. Databricks Workflows and Lakehouse technologies. PySpark, SQL and Delta Lake environments. Data ingestion, transformation and orchestration processes. Data governance, security and quality initiatives. Enterprise-wide analytics and reporting solutions, including ...

Private equity-backed organisation with ambitious growth plans Essential Skills & Experience Strong experience as a Data Engineer within modern cloud data environments Expertise in PySpark, SQL and data transformation practices Experience building production-grade data pipelines Knowledge of Microsoft Fabric Experience working with Delta Lake and Lakehouse architectures ...

GBMs, GAMs, Random Forests, Neural Networks and clustering techniques Knowledge of statistics and distributions commonly used in insurance Experience in programming languages (e.g. Python, PySpark, R, SAS, SQL) Proficient at communicating results in a concise manner both verbally and written Behaviours: Motivated by technical excellence Driven to deliver iterative ...