1 to 25 of 36 PySpark Jobs in the North of England

data engineering team while partnering closely with product, BI, data science, and business stakeholders Experience required: Strong experience with Microsoft Fabric, Delta Lake, PySpark, Python, SQL, and modern ETL/ELT frameworks, alongside CI/CD practices using Azure DevOps or GitHub Hands-on expertise integrating Azure AI/ ...

Warehouses, and real-time analytics solutions in Fabric Strong SQL/T-SQL capability, including optimisation and development of reusable objects. Proficiency in Python (PySpark, Spark SQL beneficial). Experience working with structured/unstructured data, delta lake patterns, partitioning, and scalable pipeline design. Strong understanding of data modelling ...

tooling Key skills Strong experience with Microsoft Fabric (Data Factory, Lakehouse, OneLake, Pipelines, SQL Warehouse) Solid data engineering background using SQL, Python, PySpark, Delta Lake E xperience building and optimising end-to-end data pipelines CI/CD experience for analytics and data platforms (Azure DevOps or GitHub) Exposure ...

technical architecture and detailed design. •Expertise in building scalable ingestion frameworks (batch/streaming) using ADF, Databricks or Fabric pipelines. •Strong programming skills in PySpark, SQL, Python. •Experience implementing medallion architecture, delta lakes, and metadata‐driven pipelines. •Hands-on experience with DQ frameworks, validation rules and reconciliation logic. •Ability ...

backgrounds. The technology stack is flexible, but experience in some of the following are nice to have: Data Engineering & Programming: • Python • SQL • Spark/PySpark • Scala • Data modelling and warehousing concepts Cloud & Platforms: • AWS (S3, Redshift, Glue, EMR, Lambda) • Other cloud platforms (Azure/GCP considered) • Data lake ...

modern data platform development. Deep hands-on experience with Microsoft Fabric, including Lakehouse, Data Warehouse, OneLake, Pipelines and Data Factory. Strong SQL, Spark/PySpark and data modelling capability across analytical and operational datasets. Proven experience delivering complex end-to-end data platforms in Azure or Fabric environments. Strong ...

code, with a genuine passion for software engineering in a data context. You'll work across the modern Azure data stack - ADF, ADLS Gen2, PySpark, Delta Lake - with increasing exposure to Microsoft Fabric as the platform matures. You'll collaborate closely with customers and internal teams to ensure data … architectural decisions, and help mentor junior engineers as the team scales. Responsibilities Architect and write production-grade ELT/ETL data pipelines using PySpark and Python within Azure ecosystem. Build custom, reusable data processing frameworks and libraries in Python/Scala to streamline ingestion and transformation tasks across ...

Strong experience designing and delivering data solutions in Databricks Proficient with SQL and Python Experience using Big Data technologies such as Apache Spark or PySpark Great communication skills, effectively participating with Senior Stakeholders Nice to haves: Azure/AWS Data Engineering certifications Databricks certifications What ...

business needs Ensure data quality, governance and best practice Key Requirements Strong leadership experience (5+ engineers) Solid Data Engineering/DataOps background Python/PySpark for data pipelines AWS experience (design & deployment) Strong data modelling & warehousing knowledge Agile (Jira/Azure DevOps) experience Get in touch for more details ...

Customer Experience, Finance and Operations can trust. Responsibilities Build, optimise and operate ELT/ETL pipelines into our data platform using SQL and Python (PySpark), with a focus on reliability, performance and maintainability. Develop and maintain core data models (curated layers, dimensional models, shared definitions) that enable consistent ...

sizing Fabric initiatives, including capacity needs, effort estimates, infrastructure requirements, team resourcing, and delivery timelines. Extensive knowledge in data modelling, DAX, Python/PySpark, SQL/KQL. Proven record of technical leadership and effective stakeholder engagement. Lead architectural design, capacity planning, and performance optimisation efforts. Implement governance, security ...

data projects Key Requirements Proven leadership of multiple engineering teams Strong Data Engineering/DataOps background Experience in large, dataheavy environments Python/PySpark for endto-end pipelines Strong AWS experience (design & deployment) Deep knowledge of data modelling, warehousing & lakehouse concepts Strong stakeholder management and communication skills Agile delivery ...

Collaborate with your peers on technical direction within your team Required Skills and Experience Strong analytic skills related to working with unstructured datasets Python (PySpark, Pandas, PyArrow) Distributed data processing (Apache Spark) Data ETL (Apache Airflow, AWS Step Functions, Apache NiFi) Cloud services (AWS, Azure or GCP) Messaging/ ...

production. Key skills and experience: Strong experience as a Data Engineer Advanced Python and SQL skills Experience with cloud platforms (AWS preferred) Proficiency in PySpark and/or Pandas Familiarity with data quality tools (e.g., Great Expectations) Experience with infrastructure as code (Terraform or CloudFormation) Understanding of modern development ...

will have the following skills and experience: Deep expertise with Azure Databricks, including development, deployment, and platform-level engineering. Strong software engineering capability across Pyspark, Spark SQL and Azure Data Factory. Solid understanding of DevOps practices including CI/CD and automated testing. Experience with monitoring, logging, and alerting ...

will have the following skills and experience: Deep expertise with Azure Databricks, including development, deployment, and platform-level engineering. Strong software engineering capability across Pyspark, Spark SQL and Azure Data Factory. Solid understanding of DevOps practices including CI/CD and automated testing. Experience with monitoring, logging, and alerting ...

notebooks) Experience building or working within modern data warehouse architectures Ability to get up and running quickly without heavy onboarding Nice to have: Python, PySpark or Spark experience Exposure to APIs and external data integration Experience working in evolving or transforming data environments Most importantly, youll be someone ...

notebooks) Experience building or working within modern data warehouse architectures Ability to get up and running quickly without heavy onboarding Nice to have: Python, PySpark or Spark experience Exposure to APIs and external data integration Experience working in evolving or transforming data environments Most importantly, youll be someone ...

teams. What we're looking for: Strong people leadership experience (5+ team size) Solid Data Engineering/DataOps background Experience with AWS, Python/PySpark, ETL/ELT pipelines Strong understanding of data platforms (Data Lake/Lakehouse/EDW) Hybrid working (3 days in office) + strong bonus ...

Data Engineer; Python, PySpark - Salford, Manchester, Onsite. We are seeking an experienced Data Engineer to join a growing Data Team within a fast-paced, e-commerce-led business that is growing at an astonishing rate. The company operates in the direct-to-consumer retail space, and the role will ...

Kubernetes Experience in creating production grade coding and SOLID programming principles, including test-driven development (TDD) approaches Experience in programming languages (e.g. Python, PySpark, R, SAS, SQL) Experience in source-control software, e.g., GitHub Proficient at communicating results in a concise manner both verbally and written Experience in data ...

dashboards, DAX, M queries and semantic modelling. Build robust, optimised SQL Server data models and queries. Work within a Databricks environment (SQL/PySpark exposure is beneficial; training provided if needed). Engage directly with business stakeholders to gather requirements, understand challenges and translate them into technical solutions. Support ...

data for critical client projects. How to develop and enhance your knowledge of agile ways of working and working in open source stack (PySpark/PySql). Quality engineering professionals utilise Accenture delivery assets to plan and implement quality initiatives to ensure solution quality throughout delivery. As a Data ...

Collaborative office environment with regular social and charity events And Much More!!! DATA ENGINEER – ESSTENTIAL SKILLS Strong experience with Databricks and Apache Spark (PySpark or Scala) Experience building streaming and batch pipelines using Spark Structured Streaming Hands-on experience with Kafka (MSK) and real-time data ingestion Strong understanding … submit (subject to required skills) your application to our client in conjunction with this vacancy only. KEY SKILLS Data Engineering, Databricks, Apache Spark, PySpark, SQL, Python, Kafka, AWS, Delta Lake, ETL, Data Pipelines, CI/CD, Big Data ...

GBMs, GAMs, Random Forests, Neural Networks and clustering techniques Knowledge of statistics and distributions commonly used in insurance Experience in programming languages (e.g. Python, PySpark, R, SAS, SQL) Proficient at communicating results in a concise manner both verbally and written Behaviours: Motivated by technical excellence Driven to deliver iterative ...