18 of 18 Permanent PySpark Jobs in the North of England

technical architecture and detailed design Expertise in building scalable ingestion frameworks (batch/streaming) using ADF, Databricks or Fabric pipelines Strong programming skills in PySpark, SQL, Python Experience implementing medallion architecture, delta lakes, and metadata‐driven pipelines Hands‐on experience with DQ frameworks, validation rules and reconciliation logic Ability ...

technical architecture and detailed design. Expertise in building scalable ingestion frameworks (batch/streaming) using ADF, Databricks or Fabric pipelines. Strong programming skills in PySpark, SQL, Python. Experience implementing medallion architecture, delta lakes, and metadata‐driven pipelines. Hands‐on experience with DQ frameworks, validation rules and reconciliation logic. Ability ...

backgrounds. The technology stack is flexible, but experience in some of the following are nice to have: Data Engineering & Programming: • Python • SQL • Spark/PySpark • Scala • Data modelling and warehousing concepts Cloud & Platforms: • AWS (S3, Redshift, Glue, EMR, Lambda) • Other cloud platforms (Azure/GCP considered) • Data lake ...

data platform What You Bring Strong experience with AWS data services (Glue, Lambda, S3, Redshift, EMR) Solid programming skills in Python, SQL, and PySpark Experience designing and optimising data pipelines and ETL/ELT processes Good understanding of data warehousing and modern data architectures Ability to diagnose and solve ...

data providers-ensuring data is reliable, auditable, and delivered at scale. Data Engineer Responsibilities Act as the subject matter expert across ADF, Databricks, Python, PySpark, and Spark SQL Manage the daily ingestion and processing of raw data into enterprise data platforms Collaborate with the wider team to define data ...

genuinely influence engineering direction within a scaling business What they’re looking for 👇 😍 Strong hands-on Data Engineering experience across Python, Spark, SQL and PySpark 😍 Experience working with hyperscaler cloud platforms such as AWS, Azure or GCP 😍 Exposure to modern tooling including Databricks, Microsoft Fabric and PowerBI 😍 Previous experience ...

making Your Skills and Experience Strong commercial experience as a data engineer in modern, cloud based environments Advanced Python and SQL, with experience using PySpark or similar distributed processing frameworks Experience building and maintaining data pipelines in AWS using services such as S3, Glue or EKS Hands on experience ...

Kubernetes Experience in creating production grade coding and SOLID programming principles, including test-driven development (TDD) approaches Experience in programming languages (e.g. Python, PySpark, R, SAS, SQL) Experience in source-control software, e.g., GitHub Proficient at communicating results in a concise manner both verbally and written Experience in data ...

TensorFlow Serving, TorchServe).* Familiarity with ML lifecycle tools such as MLflow, Kubeflow, or Azure ML Pipelines.* Experience working with distributed data processing (e.g., PySpark) and SQL.* Understanding of software engineering best practices, including version control (Git).* Knowledge of CI/CD principles in ML environments.* Experience with ...

qualification in Computer Science, AI, Data Science, or related fields. Strong knowledge of AI, machine learning, deep learning, and probabilistic modeling. Proficient in Python, PySpark, data modeling, and microservices. Experience in MLOps, AI lifecycle management, and LLM orchestration on cloud platforms (AWS, GCP, Azure). Familiar with DevOps tools ...

qualification in Computer Science, AI, Data Science, or related fields. Strong knowledge of AI, machine learning, deep learning, and probabilistic modeling. Proficient in Python, PySpark, data modeling, and microservices. Experience in MLOps, AI lifecycle management, and LLM orchestration on cloud platforms (AWS, GCP, Azure). Familiar with DevOps tools ...

Data Engineer; Python, PySpark - Salford, Onsite. Be part of a retail phenomenon! This is a brand new role at a company experiencing huge growth, offering a lot of autonomy and the opportunity to grow as the company grows. My client is seeking an experienced Data Engineer to join ...

Snowflake‐optimised query design and features such as materialized views Knowledge of snowpark, or transferable knowledge from other dataframe‐based frameworks such as PySpark Experience implementing batch data ingestion pipelines using technologies such as Azure Data Factory, Openflow or similar Experience delivering data solutions on Snowflake hosted in Microsoft ...

TensorFlow Serving, TorchServe). Familiarity with ML lifecycle tools such as MLflow, Kubeflow, or Azure ML Pipelines. Experience working with distributed data processing (e.g., PySpark) and SQL. Understanding of software engineering best practices, including version control (Git). Knowledge of CI/CD principles in ML environments. Experience with ...

Collaborative office environment with regular social and charity events And Much More!!! DATA ENGINEER - ESSTENTIAL SKILLS Strong experience with Databricks and Apache Spark (PySpark or Scala) Experience building streaming and batch pipelines using Spark Structured Streaming Hands-on experience with Kafka (MSK) and real-time data ingestion Strong understanding … submit (subject to required skills) your application to our client in conjunction with this vacancy only. KEY SKILLS Data Engineering, Databricks, Apache Spark, PySpark, SQL, Python, Kafka, AWS, Delta Lake, ETL, Data Pipelines, CI/CD, Big Data ...

GBMs, GAMs, Random Forests, Neural Networks and clustering techniques Knowledge of statistics and distributions commonly used in insurance Experience in programming languages (e.g. Python, PySpark, R, SAS, SQL) Proficient at communicating results in a concise manner both verbally and written Behaviours: Motivated by technical excellence Driven to deliver iterative ...

Data Architect (Databricks or Microsoft Fabric) £110,000–£125,000 Hybrid – 2 days a week in London, Manchester, or Edinburgh We’re looking for a twoexperienced Data Architects to design and deliver modern, scalable data ...

Senior Test Engineer Identifies and raises awareness of risks arising from automation test results. Identifies new and implements existing test frameworks to improve confidence in testing. Proficient in coding languages such as Python, Java, SQL ...