19 of 19 PySpark Jobs in the North of England

technical architecture and detailed design Expertise in building scalable ingestion frameworks (batch/streaming) using ADF, Databricks or Fabric pipelines Strong programming skills in PySpark, SQL, Python Experience implementing medallion architecture, delta lakes, and metadata‐driven pipelines Hands‐on experience with DQ frameworks, validation rules and reconciliation logic Ability ...

technical architecture and detailed design. Expertise in building scalable ingestion frameworks (batch/streaming) using ADF, Databricks or Fabric pipelines. Strong programming skills in PySpark, SQL, Python. Experience implementing medallion architecture, delta lakes, and metadata‐driven pipelines. Hands‐on experience with DQ frameworks, validation rules and reconciliation logic. Ability ...

engagement skills, including requirements gathering, expectation management and delivery planning. Desirable Knowledge and practical experience of hospital systems and datasets (where relevant locally). PySpark experience. Experience with NLP and/or computer vision techniques (where relevant to Trust priorities). Experience using deep learning frameworks such as TensorFlow ...

data providers-ensuring data is reliable, auditable, and delivered at scale. Data Engineer Responsibilities Act as the subject matter expert across ADF, Databricks, Python, PySpark, and Spark SQL Manage the daily ingestion and processing of raw data into enterprise data platforms Collaborate with the wider team to define data ...

Experience Technical Skills Strong SQL and data modelling experience Azure data stack (ADF, Synapse, Fabric, Databricks) Experience building ETL/ELT pipelines Python/PySpark (desirable) DBT or similar transformation tools Experience with orchestration tools (ADF, Airflow, Prefect) CI/CD, Git, and Azure DevOps ...

Builder or Code Repositories. Experience with Delta Lake, Parquet, Iceberg, CSV and JSON. Data governance, lineage, secure data-sharing and access approval experience. Python, PySpark or SQL development experience. CI/CD experience using GitLab, GitHub Actions, Azure DevOps or similar. Key Deliverables Configured and validated Foundry Databricks connector. ...

someone who combines strong engineering capability with a solid understanding of the cyber domain. Key skills required: Strong Python development experience Databricks and PySpark expertise Experience with cloud-based data platforms (AWS, Azure and/or GCP) Data Engineering and analytics at scale Apache Spark ecosystems Cyber Security ...

within NHS England, UK Health Security Agency, Department of Health and Social Care, or another central government data environment (Essential) Solid experience with Python, PySpark, Power BI, and AWS Strong communication skills - able to engage confidently with both technical peers and non-technical clinical or operational stakeholders Available ...

Kubernetes Experience in creating production grade coding and SOLID programming principles, including test-driven development (TDD) approaches Experience in programming languages (e.g. Python, PySpark, R, SAS, SQL) Experience in source-control software, e.g., GitHub Proficient at communicating results in a concise manner both verbally and written Experience in data ...

TensorFlow Serving, TorchServe).* Familiarity with ML lifecycle tools such as MLflow, Kubeflow, or Azure ML Pipelines.* Experience working with distributed data processing (e.g., PySpark) and SQL.* Understanding of software engineering best practices, including version control (Git).* Knowledge of CI/CD principles in ML environments.* Experience with ...

qualification in Computer Science, AI, Data Science, or related fields. Strong knowledge of AI, machine learning, deep learning, and probabilistic modeling. Proficient in Python, PySpark, data modeling, and microservices. Experience in MLOps, AI lifecycle management, and LLM orchestration on cloud platforms (AWS, GCP, Azure). Familiar with DevOps tools ...

qualification in Computer Science, AI, Data Science, or related fields. Strong knowledge of AI, machine learning, deep learning, and probabilistic modeling. Proficient in Python, PySpark, data modeling, and microservices. Experience in MLOps, AI lifecycle management, and LLM orchestration on cloud platforms (AWS, GCP, Azure). Familiar with DevOps tools ...

Snowflake‐optimised query design and features such as materialized views Knowledge of snowpark, or transferable knowledge from other dataframe‐based frameworks such as PySpark Experience implementing batch data ingestion pipelines using technologies such as Azure Data Factory, Openflow or similar Experience delivering data solutions on Snowflake hosted in Microsoft ...

TensorFlow Serving, TorchServe). Familiarity with ML lifecycle tools such as MLflow, Kubeflow, or Azure ML Pipelines. Experience working with distributed data processing (e.g., PySpark) and SQL. Understanding of software engineering best practices, including version control (Git). Knowledge of CI/CD principles in ML environments. Experience with ...

Collaborative office environment with regular social and charity events And Much More!!! DATA ENGINEER - ESSTENTIAL SKILLS Strong experience with Databricks and Apache Spark (PySpark or Scala) Experience building streaming and batch pipelines using Spark Structured Streaming Hands-on experience with Kafka (MSK) and real-time data ingestion Strong understanding … submit (subject to required skills) your application to our client in conjunction with this vacancy only. KEY SKILLS Data Engineering, Databricks, Apache Spark, PySpark, SQL, Python, Kafka, AWS, Delta Lake, ETL, Data Pipelines, CI/CD, Big Data ...

GBMs, GAMs, Random Forests, Neural Networks and clustering techniques Knowledge of statistics and distributions commonly used in insurance Experience in programming languages (e.g. Python, PySpark, R, SAS, SQL) Proficient at communicating results in a concise manner both verbally and written Behaviours: Motivated by technical excellence Driven to deliver iterative ...

Data Architect (Databricks or Microsoft Fabric) £110,000–£125,000 Hybrid – 2 days a week in London, Manchester, or Edinburgh We’re looking for a twoexperienced Data Architects to design and deliver modern, scalable data ...

Data Engineer PySpark/Python Location: Sheffield (Hybrid 3 days per week preferred) c.£600 per day Inside IR35 Initial 6-month contract with strong likelihood of extension We are supporting a major global banking organisation on the growth of its Data Technology function and are looking … regulatory functions across a complex global environment. Key Responsibilities Design, develop and maintain scalable data pipelines and ETL processes Develop robust solutions using Python, PySpark and SQL Work with large-scale datasets in distributed data environments Contribute to data modelling and data warehousing initiatives Support CI/ ...

Senior Test Engineer Identifies and raises awareness of risks arising from automation test results. Identifies new and implements existing test frameworks to improve confidence in testing. Proficient in coding languages such as Python, Java, SQL ...