1 to 25 of 33 Contract PySpark Jobs in the UK

security measures across the platform. Leverage Terraform or similar IaC tools for controlled and reproducible deployments. Databricks Development Develop and optimise data jobs using PySpark or Scala within Databricks. Implement the medallion architecture (bronze, silver, gold layers) and use Delta Lake for reliable data transactions. Manage cluster configurations ...

Essential Strong experience with Azure Databricks Infrastructure as Code using Terraform DevOps practices and CI/CD pipelines Git and peer review workflows Python, PySpark, and SQL Cloud operations within Azure To apply for this role please submit your CV or contact Dillon Blackburn on or at . Tenth ...

design and ETL logic. DevOps: Hands-on experience with Azure DevOps , CI/CD pipelines, and Key Vault. Bonus: Experience with Azure Databricks or PySpark is highly desirable. Randstad Technologies is acting as an Employment Business in relation to this vacancy. ...

integration. Proficiency with Databricks for data processing and analytics. Working knowledge of NodeJS would be an added advantage. Solid programming skills in Python , PySpark, Spark . Candidate must be adept at working with large-scale datasets , S3 , Python , and data cataloguing tools . Familiarity with data engineering best practices ...

CDF. High technical proficiency with Cosmos DB SDKs (e.g., .NET, Java, Python, Node.js) Thorough understanding of Spark distributed computing concepts. High technical proficiency in PySpark and Spark Concepts. Experience with concurrency patterns, CLR, and scalable application design. Deep understanding of Azure services including Azure Functions, App Services ...

CDF. High technical proficiency with Cosmos DB SDKs (e.g., .NET, Java, Python, Node.js) Thorough understanding of Spark distributed computing concepts. High technical proficiency in PySpark and Spark Concepts. Experience with concurrency patterns, CLR, and scalable application design. Deep understanding of Azure services including Azure Functions, App Services ...

Maintain documentation, lineage, governance standards. Required Skills/Experience The ideal candidate will have the following: Strong SQL; Snowflake ELT; dbt experience. Python/PySpark, ETL/ELT design. Data Vault 2.0 or dimensional modeling. AWS services (S3, Glue, Lambda, Redshift) or GCP equivalents. Experience with CI/ ...

Lead Data Engineer - Pyspark/AWS/Python/SAS - Financial Sector As a Lead PySpark Engineer, you will design, develop, and fix complex data processing solutions using PySpark on AWS. You will work hands-on with code, modernising legacy data workflows and supporting large-scale … PySpark migrations. The role requires strong engineering discipline, deep data understanding, and the ability to deliver production-ready data pipelines in a financial services environment. Essential Skills PySpark & Data Engineering Minimum 5+ years of hands-on PySpark experience. SAS to Pyspark migration experience Proven ability ...

PyCharm, VS Code, JupyterLab, Eclipse, Notepad++, Sagemaker Studio, Azure ML Studio, Vertex AI Workbench Python Libraries NumPy, Pandas, Matplotlib, Scikit-learn, TensorFlow, Keras, PyTorch, PySpark, SpaCy, SciPy, NLTK, Statsmodels, Boto3, AzureSDK NLP & LLMs BERT, Word2Vec, Universal Sentence Encoder, NLTK, embeddings, fuzzy matching, topic modelling LLM experience: GPT-3.5/ ...

Experience Strong hands-on experience with Azure Cosmos DB Query optimisation and throughput management expertise Experience with Cosmos DB Change Data Feed (CDF) Strong PySpark and Spark distributed computing knowledge Experience integrating Spark with Cosmos CDF High proficiency with Cosmos DB SDKs (.NET, Java, Python, Node.js) Deep understanding ...

Build and operate scalable data pipelines using Microsoft Fabric (OneLake/Delta Lake, Data Factory, Synapse Data Engineering). Develop batch processing solutions using PySpark, Spark SQL, Python, and SQL , with a focus on performance, resilience, and data quality. Support reporting and MI use cases, including transformations and data … Data Engineer or equivalent. Active SC clearance (BPSS minimum). Strong hands-on experience with Microsoft Fabric and Azure data services. Advanced skills in PySpark, Spark SQL, Python, and SQL . Experience delivering data engineering solutions in government or similarly regulated environments. CI/CD, DevOps, and IaC experience ...

basis, clarifying ownership and integration points, and capturing this information in a way that supports risk assessment and decommissioning decisions. Responsibilities: Python and PySpark are required as supporting capabilities, used where needed to analyse data pipelines and confirm how data moves and transforms in practice. The role also requires ...

building and operating machine learning models. Experience in multiple technologies and frameworks required such as Azure Databricks, Azure ML, MLFlow, GIT, Python and PySpark and microservices. Strong understanding of software development, DevOps, MLOps and Agentic Ops practices and microservices. Excellent communication and collaboration skills. About ...

leadership role. Hands-on expertise in the development of a data Lakehouse within the Microsoft Fabric or Databricks data platforms. Significant experience with PySpark and Spark-based data processing. Deep understanding of data warehouse design principles, Lakehouse architectures, including medallion (Bronze/Silver/Gold) patterns. Experience integrating data ...

leadership role. Hands-on expertise in the development of a data Lakehouse within the Microsoft Fabric or Databricks data platforms. Significant experience with PySpark and Spark-based data processing. Deep understanding of data warehouse design principles, Lakehouse architectures, including medallion (Bronze/Silver/Gold) patterns. Experience integrating data ...

heavily in the public sector - we're looking for Azure Data Engineers for some immediate contract roles. Key skills: Azure Data Engineering Fabric and PySpark Databricks Data Pipelines Medallion architecture Data warehouses Given the urgency of the roles and projects, we do need everyone to be available either immediately ...

across Bronze, Silver, Staging and Gold layers, operating within an established architectural framework. Key Responsibilities Develop and maintain data pipelines in Azure Databricks using PySpark and SQL Build incremental processing using Delta Lake and Change Data Feed Extend and maintain watermark-driven CDC frameworks Implement and maintain Slowly Changing ...

Looker and enabling reporting for end-users within general guardrails Your Profile Essential Skills/Knowledge/Experience Python GCP Services GCS, GKE, BigQuery PySpark, DBT IAM Role & Groups CI/CD Tools Jenkins, Harness or equivalent Design & Architecture Data product design with in guard rails and estimating FinOps ...

reporting, and downstream data services across the organisation. Key Responsibilities Design, build, and maintain data pipelines using Databricks Develop ETL/ELT processes with PySpark and Spark SQL Transform and model structured and semi-structured datasets Improve performance, reliability, and cost efficiency of data workloads Ensure compliance with data … Skills & Experience Active SC Clearance - mandatory and non-negotiable Strong hands-on experience with Databricks in production environments Expert knowledge of Apache Spark , including PySpark and Spark SQL Proficiency with Python for data engineering Experience delivering solutions on cloud data platforms ( AWS preferred , Azure acceptable) Understanding of data lake ...

Lead PySpark Engineer (Cloud Migration) Role Type: 5-Month Contract Location: Remote (UK-Based) Experience Level: Lead/Senior (5+ years PySpark) Role Overview We are seeking a Lead PySpark Engineer to drive a large-scale data modernisation project, transitioning legacy data workflows into a high-performance … cloud environment. This is a hands-on technical role focused on converting legacy SAS code into production-ready PySpark pipelines within a complex financial services landscape. Key Responsibilities Code Conversion: Lead the end-to-end migration of SAS code (Base SAS, Macros, DI Studio) to PySpark using automated ...

Defining and enforcing engineering standards across code quality, testing, CI/CD, IaC, and observability Building and optimising ELT/ETL pipelines using Glue (PySpark) , Lambda, Step Functions, and event-driven patterns Championing security-by-design in collaboration with security and assurance teams (IAM, KMS, encryption, auditability) Implementing … across data analytics and data engineering domains Strong hands-on experience with: Amazon S3 (data lake patterns, partitioning, lifecycle management) AWS Glue (Jobs, Crawlers, PySpark, orchestration) Amazon Redshift (data modelling, performance tuning, WLM, Spectrum) Advanced Python (PySpark) and SQL skills Experience implementing CI/CD pipelines and Infrastructure ...

Job Responsibilities: Payments and File Format Expertise Deep understanding of payment processing workflows, especially H2H and SWIFT. Hands-on experience with legacy formats (IDOC, EDIFACT) as well as ISO 20022 and GFF. Ability to interpret ...

Job Responsibilities: Payments and File Format Expertise Deep understanding of payment processing workflows, especially H2H and SWIFT. Hands-on experience with legacy formats (IDOC, EDIFACT) as well as ISO 20022 and GFF. Ability to interpret ...

Role: Data Engineer (Python, PySpark, SQL) Day rate: £475pd-£520pd (Inside IR35) Contract: 6 months initial We are currently recruiting for a Data Engineer to be part of a team on a Business Data Service Project, which is a Data Warehouse Replacement & Report simplification project. You will be responsible … aggregation of data from source to target in the Data Warehouse environment. Skills and experience required: Strong experience developing ETL/ELT pipelines using PySpark and Python Hands-on experience with Microsoft Fabric lakehouse or similar cloud data platforms (Azure Synapse Analytics, Databricks) Proficiency in working with Jupyter/ ...

reporting and analytics needs. Our client is looking for experience of working on a Fabric Lakehouse project - someone who has completed a Lakehouse with PySpark project rather than part way through. PySpark ETL experience is needed to manage the complexity of loading data from multiple ERPs from different … development management skills. Strong experience in designing and building enterprise data warehouses or data platforms in a leadership role. Completed a Lakehouse with PySpark project. PySpark ETL experience. Hands-on expertise in the development of a data Lakehouse within the Microsoft Fabric or Databricks data platforms. Significant experience ...