22 of 22 Contract PySpark Jobs in England

design and ETL logic. DevOps: Hands-on experience with Azure DevOps , CI/CD pipelines, and Key Vault. Bonus: Experience with Azure Databricks or PySpark is highly desirable. Randstad Technologies is acting as an Employment Business in relation to this vacancy. ...

integration. Proficiency with Databricks for data processing and analytics. Working knowledge of NodeJS would be an added advantage. Solid programming skills in Python , PySpark, Spark . Candidate must be adept at working with large-scale datasets , S3 , Python , and data cataloguing tools . Familiarity with data engineering best practices ...

CDF. High technical proficiency with Cosmos DB SDKs (e.g., .NET, Java, Python, Node.js) Thorough understanding of Spark distributed computing concepts. High technical proficiency in PySpark and Spark Concepts. Experience with concurrency patterns, CLR, and scalable application design. Deep understanding of Azure services including Azure Functions, App Services ...

CDF. High technical proficiency with Cosmos DB SDKs (e.g., .NET, Java, Python, Node.js) Thorough understanding of Spark distributed computing concepts. High technical proficiency in PySpark and Spark Concepts. Experience with concurrency patterns, CLR, and scalable application design. Deep understanding of Azure services including Azure Functions, App Services ...

Maintain documentation, lineage, governance standards. Required Skills/Experience The ideal candidate will have the following: Strong SQL; Snowflake ELT; dbt experience. Python/PySpark, ETL/ELT design. Data Vault 2.0 or dimensional modeling. AWS services (S3, Glue, Lambda, Redshift) or GCP equivalents. Experience with CI/ ...

Lead Data Engineer - Pyspark/AWS/Python/SAS - Financial Sector As a Lead PySpark Engineer, you will design, develop, and fix complex data processing solutions using PySpark on AWS. You will work hands-on with code, modernising legacy data workflows and supporting large-scale … PySpark migrations. The role requires strong engineering discipline, deep data understanding, and the ability to deliver production-ready data pipelines in a financial services environment. Essential Skills PySpark & Data Engineering Minimum 5+ years of hands-on PySpark experience. SAS to Pyspark migration experience Proven ability ...

PyCharm, VS Code, JupyterLab, Eclipse, Notepad++, Sagemaker Studio, Azure ML Studio, Vertex AI Workbench Python Libraries NumPy, Pandas, Matplotlib, Scikit-learn, TensorFlow, Keras, PyTorch, PySpark, SpaCy, SciPy, NLTK, Statsmodels, Boto3, AzureSDK NLP & LLMs BERT, Word2Vec, Universal Sentence Encoder, NLTK, embeddings, fuzzy matching, topic modelling LLM experience: GPT-3.5/ ...

Experience Strong hands-on experience with Azure Cosmos DB Query optimisation and throughput management expertise Experience with Cosmos DB Change Data Feed (CDF) Strong PySpark and Spark distributed computing knowledge Experience integrating Spark with Cosmos CDF High proficiency with Cosmos DB SDKs (.NET, Java, Python, Node.js) Deep understanding ...

Build and operate scalable data pipelines using Microsoft Fabric (OneLake/Delta Lake, Data Factory, Synapse Data Engineering). Develop batch processing solutions using PySpark, Spark SQL, Python, and SQL , with a focus on performance, resilience, and data quality. Support reporting and MI use cases, including transformations and data … Data Engineer or equivalent. Active SC clearance (BPSS minimum). Strong hands-on experience with Microsoft Fabric and Azure data services. Advanced skills in PySpark, Spark SQL, Python, and SQL . Experience delivering data engineering solutions in government or similarly regulated environments. CI/CD, DevOps, and IaC experience ...

leadership role. Hands-on expertise in the development of a data Lakehouse within the Microsoft Fabric or Databricks data platforms. Significant experience with PySpark and Spark-based data processing. Deep understanding of data warehouse design principles, Lakehouse architectures, including medallion (Bronze/Silver/Gold) patterns. Experience integrating data ...

Looker and enabling reporting for end-users within general guardrails Your Profile Essential Skills/Knowledge/Experience Python GCP Services GCS, GKE, BigQuery PySpark, DBT IAM Role & Groups CI/CD Tools Jenkins, Harness or equivalent Design & Architecture Data product design with in guard rails and estimating FinOps ...

Lead PySpark Engineer (Cloud Migration) Role Type: 5-Month Contract Location: Remote (UK-Based) Experience Level: Lead/Senior (5+ years PySpark) Role Overview We are seeking a Lead PySpark Engineer to drive a large-scale data modernisation project, transitioning legacy data workflows into a high-performance … cloud environment. This is a hands-on technical role focused on converting legacy SAS code into production-ready PySpark pipelines within a complex financial services landscape. Key Responsibilities Code Conversion: Lead the end-to-end migration of SAS code (Base SAS, Macros, DI Studio) to PySpark using automated ...

Defining and enforcing engineering standards across code quality, testing, CI/CD, IaC, and observability Building and optimising ELT/ETL pipelines using Glue (PySpark) , Lambda, Step Functions, and event-driven patterns Championing security-by-design in collaboration with security and assurance teams (IAM, KMS, encryption, auditability) Implementing … across data analytics and data engineering domains Strong hands-on experience with: Amazon S3 (data lake patterns, partitioning, lifecycle management) AWS Glue (Jobs, Crawlers, PySpark, orchestration) Amazon Redshift (data modelling, performance tuning, WLM, Spectrum) Advanced Python (PySpark) and SQL skills Experience implementing CI/CD pipelines and Infrastructure ...

work on digital projects that improve data processes and enable smarter decision-making. Key Responsibilities Build, test, and maintain ETL pipelines using Python, PySpark, and SQL Re-engineer manual data flows for scalability and repeatability Write unit/integration tests and manage code versions with Git Support operational teams … pipelines Collaborate with colleagues to brainstorm solutions and share knowledge Report progress to line managers and peers Required Experience & Skills Strong programming in Python, PySpark, and SQL Experience with data engineering, including design, coding, testing, and documentation Ability to define test conditions and report on outcomes Understanding of technical ...

Role: Data Engineer (Python, PySpark, SQL) Day rate: £475pd-£520pd (Inside IR35) Contract: 6 months initial We are currently recruiting for a Data Engineer to be part of a team on a Business Data Service Project, which is a Data Warehouse Replacement & Report simplification project. You will be responsible … aggregation of data from source to target in the Data Warehouse environment. Skills and experience required: Strong experience developing ETL/ELT pipelines using PySpark and Python Hands-on experience with Microsoft Fabric lakehouse or similar cloud data platforms (Azure Synapse Analytics, Databricks) Proficiency in working with Jupyter/ ...

reporting and analytics needs. Our client is looking for experience of working on a Fabric Lakehouse project - someone who has completed a Lakehouse with PySpark project rather than part way through. PySpark ETL experience is needed to manage the complexity of loading data from multiple ERPs from different … development management skills. Strong experience in designing and building enterprise data warehouses or data platforms in a leadership role. Completed a Lakehouse with PySpark project. PySpark ETL experience. Hands-on expertise in the development of a data Lakehouse within the Microsoft Fabric or Databricks data platforms. Significant experience ...

Role: Data Engineer (Python, PySpark, SQL) Day rate: £400pd-£440pd (Outside IR35) Contract: 6 months initial We are seeking a highly skilled Data Engineer to support a Material Spend Project. You will play a crucial role in extracting, transforming, and analysing large data sets from multiple sources, including API … integrations and ServiceNow, to drive actionable insights and support strategic decision-making. Skills and experience required: Strong experience developing ETL/ELT pipelines using PySpark and Python Hands-on experience with Microsoft Fabric lakehouse or similar cloud data platforms (Azure Synapse Analytics, Databricks) Proficiency in working with Jupyter/ ...

Role: Join our client as a Lead PySpark Engineer to modernize legacy data workflows. You will lead the hands-on transition of complex SAS environments into scalable, AWS-native PySpark pipelines for a major financial services project. What You'll Do Modernize: Convert legacy SAS code into production … ready PySpark using automated tools and manual refactoring. Build: Design and fix high-performance ETL/ELT pipelines and data marts on AWS. Your Technical Profile PySpark (Expert): 5+ years of hands-on experience. Deep knowledge of Spark optimization (partitioning, tuning) and Data Warehousing (SCDs, Facts/Dimensions ...

enhance a scalable data-engineering framework used across critical reporting processes. You'll operate as the senior hands-on engineer, shaping reusable libraries, optimising PySpark pipelines, and guiding offshore developers to deliver high-quality, production-ready code. You'll work across ingestion, validation, transformation, and mapping layers within … patterns and ensuring all deliveries meet high standards of performance, traceability, and maintainability. What you'd be doing: Building and extending reusable Databricks/PySpark libraries (ingestion, validation, transformation, mapping). Developing scalable, optimised PySpark pipelines aligned to metadata-driven or hybrid data models. Implementing robust validation ...

proven background and experience of full cycle development of a corporate, Microsoft based Data Warehouse. Desired skills: Design, Build, Test, Azure, Fabric Lakehouse, Spark, PySpark We are looking to recruit a high-calibre resource, so the client is happy to consider candidates from both a contract and permanent background. … your most recent CV to be considered for telephone screening. SQL/BI/ETL/DATAWAREHOUSE/AZURE/FABRIC/SPARK/PYSPARK/LEAD/MANAGER/HOME/REMOTE/MIDLANDS/BIRMINGHAM/SQL/BI/ETL/DATAWAREHOUSE/AZURE/FABRIC ...

Lead Data Engineer - Pyspark/AWS/Python/SAS - Financial Sector As a Lead PySpark Engineer, you will design, develop, and fix complex data processing solutions using PySpark on AWS. You will work hands-on with code, modernising legacy data workflows and supporting large-scale … PySpark migrations click apply for full job details ...

Lead PySpark Engineer (Cloud Migration) Role Type: 5-Month Contract Location: Remote (UK-Based) Experience Level: Lead/Senior (5+ years PySpark) Role Overview We are seeking a Lead PySpark Engineer to drive a large-scale data modernisation project, transitioning legacy data workflows into a high-performance ...