1 to 25 of 70 Remote/Hybrid PySpark Jobs in England

TensorFlow Serving, TorchServe). Familiarity with ML lifecycle tools such as MLflow, Kubeflow, or Azure ML Pipelines. Experience working with distributed data processing (e.g., PySpark) and SQL. Understanding of software engineering best practices, including version control (Git). Knowledge of CI/CD principles in ML environments. Experience with ...

Lead Data Engineer - Pyspark/AWS/Python/SAS - Financial Sector As a Lead PySpark Engineer, you will design, develop, and fix complex data processing solutions using PySpark on AWS. You will work hands-on with code, modernising legacy data workflows and supporting large-scale … PySpark migrations. The role requires strong engineering discipline, deep data understanding, and the ability to deliver production-ready data pipelines in a financial services environment. Essential Skills PySpark & Data Engineering Minimum 5+ years of hands-on PySpark experience. SAS to Pyspark migration experience Proven ability ...

Data Warehouse capabilities. Design, manage, and optimise data storage within OneLake , ensuring well-governed, well-structured, and shareable data assets. Develop scalable Spark (PySpark/Spark SQL) data transformations through Fabric notebooks to support high-volume processing. Create, orchestrate, and monitor robust data ingestion and transformation pipelines using Fabric ...

Azure services including Azure Data Factory, Azure Data Lake Storage, Azure Key Vault, Azure Functions and Azure Databricks Develop scalable pipelines using Delta Lake, PySpark and Unity Catalog Work with underwriting, actuarial, delegated authority, bordereaux, exposure management, reinsurance, finance and risk teams to deliver solutions aligned to Lloyd … Azure Databricks in large scale environments Deep knowledge of Delta Lake, medallion architecture, distributed compute and lakehouse data platforms Strong development skills in Python, PySpark and Spark SQL Experience implementing CI/CD for data platforms using Azure DevOps Knowledge of data governance, lineage, access control and secure cloud ...

genomic data. You have A BSc or MSc in computer science or related fields Expertise in Python, including popular Python libraries: NumPy, Pandas, PySpark and frameworks: Django, Django Rest Framework, FastAPI Hands-on experience with both relational (e.g. PostgreSQL) and non-relational databases (e.g. Elasticsearch, Redis) Extensive experience ...

reporting, and downstream data services across the organisation. Key Responsibilities Design, build, and maintain data pipelines using Databricks Develop ETL/ELT processes with PySpark and Spark SQL Transform and model structured and semi-structured datasets Improve performance, reliability, and cost efficiency of data workloads Ensure compliance with data … Skills & Experience Active SC Clearance - mandatory and non-negotiable Strong hands-on experience with Databricks in production environments Expert knowledge of Apache Spark , including PySpark and Spark SQL Proficiency with Python for data engineering Experience delivering solutions on cloud data platforms ( AWS preferred , Azure acceptable) Understanding of data lake ...

design and ETL logic. DevOps: Hands-on experience with Azure DevOps , CI/CD pipelines, and Key Vault. Bonus: Experience with Azure Databricks or PySpark is highly desirable. Randstad Technologies is acting as an Employment Business in relation to this vacancy. ...

with various databases e.g. MS SQL, Azure Cosmos DB. Skilled at optimizing large and more complicated SQL statements. Proficiency in Python and experience with PySpark Experience of REST API standards. Proficiency in writing clean, readable, and secure code for Azure backend services, i.e. functions, utility functions etc. Experience ...

CDF. High technical proficiency with Cosmos DB SDKs (e.g., .NET, Java, Python, Node.js) Thorough understanding of Spark distributed computing concepts. High technical proficiency in PySpark and Spark Concepts. Experience with concurrency patterns, CLR, and scalable application design. Deep understanding of Azure services including Azure Functions, App Services ...

Lead PySpark Engineer (Cloud Migration) Role Type: 5-Month Contract Location: Remote (UK-Based) Experience Level: Lead/Senior (5+ years PySpark) Role Overview We are seeking a Lead PySpark Engineer to drive a large-scale data modernisation project, transitioning legacy data workflows into a high-performance … cloud environment. This is a hands-on technical role focused on converting legacy SAS code into production-ready PySpark pipelines within a complex financial services landscape. Key Responsibilities Code Conversion: Lead the end-to-end migration of SAS code (Base SAS, Macros, DI Studio) to PySpark using automated ...

Databricks, including workspace design and configuration. Demonstrable experience implementing the Medallion Architecture for structured data transformation. Advanced ETL/ELT development experience using PySpark, Delta Lake and Databricks SQL. Practical experience with Unity Catalog for governance, metadata management and lineage tracking. Strong knowledge of Azure Data Platform components, including ...

marketing analytics, including MMM tools such as Meridian, Robyn, or PyMC, multi-touch attribution, customer lifecycle modelling • Proficiency in SQL, Python, R, and PySpark, with experience using BI tools such as Power BI or Looker. • Practical experience with causal inference and experimentation methods. • Strong analytical capability with the ability ...

send your application for consideration Ensure that your CV is up to date, and that you have read the job specs first. Lead PySpark Engineer (Cloud Migration) Role Type: 5-Month Contract Location: Remote (UK-Based) Experience Level: Lead/Senior (5+ years PySpark) Role Overview … seeking a Lead PySpark Engineer to drive a large-scale data modernisation project, transitioning legacy data workflows into a high-performance AWS cloud environment. This is a hands-on technical role focused on converting legacy SAS code into production-ready PySpark pipelines within a complex financial services landscape. ...

ExperienceEssential Strong experience with Azure Databricks Infrastructure as Code using Terraform DevOps practices and CI/CD pipelines Git and peer review workflows Python, PySpark, and SQL Cloud operations within Azure To apply for this role please submit your CV or contact Dillon Blackburn on or at . Tenth ...

Strong experience designing and delivering data solutions in Databricks Proficient with SQL and Python Experience using Big Data technologies such as Apache Spark or PySpark Great communication skills, effectively participating with Senior Stakeholders Nice to haves: Azure/AWS Data Engineering certifications Databricks certifications What ...

Lloyd's of London or London Market insurance sector Deep understanding of Delta Lake, Medallion architecture, Lakehouse patterns and data modelling. Strong Python, PySpark and Spark SQL capability Solid understanding of data governance Proven track record of ownership, delivery and operating effectively in fast-paced environments. What ...

Azure ML. Key skills and responsibilities, Demonstrated hands-on experience with Microsoft Fabric and Power BI. Extensive knowledge in data modelling, DAX, Python/PySpark, SQL/KQL. Proven record of technical leadership and effective stakeholder engagement. Design and deliver comprehensive Fabric solutions, including OneLake, Lakehouse, Warehouses, and Power ...

Experience 5+ years in data engineering, including experience in a senior or lead position Strong expertise across AWS Hands-on experience with Databricks, PySpark, and Lakehouse patterns Strong Python and SQL engineering ability Experience implementing scalable ETL/ELT pipelines Background in data modelling and building analytics-ready datasets ...

timely manner. Key Experience, Skills and Knowledge: Proven 2 yrs+ experience as a Data Engineer or similar role, with a strong focus on PySpark, SQL, Microsoft Azure Data platforms and Power BI an advantage Proficiency in development languages suitable for intermediate-level data engineers, such as: Python/PySpark ...

analysis, model development, and data interrogation. Strong knowledge of PyTorch or similar ML frameworks. Experience handling large datasets and distributed processing with PySpark or similar is desirable . Experience of experimentation tools such as MLflow is a bonus. Comfort with deploying ML models into production environments. Familiarity with ...

analysis, model development, and data interrogation. Strong knowledge of PyTorch or similar ML frameworks. Experience handling large datasets and distributed processing with PySpark or similar is desirable . Experience of experimentation tools such as MLflow is a bonus. Comfort with deploying ML models into production environments. Familiarity with ...

tagging, monitoring and platform operations. • Architect and design end‐to‐end data platform solutions focused on scalability, security and reliability. • Lead delivery using Databricks, PySpark, Spark SQL and Azure Data Factory. • Drive DevOps best practices including CI/CD, automated testing and infrastructure as code. • Collaborate closely with data … followed. Your Skills and Experience You will bring: • Strong commercial experience in Azure and Databricks, with deep knowledge of Lakehouse architecture. • Proficiency in PySpark, Python, SQL and Spark SQL. • Strong DevOps engineering skills including CI/CD, automated testing and deployment pipelines. • Experience with monitoring, logging and alerting solutions. ...

Title: Azure Cosmos DB Admin and Developer- with python, pyspark, Cosmos, Azure Location: Hybrid, London UK (3days/Week Onsite) Duration: 6months+ Contract Inside IR35 The Role The DB Admin and Developer will play a critical role in designing and developing scalable, secure, and high-performance applications tailored ...

data processing solutions Work with modern data platforms such as Microsoft Fabric, Databricks, or similar technologies Build and optimise data solutions using Python, Spark, PySpark, and/or SQL Support data integration, transformation, and performance optimisation Collaborate with stakeholders to ensure high-quality, reliable data architecture Essential Skills & Experience … Hands-on experience with Microsoft Fabric, Databricks, or equivalent modern data platforms Strong coding experience in Python and/or Spark/PySpark Solid xkybehq SQL skills and experience working with structured datasets Desirable Experience Knowledge of relational database design, normalisation, and data modelling Experience designing efficient, scalable data ...

Lakehouse solutions across cloud environments. Responsibilities: Design, build, and maintain data pipelines using Databricks (Spark, Delta Lake). Develop batch and streaming workloads using PySpark/Spark SQL. Implement data ingestion, transformation, and orchestration pipelines. Optimise Databricks workloads for performance, reliability, and cost. Integrate Databricks with cloud-native services … across Azure, AWS, or GCP. Skills: Strong hands-on expertise with Databricks in production environments. Advanced PySpark and Spark SQL development experience. Experience building lakehouse architectures using Delta Lake. Strong cloud experience in Azure (preferred), AWS, or GCP. Experience with orchestration tools (ADF, Airflow, dbt, or similar). Certifications ...