1 to 25 of 40 Remote/Hybrid PySpark Jobs in the UK

backgrounds. The technology stack is flexible, but experience in some of the following are nice to have: Data Engineering & Programming: • Python • SQL • Spark/PySpark • Scala • Data modelling and warehousing concepts Cloud & Platforms: • AWS (S3, Redshift, Glue, EMR, Lambda) • Other cloud platforms (Azure/GCP considered) • Data lake ...

Designing and building cloud based data and machine learning pipelines that prepare data for analytics, AI, and product use ·Writing clear, well-structured Python, PySpark, and SQL to transform and validate data from multiple upstream sources ·Taking ownership of data quality, consistency, and reliability across the pipeline lifecycle ·Shaping ...

demand delivery Provide technical guidance for team members Providing 2nd or 3rd level technical support About You Experience using SQL, SQL Server DB, Python & PySpark Experience using Azure Data Factory Experience using Data Bricks and Cloudsmith Data Warehousing Experience Project Management Experience The ability to interact with the operational ...

design and ETL logic. DevOps: Hands-on experience with Azure DevOps , CI/CD pipelines, and Key Vault. Bonus: Experience with Azure Databricks or PySpark is highly desirable. Randstad Technologies is acting as an Employment Business in relation to this vacancy. ...

integration. Proficiency with Databricks for data processing and analytics. Working knowledge of NodeJS would be an added advantage. Solid programming skills in Python , PySpark, Spark . Candidate must be adept at working with large-scale datasets , S3 , Python , and data cataloguing tools . Familiarity with data engineering best practices ...

with various databases e.g. MS SQL, Azure Cosmos DB. Skilled at optimizing large and more complicated SQL statements. Proficiency in Python and experience with PySpark Experience of REST API standards. Proficiency in writing clean, readable, and secure code for Azure backend services, i.e. functions, utility functions etc. Experience ...

CDF. High technical proficiency with Cosmos DB SDKs (e.g., .NET, Java, Python, Node.js) Thorough understanding of Spark distributed computing concepts. High technical proficiency in PySpark and Spark Concepts. Experience with concurrency patterns, CLR, and scalable application design. Deep understanding of Azure services including Azure Functions, App Services ...

Strong experience designing and delivering data solutions in Databricks Proficient with SQL and Python Experience using Big Data technologies such as Apache Spark or PySpark Great communication skills, effectively participating with Senior Stakeholders Nice to haves: Azure/AWS Data Engineering certifications Databricks certifications What ...

Lead Data Engineer - Pyspark/AWS/Python/SAS - Financial Sector As a Lead PySpark Engineer, you will design, develop, and fix complex data processing solutions using PySpark on AWS. You will work hands-on with code, modernising legacy data workflows and supporting large-scale … PySpark migrations. The role requires strong engineering discipline, deep data understanding, and the ability to deliver production-ready data pipelines in a financial services environment. Essential Skills PySpark & Data Engineering Minimum 5+ years of hands-on PySpark experience. SAS to Pyspark migration experience Proven ability ...

grades at A-level You have commercial Data Engineering experience working with technologies such as SQL, Apache Spark and Python including PySpark and Pandas You have a good understanding of modern data engineering best practices Ideally you will also have experience with Azure and Data Bricks You're excited ...

Experience Strong hands-on experience with Azure Cosmos DB Query optimisation and throughput management expertise Experience with Cosmos DB Change Data Feed (CDF) Strong PySpark and Spark distributed computing knowledge Experience integrating Spark with Cosmos CDF High proficiency with Cosmos DB SDKs (.NET, Java, Python, Node.js) Deep understanding ...

Required Strong hands-on experience with Microsoft Fabric Experience building end-to-end Lakehouse solutions Good data modelling skills (e.g. star schema) Experience using PySpark or Spark SQL Building data pipelines from APIs, ERP or SaaS systems Working with Fabric pipelines, notebooks and Dataflows Understanding of data governance ...

extend AI-enabled workflows as part of the core product (e.g. decision support, automation, insight generation) Develop and maintain data pipelines using Python, PySpark, SQL , and distributed systems Make pragmatic engineering decisions that balance short-term delivery with long-term product quality Contribute to technical and architectural discussions ...

marketing analytics, including MMM tools such as Meridian, Robyn, or PyMC, multi-touch attribution, customer lifecycle modelling * Proficiency in SQL, Python, R, and PySpark, with experience using BI tools such as Power BI or Looker. * Practical experience with causal inference and experimentation methods. * Strong analytical capability with the ability ...

analysis, model development, and data interrogation. Strong knowledge of PyTorch or similar ML frameworks. Experience handling large datasets and distributed processing with PySpark or similar is desirable . Experience of experimentation tools such as MLflow is a bonus. Comfort with deploying ML models into production environments. Familiarity with ...

analysis, model development, and data interrogation. Strong knowledge of PyTorch or similar ML frameworks. Experience handling large datasets and distributed processing with PySpark or similar is desirable . Experience of experimentation tools such as MLflow is a bonus. Comfort with deploying ML models into production environments. Familiarity with ...

different customer contexts and priorities Design and implement AI-enabled workflows that support decision-making and automation Build and maintain data integrations using Python, PySpark, SQL , and distributed systems Partner with customers to define success metrics and ensure solutions deliver real outcomes Feed learnings from the field back into ...

model development. What you’ll bring: Minimum of two years’ experience in a data science, machine learning, or similar role Strong Python (including PySpark) and SQL skills for data analysis, feature engineering, and modelling Experience building and deploying end-to-end predictive models from scoping through validation and stakeholder ...

leadership role. Hands-on expertise in the development of a data Lakehouse within the Microsoft Fabric or Databricks data platforms. Significant experience with PySpark and Spark-based data processing. Deep understanding of data warehouse design principles, Lakehouse architectures, including medallion (Bronze/Silver/Gold) patterns. Experience integrating data ...

leadership role. Hands-on expertise in the development of a data Lakehouse within the Microsoft Fabric or Databricks data platforms. Significant experience with PySpark and Spark-based data processing. Deep understanding of data warehouse design principles, Lakehouse architectures, including medallion (Bronze/Silver/Gold) patterns. Experience integrating data ...

have but not essential Experience working with Feedzai, behavioural data, or complementary analytics tools. Experience working with notebooks in Jupyter Labs and PySpark on large dataframes. Exposure to industry forums, law-enforcement collaboration, or external fraud intelligence networks. Practical understanding of machine-learning concepts or model performance techniques that ...

engineering is done across the organisation. Skills & Experience Strong hands-on experience with Azure data platforms Advanced SQL skills Commercial experience using Databricks and PySpark Proven background building and maintaining scalable data pipelines If you're looking for a role where you can combine technical depth, ownership, and influence ...

across Bronze, Silver, Staging and Gold layers, operating within an established architectural framework. Key Responsibilities Develop and maintain data pipelines in Azure Databricks using PySpark and SQL Build incremental processing using Delta Lake and Change Data Feed Extend and maintain watermark-driven CDC frameworks Implement and maintain Slowly Changing ...

Your Role: Design, develop, and optimise large-scale cloud data platforms using Azure Data Factory, Data Lake, Key Vault, Azure Functions, Databricks, Delta Lake, PySpark, and Unity Catalog. Collaborate with underwriting, actuarial, Delegated Authority, bordereaux, exposure management, reinsurance, finance, and Solvency II teams to meet Lloyd's market … highly regulated environments. Deep understanding of Delta Lake, Medallion architecture, Lakehouse patterns, distributed compute, data modelling, and performance optimisation. Strong programming skills in Python, PySpark, and Spark SQL, with experience implementing CI/CD pipelines (Azure DevOps, branching/merging, automation). Solid knowledge of data governance, lineage, access ...

Lead PySpark Engineer (Cloud Migration) Role Type: 5-Month Contract Location: Remote (UK-Based) Experience Level: Lead/Senior (5+ years PySpark) Role Overview We are seeking a Lead PySpark Engineer to drive a large-scale data modernisation project, transitioning legacy data workflows into a high-performance … cloud environment. This is a hands-on technical role focused on converting legacy SAS code into production-ready PySpark pipelines within a complex financial services landscape. Key Responsibilities Code Conversion: Lead the end-to-end migration of SAS code (Base SAS, Macros, DI Studio) to PySpark using automated ...