24 of 24 PySpark Jobs in Central London

Azure ML. Key skills and responsibilities, Demonstrated hands-on experience with Microsoft Fabric and Power BI. Extensive knowledge in data modelling, DAX, Python/PySpark, SQL/KQL. Proven record of technical leadership and effective stakeholder engagement. Design and deliver comprehensive Fabric solutions, including OneLake, Lakehouse, Warehouses, and Power ...

performance, efficiency, and cost-effectiveness. Implement data quality checks and validation rules within data pipelines. Data Transformation & Processing: Implement complex data transformations using Spark (PySpark or Scala) and other relevant technologies. Develop and maintain data processing logic for cleaning, enriching, and aggregating data. Ensure data consistency and accuracy throughout … years of experience in data engineering, with at least 3+ years of hands-on experience with Azure Databricks. Strong proficiency in Python and Spark (PySpark) or Scala. Deep understanding of data warehousing principles, data modelling techniques, and data integration patterns. Extensive experience with Azure data services, including Azure Data ...

agile environment What we need Strong experience as a Data Engineer in Azure environments Strong hands-on experience with: Azure Data Factory Databricks/PySpark SQL Server/T-SQL Experience building production-grade data pipelines and data models Exposure to identity/access/security data Understanding ...

RBAC and governance controls across Azure services Tech stack Azure Azure Databricks Terraform Azure DevOps/CI/CD Python SQL Apache Spark/PySpark Entra ID/RBAC/governance Networking/platform security/automation What were looking for Strong experience in Azure platform engineering Strong hands ...

Doing Designing and delivering end‐to‐end data and analytics solutions across complex enterprise environments Building data pipelines and workflows using Python, SQL, PySpark, and low‐/no‐code tooling Supporting the development of user‐facing applications and dashboards (TypeScript/React exposure beneficial) Working closely with stakeholders ...

Microsoft Data & Analytics within consultancy or large enterprise environments Strong expertise in Microsoft Fabric, Purview, Power BI , data architecture and modelling Experience with Python, PySpark and T-SQL Confident influencing senior stakeholders and shaping enterprise solutions Eligible for SC Clearance (five years continuous UK residency) Please apply with ...

scalable, secure data solutions using Azure Databricks, Data Factory and Data Lake Storage. Develop and optimise advanced data pipelines with Python, SQL, Spark/PySpark and Delta Lake. Champion strong data quality, governance and observability practices. Modernise legacy systems and support large-scale migrations into Azure. Provide technical leadership … Criteria Extensive experience with Azure services including Azure Databricks, Azure Data Lake Storage, and Azure Data Factory. Advanced proficiency in SQL, Python, and Spark (PySpark), with a strong focus on performance optimization and distributed processing. Proven experience in CI/CD practices using industry-standard tools (e.g., GitHub Actions ...

outputs Set up monitoring and ensure data health for outputs What we are looking for Proficiency in Python, with experience in Apache Spark and PySpark Previous experience with data analytics softwares Ability to scope new integrations and translate user requirements into technical specifications What ...

landscapes and building workflows that support better decision making. The role is end to end. That means data engineering in Palantir Foundry, Python and PySpark, through to user-facing applications in TypeScript or React, and embedding AI workflows using AIP Logic and AIP Analysts. You will have ownership over ...

Strong experience building and supporting data pipelines in production Experience working with Palantir Foundry Proficiency in Python and large scale data processing (e.g. PySpark) Experience working within cloud environments such as AWS or GCP, Azure also considered Good understanding of end to end data workflows Strong problem solving skills ...

governed data solutions using modern cloud technologies. The Role Own the end-to-end design, build, optimisation, and support of scalable Spark/PySpark data pipelines (batch and streaming) Define and implement lakehouse architecture standards (medallion model: bronze, silver, gold), including governance, lineage, and data quality controls Design … solutions Required Skills & Experience Strong experience as a Senior or Lead Data Engineer with ownership of end-to-end data solutions Expertise in Databricks, PySpark/Spark, SQL, and Python Proven experience building and optimising large-scale data pipelines in production environments Strong knowledge of cloud data architectures , particularly ...

governed data solutions using modern cloud technologies. The Role Own the end-to-end design, build, optimisation, and support of scalable Spark/PySpark data pipelines (batch and streaming) Define and implement lakehouse architecture standards (medallion model: bronze, silver, gold), including governance, lineage, and data quality controls Design … solutions Required Skills & Experience Strong experience as a Senior or Lead Data Engineer with ownership of end-to-end data solutions Expertise in Databricks, PySpark/Spark, SQL, and Python Proven experience building and optimising large-scale data pipelines in production environments Strong knowledge of cloud data architectures , particularly ...

Architecture . Build and optimize batch and real-time data pipelines across underwriting, pricing, claims, reinsurance, and bordereaux data. Develop scalable solutions using Python, PySpark, Databricks, and/or Snowflake . Integrate data from policy administration systems, claims platforms, broker systems, third-party providers, and market feeds. Ensure strong … background in Insurance , ideally within the London Market ecosystem. Deep knowledge of underwriting, claims, delegated authority, pricing, and reinsurance. Hands-on expertise in Python, PySpark, Databricks, Snowflake, SQL . Experience with Azure, AWS, or GCP . Strong understanding of batch/streaming pipelines and data modelling. Familiarity with DevOps ...

outputs Set up monitoring and ensure data health for outputs What we are looking for Proficiency in Python, with experience in Apache Spark and PySpark Previous experience with data analytics softwares Ability to scope new integrations and translate user requirements into technical specifications What ...

direction, maintaining standards, and driving output Designing and delivering data solutions across platforms, including low/no-code tooling and code-based approaches (Python, PySpark, SQL) Overseeing the development of user-facing applications and workflows using TypeScript and React Acting as the primary client interface, shaping requirements and translating ...

/PnL/Trade lifecycle/Reconciliation Understanding of financial products such as: Derivatives (swaps, futures, options), bonds Good-to-Have Exposure to PySpark or large-scale data processing Experience working with quants or pricing models Basic scripting (Unix/Bash) Prior experience in global banking environments Experience ...

ownership of data pipelines and automation of data workflows. Finally have 5 years Strong programming experience at an advanced level in Python, PySpark, and SQL scripting. Benefits Salary: 50,000 - 65,000 per annum 25 day's holiday Company car/Allowance Variable annual bonus based 5-15% Pension … leveraging tools such as Foundry Code Workbooks, Pipelines, and Object Explorer for efficient data processing and integration. Strong coding skills in Python (Pandas) and PySpark: Essential experience in using Python (Pandas) and PySpark for data manipulation, transformation, and pipeline development within Palantir Foundry, ensuring high performance and scalability ...

properly, you’ll fit right in. You’ll Work With Azure Data Factory, Data Lake, Key Vault, Azure Functions Databricks, Delta Lake, Unity Catalog, PySpark, Spark SQL Medallion architecture, Lakehouse patterns, distributed compute CI/CD pipelines using Azure DevOps Cross‐functional teams in underwriting, actuarial, delegated authority, exposure … have: Strong hands‐on experience with Azure Data Services and Databricks Solid understanding of Delta Lake, Medallion architecture and Lakehouse patterns Proficiency in Python, PySpark and Spark SQL Experience building CI/CD pipelines using Azure DevOps Knowledge of data governance, lineage, access controls and FinOps Confident communication skills ...

Novatus Global is a Series B scale-up RegTech SaaS provider and boutique advisory firm, helping financial institutions manage their most complex regulatory requirements. We combine deep consulting expertise with cutting-edge SaaS solutions, enabling ...

native architectures, with experience in the financial sector. Experience with CI/CD , microservices , and event-driven architectures is essential, and knowledge of Python (pyspark) for data-driven workflows is advantageous Key Responsibilities Design, develop, and maintain full stack applications using React JS , C#.NET Core , and supporting technologies. Build … C#, REST APIs, Web APIs, and microservices-based development. Experience in transforming monolithic applications into composable microservices, or modular monoliths is a plus. Python (pyspark): Some hands-on experience with distributed data processing, data pipelines, and analytic transformations. Power BI: Some experience developing dashboards, data models, DAX calculations ...

environments. Key Responsibilities Design and build scalable cloud-based data platforms using Medallion Architecture (Bronze/Silver/Gold) Engineer pipelines using Python and PySpark on platforms such as Databricks Ingest and integrate data from multiple sources including policy, claims, broker, and third‐party systems Contribute to CI/… business stakeholders Required Experience Strong experience as a Senior Data Engineer within Insurance, ideally the London Market Proven hands-on expertise in: Python & PySpark Databricks Cloud platforms (Azure, AWS, or GCP) Solid understanding of Medallion Architecture and analytics-focused data modelling Strong domain knowledge across: Lloyd’s Syndicates Delegated ...

Skills and Experience Essential Skills Strong hands‐on experience with Palantir Foundry , including Code Repositories, Workshop, AIP Logic, and Automate Proficiency in Python and PySpark or TypeScript Experience working with AWS services such as S3, Lambda, Glue, and related data services Solid understanding of data pipeline architectures, ETL processes … problem‐solving skills with the ability to communicate clearly with both technical and non‐technical stakeholders Preferred Qualifications Additional experience with Foundry, AIP, and PySpark Knowledge of monitoring, alerting, and logging tools and frameworks Exposure to TypeScript/React for Workshop applications and workflow orchestration Experience working within ...

Your Responsibilities: Design and maintain scalable data pipelines using PySpark, Python, and distributed computing frameworks to support high‐volume data processing. Architect and optimize AWS-based data and AI infrastructure, ensuring secure, performant, and cost‐efficient ingestion, transformation, and storage. Develop, finetune, benchmark, and evaluate GenAI/LLM models … cross-functional teams to integrate GenAI capabilities into production systems and ensure high-quality data, governance, and operational reliability Your Profile: Strong experience with PySpark, distributed data processing, and largescale ETL/ELT pipelines. Strong SQL expertise including star/snowflake schema design, indexing strategies, writing optimized queries ...

Lakehouse architecture Design and deliver high-quality, analytics-ready data models that support trusted reporting Translate complex business requirements into robust SQL and PySpark transformation pipelines Shape how data is structured, modelled and served across multiple domains Embed data quality, testing, reliability and performance standards into pipelines Contribute … Experience Strong commercial experience as an Analytics Engineer operating in a modern data platform Advanced data modelling capability, including dimensional and semantic modelling Strong PySpark development experience within a Lakehouse environment Advanced SQL skills for complex transformations Hands-on experience with Databricks, including pipelines and governance tooling A strong ...