16 of 16 Permanent PySpark Jobs in Central London

data warehouses or similar analytical platforms. Experience developing reliable ETL or ELT processes in the Medallion architecture. Experience with Microsoft Fabric data engineering, including Pyspark and dbt. Practical knowledge of data quality, testing, monitoring and operational support. Strong problem-solving skills and attention to detail. Clear communication ...

deployment automation and infrastructure-as-code practices. What We're Looking For Essential Strong commercial experience building production-grade data pipelines using Apache Spark (PySpark preferred). Experience delivering modern data platforms using Databricks or an equivalent Spark-based platform. Strong SQL skills and experience with data modelling. Good ...

data platform on Microsoft Fabric, Databricks, Snowflake or similar, using a medallion architecture, including managing ingestion performance and forecasting platform costs. Strong Python/PySpark and SQL, with a track record of production ETL/ELT pipelines integrating data from varied business systems, including less well-documented sources. Experience ...

calculation logic (cash flows, commitments, valuations, NAV, and return metrics) across data pipelines. Automated Data Quality Frameworks: Develop automated test suites using Python (PyTest) , PySpark , and advanced SQL to catch discrepancies, nulls, and duplicate records. CI/CD Quality Gates: Integrate automated testing and data validation checks into Azure ...

best practices and driving a culture of continuous improvement. Requirements 8+ years of experience in data engineering with a strong focus on Databricks (SQL & PySpark). Extensive experience with Azure data services, including ADF, ADLS, and Azure Functions. Proven track record of designing and implementing metadata‐driven data pipelines. ...

Gen2), Purview, Cosmos DB, OpenAI, Azure ML, and AI Foundry Solid understanding of software engineering concepts, with hands‐on experience using Python, Scala or PySpark, data modelling techniques, database technologies, and SQL Confident communicator, able to explain technical concepts to non‐technical audiences and mentor junior colleagues For Senior ...

Data Lake Storage (Gen2), Purview, Cosmos DB, OpenAI, AzureML, AIFoundry, Kubernetes Understanding of software engineering tools and concepts including experience in Python, Scala or PySpark, database technologies, data modelling and SQL Confident communicator who is able to explain technical terms to non‐technical audiences and mentor junior colleagues Leads ...

target reconciliation, validate financial calculations, and ensure 100% data integrity across Bronze/Silver/Gold layers. Responsibilities include building automated frameworks with PyTest, PySpark, and advanced SQL, integrating checks into Azure DevOps pipelines, and monitoring schema drift and #J-18808-Ljbffr ...

Microsoft Fabric sits at the centre of the platform: Lakehouse, Warehouse, OneLake, pipelines and notebooks, built around a medallion architecture. You'll use Python, PySpark and advanced SQL to build the pipelines, models and integrations behind the business's reporting, analytics and wider data needs. You'll also: · Integrate … year in that role working with Fabric, so you've lived with your own design decisions. You'll also bring: · Strong Python/PySpark and advanced SQL · Sound data modelling, ETL/ELT, orchestration and data quality experience · A good grasp of data governance, privacy and security ...

Physics, Statistics, or a related quantitative field. Core Technical Stack: Hands-on proficiency with Python, SQL, and Excel, alongside exposure to platforms like Databricks, PySpark, Palantir Foundry, or R. Specialized Experience: Proven track record in delivering advanced analytics, algorithm design, operational research, or big data engineering solutions. Energy & Utilities ...

ownership of client problems, handling everything from deep data engineering knowing that may require low/no code Palantir Foundry tooling, or Python and PySpark code repos - to building user-facing front-end applications and widgets in TypeScript or REACT Lead Client Advisory: Sit directly alongside clients, goals ...

ownership of client problems, handling everything from deep data engineering knowing that may require low/no code Palantir Foundry tooling, or Python and PySpark code repos - to building user-facing front-end applications and widgets in TypeScript or REACT Client Advisory & Translation: Sit directly alongside clients, distinguish their ...

bonus and an extensive benefits package. What you'll be doing Design, build and maintain scalable data pipelines and data products using Databricks , PySpark, Spark SQL and Delta Lake. Develop and optimise ETL/ELT processes to ingest, transform and curate data from internal and external systems. Shape enterprise … help mature a growing data function. What you need to bring “Senior” level experience in data engineering, with strong hands-on Databricks skills (PySpark, Spark SQL, Delta Lake, etc) Advanced SQL and Python coding skills. A track record of building and optimising enterprise-scale data pipelines and ETL/ ...

bonus and an extensive benefits package. What you'll be doing Design, build and maintain scalable data pipelines and data products using Databricks , PySpark, Spark SQL and Delta Lake. Develop and optimise ETL/ELT processes to ingest, transform and curate data from internal and external systems. Shape enterprise … help mature a growing data function. What you need to bring “Senior” level experience in data engineering, with strong hands-on Databricks skills (PySpark, Spark SQL, Delta Lake, etc) Advanced SQL and Python coding skills. A track record of building and optimising enterprise-scale data pipelines and ETL/ ...

Job Title Senior Data Engineer Job Description Summary Senior Data Engineer | Databricks | Agentic AI | EMEA & APAC We are the team leading the data transformation of our EMEA and APAC business, and we are hiring a ...

maintain production-quality Python and SQL code. Use Git-based source control and branch management practices across multiple deployment environments. Work with Spark and PySpark to process and analyse large-scale datasets. Develop and manage machine learning workflows within an MLOps environment. Use tooling such as MLflow, VS Code … model validation. Practical experience with Git, Agile delivery and collaborative software-development workflows. Experience using distributed data-processing and MLOps tools such as Spark, PySpark, MLflow, JupyterHub or equivalent technologies. Understanding of data-product lifecycles, data quality, governance, reproducibility and auditable insightful workflows. Ability to translate operational or business ...