12 of 12 Permanent PySpark Jobs in the City of London

data warehouses or similar analytical platforms. Experience developing reliable ETL or ELT processes in the Medallion architecture. Experience with Microsoft Fabric data engineering, including Pyspark and dbt. Practical knowledge of data quality, testing, monitoring and operational support. Strong problem-solving skills and attention to detail. Clear communication ...

enterprise-scale environments. Deep expertise in Azure Databricks , Azure Data Factory , Azure Synapse , and Azure Data Lake . Hands-on experience with Spark (PySpark/SparkSQL) , ETL/ELT design, and data pipeline orchestration . Strong data modelling skills (Dimensional, ODS, Data Vault) and experience with data warehousing concepts. ...

deployment automation and infrastructure-as-code practices. What We're Looking For Essential Strong commercial experience building production-grade data pipelines using Apache Spark (PySpark preferred). Experience delivering modern data platforms using Databricks or an equivalent Spark-based platform. Strong SQL skills and experience with data modelling. Good ...

data platform on Microsoft Fabric, Databricks, Snowflake or similar, using a medallion architecture, including managing ingestion performance and forecasting platform costs. Strong Python/PySpark and SQL, with a track record of production ETL/ELT pipelines integrating data from varied business systems, including less well-documented sources. Experience ...

strong problem solving, communication, and stakeholder management skills. Requirements 4-6 years of experience in data engineering with a strong focus on Databricks (SQL & PySpark). Strong hands-on experience with Azure data services, including ADF, ADLS, and Azure Functions. Proven experience designing and implementing metadata-driven data pipelines. ...

calculation logic (cash flows, commitments, valuations, NAV, and return metrics) across data pipelines. Automated Data Quality Frameworks: Develop automated test suites using Python (PyTest) , PySpark , and advanced SQL to catch discrepancies, nulls, and duplicate records. CI/CD Quality Gates: Integrate automated testing and data validation checks into Azure ...

Gen2), Purview, Cosmos DB, OpenAI, Azure ML, and AI Foundry Solid understanding of software engineering concepts, with hands‐on experience using Python, Scala or PySpark, data modelling techniques, database technologies, and SQL Confident communicator, able to explain technical concepts to non‐technical audiences and mentor junior colleagues For Senior ...

Data Lake Storage (Gen2), Purview, Cosmos DB, OpenAI, AzureML, AIFoundry, Kubernetes Understanding of software engineering tools and concepts including experience in Python, Scala or PySpark, database technologies, data modelling and SQL Confident communicator who is able to explain technical terms to non‐technical audiences and mentor junior colleagues Leads ...

target reconciliation, validate financial calculations, and ensure 100% data integrity across Bronze/Silver/Gold layers. Responsibilities include building automated frameworks with PyTest, PySpark, and advanced SQL, integrating checks into Azure DevOps pipelines, and monitoring schema drift and #J-18808-Ljbffr ...

Physics, Statistics, or a related quantitative field. Core Technical Stack: Hands-on proficiency with Python, SQL, and Excel, alongside exposure to platforms like Databricks, PySpark, Palantir Foundry, or R. Specialized Experience: Proven track record in delivering advanced analytics, algorithm design, operational research, or big data engineering solutions. Energy & Utilities ...

ownership of client problems, handling everything from deep data engineering knowing that may require low/no code Palantir Foundry tooling, or Python and PySpark code repos - to building user-facing front-end applications and widgets in TypeScript or REACT Client Advisory & Translation: Sit directly alongside clients, distinguish their ...

Job Title Senior Data Engineer Job Description Summary Senior Data Engineer | Databricks | Agentic AI | EMEA & APAC We are the team leading the data transformation of our EMEA and APAC business, and we are hiring a ...