20 of 20 PySpark Jobs in Central London

security measures across the platform. Leverage Terraform or similar IaC tools for controlled and reproducible deployments. Databricks Development Develop and optimise data jobs using PySpark or Scala within Databricks. Implement the medallion architecture (bronze, silver, gold layers) and use Delta Lake for reliable data transactions. Manage cluster configurations ...

security measures across the platform. Leverage Terraform or similar IaC tools for controlled and reproducible deployments. Databricks Development Develop and optimise data jobs using PySpark or Scala within Databricks. Implement the medallion architecture (bronze, silver, gold layers) and use Delta Lake for reliable data transactions. Manage cluster configurations ...

DevOps and broader software engineering standards. Cloud experience (any platform). Previous mentoring experience. Nice to have: Snowflake or Databricks Spark, PySpark, Hadoop or similar big data tooling BI exposure (PowerBI, Tableau, etc.) Interview Process The process is fully structured, transparent, and efficient: Video call - high-level overview ...

teams ✔ Strong Insurance domain experience (Policy, Claims, Regulatory Data) ✔ 8–10+ years in Data Engineering/Integration ✔ Deep expertise in Azure & Databricks ✔ Strong Spark (PySpark/SparkSQL) skills ✔ Experience designing scalable, high-performance data platforms ✔ Confident with SQL, CI/CD & DevOps practices Why Join? You’ll be part ...

Required Medical industry experience (essential) Strong knowledge of Azure Databricks, Palantir Foundry, and Azure DevOps Proficiency in Terraform, Git, and automation tools Experience with PySpark, Python, and SQL Familiarity with PowerBI, Azure Functions, and data streaming Understanding of cloud infrastructure and public cloud operations Excellent problem-solving and analytical ...

Salary range: £75,000-£85,000 + 10% bonus + benefits Purpose: Build and maintain large, scalable Data Lakes, processes and pipelines Tech: Python, PySpark, AWS, Iceberg/Kafka, Spark/Glue, CI/CD Industry: Financial services/securities trading Immersum continue to support a leading SaaS securities ...

across multiple time zones, including offshore development teams Skills required: Proven track record building scalable data pipelines (batch and streaming) in production Expert Python, PySpark, Celery and RabbitMQ skills; deep experience with AWS data stack (Glue, OpenSearch, RDS) Expert skills within SQL with experience in both transactional RDBMS systems ...

skills/experience you'll already have? 3+ years commercial experience in Data Engineering SQL Server Spark SQL Strong Python coding skills PySpark Azure experience including Azure SQL, Data Factory and Functions Experience collaborating with Software and Test Engineers Creating, documenting and maintaining database schemas To find out more ...

reservations, Event Hub TUs, serverless compute clusters, and Terraform deployments Required Skills * Azure Databricks * Git/GitHub * DevOps & Cloud Operations * Azure Functions * Data Streaming * PySpark, Python, SQL * Automation & Public Cloud * Cloud Infrastructure * Terraform * Palantir Foundry * PowerBI * Software Lifecycle Management If you meet the requirements please send me a copy ...

troubleshooting and incident response for deployed ML systems Required Skills and Experience Strong programming skills in Python; experience with ML libraries such as Snowpark, PySpark, or PyTorch Experience with containerization tools like Docker and orchestration tools like Airflow or Astronomer Familiarity with cloud platforms (AWS, Azure) and ML services ...

timely manner. Key Experience, Skills and Knowledge: Proven 2 yrs+ experience as a Data Engineer or similar role, with a strong focus on PySpark, SQL, Microsoft Azure Data platforms and Power BI an advantage Proficiency in development languages suitable for intermediate-level data engineers, such as: Python/PySpark ...

independently and collaborate within cross-functional teams Some knowledge of programming languages such as Python is preferred Experience with data processing technologies such as PySpark is preferred Familiarity with AWS* services and cloud-based data workflows is preferred ...

Pipeline Builder Knowledge of end-to-end work - Ontology and Data connection Experience around Architectural principles Programming Skills - proficiency in Python, SQL and PySpark Data modeling and basic concepts of data science (classification, clustering, regression) Strong client-facing skills: You are comfortable interacting with both business and technical clients ...

solutions. Hands-on knowledge of Palantir Foundry (Ontology, pipelines, apps, AIP); Palantir certifications desirable. Proficiency in at least one coding language (e.g., Python/PySpark, R, Java, SQL, Typescript) and experience with object-oriented design or software engineering. Strong understanding of data engineering, modelling, platform integration, UX design ...

with ambiguity and will want to dive deep and fix problems at scale. The environment looks a bit like this 👇 👉 Python (pandas, numpy, scipy, PySpark) 👉 SQL 👉 AWS or GCP (BigQuery/RedShift/Snowflake) 👉 Spark/Databricks 👉 Airflow/Dagster ...

Looking For Strong data modelling expertise and experience with Data Lineage Maps . Hands-on skills with SQL/Python , Synapse Notebooks , PySpark , and Delta Lake . Familiarity with MS Fabric and medallion architecture . Excellent communication and stakeholder management skills. Agile mindset and ability to thrive in cross ...

support analytics and downstream use cases Developing scalable data solutions using Python , with a focus on maintainability and performance Working with Apache Spark/PySpark to process and transform large datasets Supporting the ingestion, transformation and validation of complex financial data Improving the performance, reliability and resilience of existing … Succeed Essential Commercial experience as a Data Engineer at a mid‐level Strong Python development skills Hands‐on experience with Apache Spark/PySpark Solid experience building ETL/ELT pipelines Background within the financial services industry (investment management experience desirable) Comfortable working with production systems in a regulated ...

Adapt and deploy the platform to customer-specific contexts and priorities Design and implement scalable generative AI-enabled workflows Execute complex data integrations using PySpark and distributed data technologies Collaborate closely with stakeholders to define success metrics and outcomes What We’re Looking For 2–4+ years’ experience … strong desire to work in such environments) Proven ability to translate ambiguous business requirements into clean, maintainable data pipelines Strong fundamentals in Python, PySpark, and SQL Ability to learn new technologies quickly and operate with a high degree of ownership Nice to Have Experience with dashboarding or data visualisation ...

solutions. GenAI Implementation: Hands-on deployment of Azure OpenAI, Vector Databases, and RAG patterns. Data Engineering: Oversee large-scale data processing using Python and PySpark to support AI model fine-tuning. Leadership: Manage technical delivery teams and deliver upskilling sessions to the customer. Stakeholder Management: Gain consensus on complex … roadmaps with diverse client groups. Technical Skills & Experience Core AI: Azure OpenAI, Prompt Engineering, Cognitive Search, Form Recognizer. Data/Coding: Expert-level Python, PySpark, and Vector DB integration. Ops: Practical knowledge of MLOps and LLMOps lifecycles. Security: Must hold active UK Government SC Clearance. Certifications Mandatory ...

champion best practice methods; upskill the team through coaching and code reviews. Query and engineer large datasets (e.g., Python/R/SQL/PySpark) on modern platforms (e.g., Azure Databricks). Communicate pricing proposals and insights clearly and persuasively to stakeholders at all levels. Collaborate with Modelling, Optimisation … insurance pricing. Strong technical expertise in predictive modelling techniques (e.g. GLMs and machine learning models). Proficiency in programming languages such as R, Python, PySpark, SAS, or SQL. Good knowledge of Azure Databricks and modern data technologies. Experience with WTW Radar software and optimisation tools. A quantitative degree ...