1 to 25 of 38 Permanent PySpark Jobs in London

data services and integration systems (e.g. Azure Data Factory, Azure Data Lake) and how they complement Fabric/Databricks Practical experience with Spark/PySpark (desirable but strong Spark experience counts) Experience implementing platform security practices (secure connectivity, secrets management, RBAC concepts) aligned to client policies Experience implementing data ...

Azure ML. Key skills and responsibilities, Demonstrated hands-on experience with Microsoft Fabric and Power BI. Extensive knowledge in data modelling, DAX, Python/PySpark, SQL/KQL. Proven record of technical leadership and effective stakeholder engagement. Design and deliver comprehensive Fabric solutions, including OneLake, Lakehouse, Warehouses, and Power ...

Advanced SQL skills for complex query validation. Experience with Agile/DevOps delivery models and JIRA test management. Familiarity with automation frameworks (SQL/PySpark preferred). Knowledge of CI/CD and release management processes. Experience with Power BI, Alteryx, and VS Code , particularly in testing integrations with ...

exposure to enterprise-level data platforms and best practices. The role will involve developing and maintaining data pipelines, transforming data using Python, SQL and PySpark , integrating data from various internal and external sources, and supporting day-to-day data operations. You’ll also contribute to data quality initiatives, documentation ...

Hands-on experience with Microsoft Purview • Strong Azure Data Platform experience: ADLS Gen2, Azure Databricks, Data Factory • Hands-on data engineering experience (Spark/PySpark, Delta Lake) • Knowledge of security and compliance controls, GDPR, PII/DLP, RBAC/ABAC • Experience leading Agile teams and managing risks, dependencies ...

/Ansible Work closely with engineering teams to deliver real-world ML systems across multiple sectors. Tech Stack You’ll Work With Python, SQL, PySpark PyTorch, TensorFlow, Scikit‐learn MLflow (core focus), Databricks, Azure IaC: Terraform, ARM, Ansible Git, Azure DevOps, Jenkins What We’re Looking For ✔️ 3+ years ...

expertise Good knowledge of Unity Catalog and Hive Metastore AWS IAM, S3, KMS, and Terraform experience Strong troubleshooting and integration support skills Desirable Python, PySpark, SQL Delta Lake/Iceberg CI/CD tools (GitLab, GitHub Actions, Azure DevOps) Data governance and lineage experience ...

understanding of DAG (Directed Acyclic Graphs) and their role in pipeline orchestration It would be nice if you could bring (desirables): Professional experience with PySpark, MySQL, Docker, Kubernetes Experience with best-practice pipeline orchestration software (Airflow, Dagster). Experience migrating code from prototype notebooks to .py, while maintaining logic ...

making Your Skills and Experience Strong commercial experience as a data engineer in modern, cloud based environments Advanced Python and SQL, with experience using PySpark or similar distributed processing frameworks Experience building and maintaining data pipelines in AWS using services such as S3, Glue or EKS Hands on experience ...

Doing Designing and delivering end‐to‐end data and analytics solutions across complex enterprise environments Building data pipelines and workflows using Python, SQL, PySpark, and low‐/no‐code tooling Supporting the development of user‐facing applications and dashboards (TypeScript/React exposure beneficial) Working closely with stakeholders ...

outputs Set up monitoring and ensure data health for outputs What we are looking for Proficiency in Python, with experience in Apache Spark and PySpark Previous experience with data analytics softwares Ability to scope new integrations and translate user requirements into technical specifications What ...

landscapes and building workflows that support better decision making. The role is end to end. That means data engineering in Palantir Foundry, Python and PySpark, through to user-facing applications in TypeScript or React, and embedding AI workflows using AIP Logic and AIP Analysts. You will have ownership over ...

into scalable technical solutions Building end-to-end data workflows using a mix of low/no-code platforms and code-based approaches (Python, PySpark, SQL) Developing user-facing applications, dashboards, and workflows using modern front-end tools such as TypeScript and React Supporting the design and evolution ...

Strong experience designing modern cloud-based data platforms. Hands-on architectural experience with: AWS (essential): S3, EMR, Glue Kafka/event streaming architectures Python & PySpark-based data processing Experience designing data ingestion pipelines (batch and real-time). Proficiency in Infrastructure as Code (Terraform). Experience with GitHub-based ...

Architecture . Build and optimize batch and real-time data pipelines across underwriting, pricing, claims, reinsurance, and bordereaux data. Develop scalable solutions using Python, PySpark, Databricks, and/or Snowflake . Integrate data from policy administration systems, claims platforms, broker systems, third-party providers, and market feeds. Ensure strong … background in Insurance , ideally within the London Market ecosystem. Deep knowledge of underwriting, claims, delegated authority, pricing, and reinsurance. Hands-on expertise in Python, PySpark, Databricks, Snowflake, SQL . Experience with Azure, AWS, or GCP . Strong understanding of batch/streaming pipelines and data modelling. Familiarity with DevOps ...

outputs Set up monitoring and ensure data health for outputs What we are looking for Proficiency in Python, with experience in Apache Spark and PySpark Previous experience with data analytics softwares Ability to scope new integrations and translate user requirements into technical specifications What ...

direction, maintaining standards, and driving output Designing and delivering data solutions across platforms, including low/no-code tooling and code-based approaches (Python, PySpark, SQL) Overseeing the development of user-facing applications and workflows using TypeScript and React Acting as the primary client interface, shaping requirements and translating ...

ownership of client problems, handling everything from deep data engineering knowing that may require low/no code Palantir Foundry tooling, or Python and PySpark code repos - to building user-facing front-end applications and widgets in TypeScript or REACT Client Advisory & Translation: Sit directly alongside clients, distinguish their ...

ownership of client problems, handling everything from deep data engineering knowing that may require low/no code Palantir Foundry tooling, or Python and PySpark code repos - to building user-facing front-end applications and widgets in TypeScript or REACT Client Advisory & Translation: Sit directly alongside clients, distinguish their ...

Azure or AWS Developing modern lakehouse architectures using: Databricks (Spark, SQL, Delta Lake) Azure Data Lake Storage or Amazon S3 Writing high-quality PySpark, Python, and SQL for ingestion, transformation, and modelling Implementing Bronze/Silver/Gold data layers Embedding data quality, validation and monitoring within pipelines Supporting … Data Engineering roles Strong experience working with Azure or AWS cloud platforms Commercial, hands-on experience using Databricks in production Proven capability across: PySpark/Python/SQL Cloud data storage (ADLS or S3) ETL/ELT pipeline development Solid understanding of data architecture, modelling, and governance Experience building ...

governed data solutions using modern cloud technologies. The Role Own the end-to-end design, build, optimisation, and support of scalable Spark/PySpark data pipelines (batch and streaming) Define and implement lakehouse architecture standards (medallion model: bronze, silver, gold), including governance, lineage, and data quality controls Design … solutions Required Skills & Experience Strong experience as a Senior or Lead Data Engineer with ownership of end-to-end data solutions Expertise in Databricks, PySpark/Spark, SQL, and Python Proven experience building and optimising large-scale data pipelines in production environments Strong knowledge of cloud data architectures , particularly ...

/PnL/Trade lifecycle/Reconciliation Understanding of financial products such as: Derivatives (swaps, futures, options), bonds Good-to-Have Exposure to PySpark or large-scale data processing Experience working with quants or pricing models Basic scripting (Unix/Bash) Prior experience in global banking environments Experience ...

individual ownership of client problems, handling everything from deep data engineering that may require low/no code Palantir Foundry tooling, or Python and PySpark code repos – to building user‐facing front‐end applications and widgets in TypeScript or React Client Advisory & Translation: Sit directly alongside clients, distinguish their ...

with ability to run various data analytics tasks and queries. Exposure to Java script technologies (preferably React) Exposure to Azure Databricks environment is desirable (PySpark, notebooks, integrate repos, job workflows, delta lake, unity catalog etc) Exposure to azure environment and various interconnected azure components (Functions, App Service, Containers etc. … Risk management is highly desirable Qualifications 8+ years of prior experience as a developer in the required technologies CORE TECHNOLOGIES Scripting : Python, SQL, Bash, PySpark Web frameworks: Django, Flask, or FastAPI. Frontend Technologies: ReactJS, Java Script Azure : Azure Databricks, Azure Synapse Analytics, Azure Functions, Azure Data Lake Storage Gen2 ...

properly, you’ll fit right in. You’ll Work With Azure Data Factory, Data Lake, Key Vault, Azure Functions Databricks, Delta Lake, Unity Catalog, PySpark, Spark SQL Medallion architecture, Lakehouse patterns, distributed compute CI/CD pipelines using Azure DevOps Cross‐functional teams in underwriting, actuarial, delegated authority, exposure … have: Strong hands‐on experience with Azure Data Services and Databricks Solid understanding of Delta Lake, Medallion architecture and Lakehouse patterns Proficiency in Python, PySpark and Spark SQL Experience building CI/CD pipelines using Azure DevOps Knowledge of data governance, lineage, access controls and FinOps Confident communication skills ...