101 to 125 of 142 PySpark Jobs in London

ownership of client problems, handling everything from deep data engineering knowing that may require low/no code Palantir Foundry tooling, or Python and PySpark code repos - to building user-facing front-end applications and widgets in TypeScript or REACT Client Advisory & Translation: Sit directly alongside clients, distinguish their ...

e.g. Mathematics, Statistics, Computer Science). Nice to Have: Familiarity with production environments, MLOps, or data pipelines. Experience working with large-scale data using PySpark or equivalent distributed data processing tools. Experience in gaming, e-commerce, or subscription-based products. Benefits: Discretionary bonus opportunity Private Medical Insurance Dental Scheme ...

will need for success****Technical knowledge and experience:** * Proven experience using Databricks (essential)* Strong programming skills in Python (essential)* Experience working with: + PySpark + Large-scale data processing environments + Complex behavioural datasets* Experience cleaning, preparing and processing: + Large datasets + Real-world data containing inconsistencies ...

driven patterns). Implement secure authentication, authorization, rate limiting, and multi-tenant access controls. Data Engineering & Pipeline Development Design and maintain data pipelines using PySpark and Databricks, following the medallion architecture (Bronze/Silver/Gold) for structured data transformation and quality. Build and optimize ETL/ELT workflows … architectures. Proficiency with SQL and NoSQL databases (PostgreSQL, DynamoDB, Redis, MongoDB) and familiarity with lakehouse platforms. Experience with or strong willingness to work in PySpark and Databricks, including the medallion architecture (Bronze/Silver/Gold). Hands-on experience with Docker, Kubernetes, CloudFormation, and cloud platforms ...

requirements into scalable data solutions. Key Responsibilities Design scalable enterprise data models and lakehouse architectures Build production-grade data pipelines using SQL, Python/PySpark and Databricks Implement data governance, security and access controls through Unity Catalog Work directly with business stakeholders to understand and prioritise requirements Establish data … data engineering and advanced data modelling Hands-on expertise in Databricks, Delta Lake, Unity Catalog and Databricks Workflows Production experience with SQL, Python/PySpark and dbt and/or Spark Declarative Pipelines Strong understanding of data governance, security and access-control design Excellent stakeholder-management and business-communication ...

bonus and an extensive benefits package. What you'll be doing Design, build and maintain scalable data pipelines and data products using Databricks , PySpark, Spark SQL and Delta Lake. Develop and optimise ETL/ELT processes to ingest, transform and curate data from internal and external systems. Shape enterprise … help mature a growing data function. What you need to bring “Senior” level experience in data engineering, with strong hands-on Databricks skills (PySpark, Spark SQL, Delta Lake, etc) Advanced SQL and Python coding skills. A track record of building and optimising enterprise-scale data pipelines and ETL/ ...

plus 10% bonus and an extensive benefits package. What you'll be doingDesign, build and maintain scalable data pipelines and data products using Databricks, PySpark, Spark SQL and Delta Lake.Develop and optimise ETL/ELT processes to ingest, transform and curate data from internal and external systems.Shape enterprise data … engineer and help mature a growing data function.What you need to bring“Senior” level experience in data engineering, with strong hands-on Databricks skills (PySpark, Spark SQL, Delta Lake, etc)Advanced SQL and Python coding skills.A track record of building and optimising enterprise-scale data pipelines and ETL/ ...

bonus and an extensive benefits package. What you'll be doing Design, build and maintain scalable data pipelines and data products using Databricks , PySpark, Spark SQL and Delta Lake. Develop and optimise ETL/ELT processes to ingest, transform and curate data from internal and external systems. Shape enterprise … help mature a growing data function. What you need to bring “Senior” level experience in data engineering, with strong hands-on Databricks skills (PySpark, Spark SQL, Delta Lake, etc) Advanced SQL and Python coding skills. A track record of building and optimising enterprise-scale data pipelines and ETL/ ...

What you'll do Data Engineering Delivery: Design, build and optimise scalable data pipelines, ETL/ELT workflows and data transformation solutions using Databricks, PySpark, SQL and dbt Cloud, delivering trusted, high-quality data products that support reporting, analytics, AI and data science. Data Architecture & Modelling: Develop and maintain … Advanced Data Engineering Experience (5-8 Years): Strong hands-on experience designing, building and optimising enterprise-scale data pipelines and data platforms using Databricks, PySpark and advanced SQL. Cloud Data Platforms & Architecture: Proven experience working with Azure-based data technologies and designing scalable lakehouse architectures, data models and modern ...

Azure Data Engineer - 1/2 days onsite 18 month FTC Summary: Join a team building a modern Azure-based data platform. This hands-on engineering role involves designing and developing scalable, automated data pipelines ...

will play a key role in data-related activities for datasets used by Quants and Traders. The role requires strong SQL and Python skills (PySpark, Pandas, Jupyter Notebooks), plus experience with Airflow, Linux/Bash, and data-quality practices. You will collaborate with stakeholders and help improve data pipelines. ...

scale a global organization through resourcing/location strategy. Qualifications Proven track record delivering solutions in complex organizations. Experience with AWS, Databricks, DBT, PySpark, React, JavaScript Terraform, etc. Fast learner, thrives in ambiguity, translates vision into product scope. Passion for automation and user experience innovation. Builds high-performing teams ...

solutions that enhance business decision-making. This is an exciting opportunity for a hands-on engineer with strong expertise in Databricks, Python, SQL, and PySpark , combined with practical experience using modern AI-assisted development tools such as GitHub Copilot, ChatGPT, Claude Code, Cursor, and VS Code . Key Responsibilities … practices. Required Skills & Experience 3+ years of experience within Data Engineering, Analytics Engineering, or Data Analytics. Strong hands-on experience with: Databricks SQL Python PySpark Delta Lake Experience designing ETL/ELT pipelines and data transformation frameworks. Strong understanding of data modelling, performance optimisation, and data quality practices. Experience ...

understand their needs and translate them into technical solutions Transform data in Lakehouses and deliver clean, high-quality data for analysis and reporting Utilize PySpark to transform raw data into high-quality, analysis-ready products Implement rigorous automated data quality checks and proactive monitoring to ensure clean Data … Built and maintained data products using the Microsoft Data Stack (Azure Data Factory, Data Lake Storage, Fabric or Synapse Analytics) Expert-level proficiency in PySpark/Python Applied experience of DBT, AirFlow, or similar for data transformation and orchestration Deep understanding of data modeling, warehousing concepts, and the Medallion ...

Job Title Senior Data Engineer Job Description Summary Senior Data Engineer | Databricks | Agentic AI | EMEA & APAC We are the team leading the data transformation of our EMEA and APAC business, and we are hiring a ...

Cognizant in London is seeking a Lead Developer to design, implement and optimize Python and cloud-based solutions using AWS and PySpark for data-intensive, regulatory-focused applications. Hybrid day shift model in Stratford London (3 days a week). You will own end-to-end technical delivery, ensure ...

Senior Data Engineer - Analytics Engineering - Databricks - Pyspark - Dimensional Models - Perm We're looking for a Senior Analytics Engineer to lead the design and delivery of curated, analytics-ready data within our Lakehouse environment. You'll translate complex business requirements into robust SQL and PySpark pipelines, own the data … deliver dimensional and semantic data models for analytics. Define and maintain KPIs, conformed dimensions and consistent data definitions. Develop and optimise SQL and PySpark pipelines using Databricks. Ensure high standards of data quality, testing, performance and observability. Lead modelling reviews and contribute to technical and architectural decisions. Work closely ...

Data Science team. The Data Scientist will have strong hands-on experience writing technical code in python and have strong knowledge within PySpark optimisation. The ideal Data Scientist will be able to manage teams of Data Scientists and will be able to work closely with engineers and software developers. ...

engineering, or a related field Experience analysing gambling marketing activities The ability to write clear, well-structured code Hands‐on experience with Python, SQL, PySpark, or similar tools A flair for problem‐solving and storytelling with data — and the confidence to present findings to clients Sound like ...

facing team in London. You will mentor data scientists, oversee complex data products, and ensure analytical excellence. The ideal candidate has strong SQL and PySpark experience, excellent communication skills, and a Bachelor’s degree in a related field. This position offers a collaborative environment, competitive benefits, and a chance ...

maintain production-quality Python and SQL code.* Use Git-based source control and branch management practices across multiple deployment environments.* Work with Spark and PySpark to process and analyse large-scale datasets.* Develop and manage machine learning workflows within an MLOps environment.* Use tooling such as MLflow, VS Code … model validation.* Practical experience with Git, Agile delivery and collaborative software-development workflows.* Experience using distributed data-processing and MLOps tools such as Spark, PySpark, MLflow, JupyterHub or equivalent technologies.* Understanding of data-product lifecycles, data quality, governance, reproducibility and auditable insightful workflows.* Ability to translate operational or business ...

maintain production-quality Python and SQL code. Use Git-based source control and branch management practices across multiple deployment environments. Work with Spark and PySpark to process and analyse large-scale datasets. Develop and manage machine learning workflows within an MLOps environment. Use tooling such as MLflow, VS Code … model validation. Practical experience with Git, Agile delivery and collaborative software-development workflows. Experience using distributed data-processing and MLOps tools such as Spark, PySpark, MLflow, JupyterHub or equivalent technologies. Understanding of data-product lifecycles, data quality, governance, reproducibility and auditable insightful workflows. Ability to translate operational or business ...

improving data pipelines. Working with Databricks to improve the existing data platform. Getting hands-on with data transformation and modelling. Using Python, SQL and PySpark to deliver data solutions. Supporting the migration from QuickSight to Power BI. Working closely with teams across Finance, Pricing, Regulatory and Compliance. Helping … small team where everyone gets involved and priorities can change. Insurance experience would be useful, as would any Power BI, AWS and PySpark experience. What's on offer? £80,000–£85,000 salary. Generous bonus scheme 25 days' holiday, increasing to 30 with length of service. Hybrid working, with ...

data exploration skills Experience of working in Agile methodology Good knowledge and hands-on experience of developing Stored Procedures Experience of working with PySpark Notebooks for data ingestion and provision Practical experience of improving data quality and efficiency in data pipelines Solid understanding of DevOps and CI/… Understanding of compliance and lifecycle of data management Desirable Skills: Coding experience in any relevant language (Python, Pyspark etc.) Experience working with other Azure technologies such as Fabric, Power Automate Experience of working with non-relational databases (CosmosDB, MongoDB, etc.) and handling unstructured data in ETL pipelines Our values ...

Skills: MDM, Databricks, PySpark, SQL, Delta Lake, Unity Catalog, Data Quality, Data Governance Role Summary We are looking for an experienced MDM/Databricks Data Engineer to build scalable master data and integration solutions on a modern Databricks Lakehouse. The role will focus on common IDs, Golden Records, entity … entity matching, deduplication, and survivorship rules. Integrate data from multiple source systems into the Databricks Lakehouse . Develop and optimize data pipelines using Databricks, PySpark, SQL, and Delta Lake . Build and enhance Bronze, Silver, and Gold data layers. Implement data quality, standardization, reconciliation, and validation processes. Support Unity ...