51 to 60 of 60 PySpark Jobs in London

Azure Developing and deploying the Lakehouse platform using Databricks Building automation for cluster management, integration runtimes, and infrastructure components Leading implementation with Databricks, PySpark, Spark SQL, and Azure Data Factory Creating APIs for integration and automation Driving CI/CD, automated testing, and DevOps best practice Your Skills … Experience You will have strong capability in: Azure and Databricks engineering PySpark, SQL, Python, and Spark SQL DevOps best practices including CI/CD, automation, and testing Infrastructure as code such as Terraform Stakeholder communication and cross-team collaboration How to Apply If you are a strong Data Platform ...

data processing solutions Work with modern data platforms such as Microsoft Fabric, Databricks, or similar technologies Build and optimise data solutions using Python, Spark, PySpark, and/or SQL Support data integration, transformation, and performance optimisation Collaborate with stakeholders to ensure high-quality, reliable data architecture Essential Skills & Experience … Hands-on experience with Microsoft Fabric, Databricks, or equivalent modern data platforms Strong coding experience in Python and/or Spark/PySpark Solid SQL skills and experience working with structured datasets Desirable Experience Knowledge of relational database design , normalisation , and data modelling Experience designing efficient, scalable data architectures ...

transform them into robust, production-grade code libraries that other teams can actually use. Build Data Pipelines: Design and deploy scalable pipelines using PySpark to handle massive datasets for experiment execution. Collaborate: Work cross-functionally with Data Scientists and Software Engineers to turn complex requirements into seamless deployments. What … Looking for: You have a degree in CS, Math, or Engineering, Data Science3 Expert with Python, SQL, and PySpark . 3+ years in a similar role with a globally recognised large brand Production Experience: You’ve built and shipped scalable systems before (ideally with ML/AI components) DevOps ...

Data Engineer London, £550 per day Outside IR35, Hybrid This is an exciting opportunity to play a key role in a major data modernisation programme, focused on migrating a large SQL Server estate into a ...

Role: Join our client as a Lead PySpark Engineer to modernize legacy data workflows. You will lead the hands-on transition of complex SAS environments into scalable, AWS-native PySpark pipelines for a major financial services project. What You'll Do Modernize: Convert legacy SAS code into production … ready PySpark using automated tools and manual refactoring. Build: Design and fix high-performance ETL/ELT pipelines and data marts on AWS. Your Technical Profile PySpark (Expert): 5+ years of hands-on experience. Deep knowledge of Spark optimization (partitioning, tuning) and Data Warehousing (SCDs, Facts/Dimensions ...

Data Testing Analyst - Databricks - Pyspark - SQL - Python - Lloyds Markets - London (Hybrid 3 days onsite) I have an urgent requirement for an experienced Data Tester with background working on Lloyds Markets/Lloyds Insurance Domain data flows. The role will at first be until the end of year, working … Data Tester on Lloyds Market data flows Excellent manual testing experience to provide end-to-end data testing Advanced skills in SQL, Databricks, Pyspark, Spark SQL and scripting skills in Python preferred Experience with Azure DevOps and Delta Lake concepts would also be an added advantage Interested? Apply ...

requiring strong asynchronous communication skills and the ability to understand requirements with limited oversight. The role involves designing and building reusable Python libraries using PySpark/DataFrames, developing Python-based Azure Functions for microservices, and creating shared tooling to accelerate delivery across multiple teams. After completing the initial predefined … libraries that support common business needs in a fast-paced, high-ownership environment. Required Skills and Experience: Strong proficiency in Python. Experience with PySpark and DataFrame-based data processing. Solid understanding of SQL. Hands-on experience with Azure infrastructure. Good understanding of Containers, Microservices, and Functional design patterns. Experience ...

TensorFlow Serving, TorchServe, ONNX, and TensorRT. Build and manage ML pipelines using MLflow, Kubeflow, and Azure ML Pipelines. Work with large-scale data using PySpark and integrate ML solutions into production environments. Monitor and improve model performance to ensure accuracy and efficiency. Contribute to team knowledge by mentoring … serving tools (ONNX, TensorRT, TensorFlow Serving, TorchServe). Experience with ML lifecycle tools (MLflow, Kubeflow, Azure ML Pipelines). Knowledge of distributed data processing (PySpark) and software engineering principles (Git). A collaborative mindset and excellent problem-solving abilities. Experience in data cleansing, exploratory data analysis, and visualisation. ...

Lead Data Engineer - Pyspark/AWS/Python/SAS - Financial Sector As a Lead PySpark Engineer, you will design, develop, and fix complex data processing solutions using PySpark on AWS. You will work hands-on with code, modernising legacy data workflows and supporting large-scale … PySpark migrations click apply for full job details ...

Lead PySpark Engineer (Cloud Migration) Role Type: 5-Month Contract Location: Remote (UK-Based) Experience Level: Lead/Senior (5+ years PySpark) Role Overview We are seeking a Lead PySpark Engineer to drive a large-scale data modernisation project, transitioning legacy data workflows into a high-performance ...