51 to 75 of 83 PySpark Jobs in London

re used daily by home builders, mortgage brokers, local councils, and more to make informed property purchasing decisions. We've migrated key legacy SQL Server/SSIS pipelines to PySpark and Databricks, and we're in the home stretch of our modernisation programme. Now we're looking to unlock the power of our disparate data, and make it accessible … working to high standards of compliance (inc ISO-27001, GDPR), Data Governance, and Information Security Experienced in migrating from SQL based data architectures to modern Data Engineering technologies, using PySpark, Databricks, Terraform, and Pandas Someone able to explore, analyse and understand our data and its uses Ideally experienced in a multi-cloud environment (Databricks across Azure and AWS) solving More ❯

re used daily by home builders, mortgage brokers, local councils, and more to make informed property purchasing decisions. We’ve migrated key legacy SQL Server/SSIS pipelines to PySpark and Databricks, and we’re in the home stretch of our modernisation programme. Now we’re looking to unlock the power of our disparate data, and make it accessible … working to high standards of compliance (inc ISO-27001, GDPR), Data Governance, and Information Security Experienced in migrating from SQL based data architectures to modern Data Engineering technologies, using PySpark, Databricks, Terraform, and Pandas Someone able to explore, analyse and understand our data and its uses Ideally experienced in a multi-cloud environment (Databricks across Azure and AWS) solving More ❯

Align with technical leaders to shape and implement Shell's strategy What you bring Have substantial experience in technical and process guidance Experience in Python FastAPI development, Spark/pySpark, Typescript/React,T-SQL/SQL/Azure SQL and other programming frameworks and paradigm Able to mix strategic and pragmatic approaches to drive product delivery Experience in More ❯

s/PhD in Computer Science, Data Science, Mathematics, or related field. 5+ years of experience in ML modeling, ranking, or recommendation systems . Proficiency in Python, SQL, Spark, PySpark, TensorFlow . Strong knowledge of LLM algorithms and training techniques . Experience deploying models in production environments. Nice to Have: Experience in GenAI/LLMs Familiarity with distributed computing More ❯

Data Engineer | Data Consultant | Azure | Fabric | Python | SQL | PySpark Fabric Data Consultant - Up to £70,000 London - 3 days in-office Method Resourcing are thrilled to be partnering with a Microsoft Solutions Partner to support them in hiring a Data Consultant to focus on and specialise on their current and upcoming Fabric projects. This is a fantastic time to … is offering a salary of up to £70,000 dependent on experience + Bonus & Benefits. Please apply now for immediate consideration. Data Engineer | Data Consultant | Azure | Fabric | Python | SQL | PySpark Fabric Data Consultant - Up to £70,000 London - 3 days in-office RSG Plc is acting as an Employment Agency in relation to this vacancy. More ❯

of you: It's an exciting time to join Markel as our business grows. We are seeking a motivated and detail-oriented Python Developer with experience in Databricks and PySpark to join our dynamic team. What you will bring to the role: Develop and maintain Python scripts and applications using Databricks and PySpark Collaborate with cross-functional teams More ❯

communicate (with strong written and spoken English) - experience leading meetings or sessions would be ideal. An understanding of appropriate communication styles for different client seniorities. Experience with Python (and pyspark/pandas/polars), or another programming language would be beneficial. Differentiators We would particularly like to see applicants who: Can demonstrate a strong proactive attitude towards learning, developing More ❯

and value creation from data curation activities. Agile mindset with the ability to deliver prototypes quickly and iterate improvements based on stakeholder feedback Experience in Python, Databricks, Delta Lake, PySpark, Pandas, other data engineering frameworks and applying them to achieve industry standards-compliant datasets Strong communication skills and expertise to translate business needs into technical data requirements and processes More ❯

ETL) Future Talent Pool, GCP Data Engineer, London, hybrid role new workstreams on digital Google Cloud transformation programme Proficiency in programming languages such as Python and Java Programming languages Pyspark & Java develop ETL processes for Data ingestion & preparation SparkSQL CloudRun, DataFlow, CloudStorage GCP BigQuery Google Cloud Platform Data Studio Unix/Linux Platform Version control tools (Git, GitHub), automated More ❯

of ETL Processing/Data Warehousing testing; including Databricks and Data Factory. - Hands-on experience with SQL or Azure SQL. - Experience using automated testing on Python frameworks (Pystest/Pyspark) - Experience with Specflow and other frameworks. If you are interested in this Data Tester role please apply with your most recent CV alterntaviely reach out to me jordan . More ❯

of ETL Processing/Data Warehousing testing; including Databricks and Data Factory. - Hands-on experience with SQL or Azure SQL. - Experience using automated testing on Python frameworks (Pystest/Pyspark) - Experience with Specflow and other frameworks. If you are interested in this Data Tester role please apply with your most recent CV alterntaviely reach out to me jordan . More ❯

using CI/CD, along with proficiency in designing and implementing CI/CD pipelines in Cloud environments. Excellent practical expertise in Performance tuning and system optimisation. Experience with PySpark and Azure Databricks for distributed data processing and large-scale data analysis. Proven experience with web frameworks , including knowledge of Django and experience with Flask, along with a solid More ❯

Experience in Cloud Data Pipelines Building cloud data pipelines involves using Azure native programming techniques such as PySpark or Scala and Databricks. These pipelines are essential for tasks like sourcing, enriching, and maintaining structured and unstructured data sets for analysis and reporting. They are also crucial for secondary tasks such as flow pipelines, streamlining AI model performance, and enhancing More ❯

recent graduates are encouraged to apply! A degree in Mathematics, Engineering, Statistics, Computer Science, Physics, or a related field. An advanced degree is highly preferred. Proficient in Python and PySpark; experience with SQL or similar querying languages. Solid foundation in machine learning principles, including model evaluation, optimization, and deployment best practices. Self-motivated, collaborative, and adaptable, with a "can More ❯

Data Engineering Manager £110,000-£115,000 + 10% bonus ️Databricks, Snowflake, Terraform, Pyspark, Azure London, hybrid working (2 days in office) Leading property data & risk software company We are partnered with a leading property data & risk software company that contributes valuations, insights, and decisioning technology to over 1 million mortgage approvals each year. They are looking for a … visualization, and data modeling. Engage in projects that influence the company's bottom line. Drive the business forward by enabling better decision-making processes. Tech Stack : Databricks, Azure, Python, Pyspark, Terraform. What's in it for you 7.5% pension contribution by the company Discretionary annual bonus up to 10% of base salary 25 days annual leave plus extra days More ❯

Data Scientist, you will work using data engineering, statistical, and ML/AI approaches to uncover data patterns and build models. We use Microsoft tech stack, including Azure Databricks (Pyspark, python), and we are expanding our data science capabilities. To be successful in the role, you will need to have extensive experience in data science projects and have built More ❯

experience in data science, machine learning, and business analytics. Extensive experience with generative AI and machine learning frameworks (GANs, VAEs, transformer-based architectures, LLMs). Programming experience with Python, PySpark, microservices, and data modeling. Proficient in LLM orchestration on platforms such as OpenAI on Azure, AWS Bedrock, GCP Vertex AI, or Gemini AI. Architecture & Deployment: Designing, deploying, and scaling More ❯

Fabric - UK role (WFH) - 6 months initial contract - Top rates - Outside IR35 Major consultancy urgently requires a Data Engineer with experience of MS Fabric (tech stack is Microsoft Fabric, PySpark/RSpark and Github) for an initial 6 months contract (WFH and Outside IR35) who is passionate about building new capabilities from the ground up and want to help More ❯

Fabric - UK role (WFH) - 6 months initial contract - Top rates - Outside IR35 Major consultancy urgently requires a Data Engineer with experience of MS Fabric (tech stack is Microsoft Fabric, PySpark/RSpark and Github) for an initial 6 months contract (WFH and Outside IR35) who is passionate about building new capabilities from the ground up and want to help More ❯

related analytical field experience PREFERRED QUALIFICATIONS - Experience in Amazon Redshift and other AWS technologies - Experience creating complex SQL queries joining multiple datasets, ETL DW concepts - Experience in SCALA and Pyspark Our inclusive culture empowers Amazonians to deliver the best results for our customers. If you have a disability and need a workplace accommodation or adjustment during the application and More ❯

WE NEED THE PYTHON/DATA ENGINEER TO HAVE. Current DV Security Clearance (Standard or Enhanced) Experience with big data tools such as Hadoop, Cloudera or Elasticsearch Python/PySpark experience Experience With Palantir Foundry is nice to have Experience working in an Agile Scrum environment with tools such as Confluence/Jira Experience in design, development, test and More ❯

Skills & Experience:4+ years of experience in AI/ML engineering or data-intensive systems.Strong proficiency in Python for AI, ML, and data engineering tasks.Deep experience with Apache Spark (PySpark or Scala-based implementations).Solid understanding and hands-on experience with modelling intelligent agents, including symbolic, neural, or hybrid approaches.Experience deploying to and managing workloads on MCP or distributed More ❯

great if you have: Built search related products e.g. chatbots Exposure to building data products that use generative AI and LLMs Previous experience using Spark (either via Scala or Pyspark) Experience with statistical methods like regression, GLMs or experiment design and analysis, shipping productionized machine learning systems or other advanced techniques are also welcome All your information will be More ❯

and end users to define, test and deliver technical and functional requirements. - You will need experience implementing Master Data Management programmes, this is mandatory for the role. - SQL and PySpark knowledge - Azure Databricks experience. - Experience with Data Querying, and Data Profiling. - Experience working with Large data sets especially analysing and cleansing said data. - Strong Communication Skills. - Experience working within More ❯

and end users to define, test and deliver technical and functional requirements. - You will need experience implementing Master Data Management programmes, this is mandatory for the role. - SQL and PySpark knowledge - Azure Databricks experience. - Experience with Data Querying, and Data Profiling. - Experience working with Large data sets especially analysing and cleansing said data. - Strong Communication Skills. - Experience working within More ❯