101 to 125 of 141 PySpark Jobs in the UK

ACCA/CIMA Qualified Experience in a Systems Accountant role Experience in using PowerBI/Microsoft Fabric an advantage Experience in using SQL/PySpark beneficial Strong Excel skills Driven to improve processes and clarity of outputs Strong communicator and able to engage with key internal and external stakeholders ...

Collaborative office environment with regular social and charity events And Much More!!! DATA ENGINEER - ESSTENTIAL SKILLS Strong experience with Databricks and Apache Spark (PySpark or Scala) Experience building streaming and batch pipelines using Spark Structured Streaming Hands-on experience with Kafka (MSK) and real-time data ingestion Strong understanding … submit (subject to required skills) your application to our client in conjunction with this vacancy only. KEY SKILLS Data Engineering, Databricks, Apache Spark, PySpark, SQL, Python, Kafka, AWS, Delta Lake, ETL, Data Pipelines, CI/CD, Big Data ...

Clearance to be considered for this role. In order to be successful, you will have the following experience: Experienced in the Python data stack (PySpark, numpy, pandas, scikit-learn, etc.) Experienced within AWS-based infrastructure and ser... LFWQ1_UKTJ ...

GBMs, GAMs, Random Forests, Neural Networks and clustering techniques Knowledge of statistics and distributions commonly used in insurance Experience in programming languages (e.g. Python, PySpark, R, SAS, SQL) Proficient at communicating results in a concise manner both verbally and written Behaviours: Motivated by technical excellence Driven to deliver iterative ...

Databricks, APIs, and integration platforms Strong understanding of MLOps and productionizing machine learning Experience leading and scaling engineering teams Tech stack: Azure, Databricks, Python, PySpark, SQL, JavaScript, Node, Bash THE ROLE: You’ll own the technical vision and delivery of our core platforms: Build world‐class platforms Architect cloud … technical leadership where required Define and own the engineering roadmap and align to resource needs Work with: Azure, Databricks, Bash, JavaScript, Node, Python, PySpark, SQL Integration & Ecosystem Enablement Own the strategy and delivery of integration services, including public and private APIs, event‐driven and asynchronous integrations, and authentication/ ...

work on large-scale government projects while providing technical leadership and mentoring within Agile delivery teams. Key Responsibilities: Design and develop data pipelines using PySpark, Python, SQL, Informatica , and AWS services including EMR, S3, and Lambda . Lead the migration of legacy ETL and reporting solutions to modern cloud … stakeholders across product, data, and platform teams to deliver robust data solutions. Essential Skills & Experience: Active SC Clearance Hands-on expertise with: Informatica, AWS, PySpark and Python Strong Agile delivery experience using Jira and Confluence. Excellent communication and stakeholder management skills. £435.00/day Talent International ...

internal teams. Ideal candidates will have strong programming skills in Python, a solid background in databases, and excellent collaboration abilities. Experience with AWS, PySpark, or Terraform is advantageous. We promote an inclusive work environment and provide accommodations for individuals with disabilities. #J-18808-Ljbffr ...

Job Title: Technical Delivery Lead - Data Engineering Lead (Hands-on Python/Databricks) Contract: 6 months - Potentially extension. Location: London Hybrid (5 days in 10 in the office) Rate: Highly competitive Status: Inside IR35 Working ...

Azure platform — including infrastructure-as-code (Pulumi), CI/CD automation, monitoring, security, and Databricks workspace administration. The ideal candidate combines strong Python/PySpark engineering skills with deep Azure platform knowledge and a service-excellence mindset. Duties and Responsibilities Lead solution design activities, collaborating with peers and mentoring … Databricks: hands-on experience building and optimizing pipelines, managing Delta Lake, and administering workspaces (cluster policies, Unity Catalog, Secret Scopes, Workflows). Python/PySpark: strong programming skills for data processing, automation, and scripting. Azure data stack: Synapse, Databricks, ADLS Gen2, Key Vault — including Linked Services, Managed Identity ...

within Azure cloud environments. Proven hands-on expertise with Azure Databricks in enterprise-scale production environments. Strong programming skills using: Python/SQL/PySpark/Spark SQL. Experience designing and building scalable data pipelines and data processing frameworks. Strong understanding of modern Lakehouse architecture principles. Experience working with … technical teams to understand data requirements and translate them into scalable engineering solutions. Design, develop and maintain high-performance data pipelines using Azure Databricks, PySpark, Spark SQL and Python. Contribute to the design and implementation of Lakehouse architecture using Delta Lake and Databricks best practices. Develop and manage data ...

Senior Data Engineer, Lead Data Engineer, Data Engineer, ETL, AWS, AWS Glue, AWS Lambda, AWS S3, AWS Redshift, Amazon EMR, Python, SQL, Pipeline Orchestration, PySpark, data warehousing, data lakehouse, data architecture, Agile, Senior Data Engineer, Lead Data Engineer, Data Engineer) Our client is a global innovator and world leader … with AWS services such as AWS Glue, AWS Lambda, AWS S3, AWS Redshift, and Amazon EMR. Proven expertise in Python, SQL, Pipeline Orchestration and PySpark is also expected, as is experience with data lakehouse architectures and data warehousing solutions. Agile experience is also key, as are excellent communication skills ...

contribute to building scalable data pipelines and improving engineering standards across the platform. Responsibilities include: Building and maintaining robust data pipelines using Python and PySpark Supporting CI/CD processes and applying DevOps best practices Contributing to code deployment, testing, and release processes Collaborating with engineers, QA specialists … exposure to data modelling and machine learning workflows over time Your Skills & Experience Hands-on experience with Python and data engineering concepts Exposure to PySpark and large-scale data processing Familiarity with Databricks or similar modern data platforms Understanding of software engineering principles and version control Interest ...

experience delivering AI, Machine Learning or Generative AI solutions Strong understanding of Large Language Models (LLMs) and modern AI architectures Python and/or PySpark experience Experience working with cloud platforms (Azure preferred but AWS and GCP backgrounds welcomed) Knowledge of Databricks, Microsoft Fabric or modern data platforms Strong … skills) your application to our client in conjunction with this vacancy only. KEY SKILLS Artificial Intelligence, Generative AI, LLMs, Azure, Databricks, Microsoft Fabric, Python, PySpark, AI Consulting, Machine Learning, Data Engineering, Stakeholder Management, Solution Architecture, Microsoft Technologies ...

Data Architect (Databricks or Microsoft Fabric) £110,000–£125,000 Hybrid – 2 days a week in London, Manchester, or Edinburgh We’re looking for a twoexperienced Data Architects to design and deliver modern, scalable data ...

Data Engineering Lead Certain Advantage are hiring for a Data Engineering Lead based in London. This role is on an initial contract till the end of the year and is required to be onsite 3 ...

salary of up to £120,000. The successful candidate will engage with clients, focusing on architecture and data solutions while leveraging expertise in Spark, PySpark, and Delta Lake. You'll lead workshops, translate complex requirements, and contribute to modernizing enterprise data platforms. #J-18808-Ljbffr ...

requirements into technical solutions and actionable data initiatives Provide hands-on technical leadership across a Microsoft data stack, leveraging tools such as Databricks and PySpark Oversee end-to-end delivery of data solutions, ensuring designs are effectively implemented by engineering teams If this role sounds of interest ...

maintain ETL/ELT pipelines and data models in Fabric (Data Factory, Notebooks, Spark) Write high-performance Spark SQL, T-SQL, Python/PySpark Manage ingestion, transformation, and loading from multiple sources Translate stakeholder requirements into scalable technical solutions Mentor team members and establish engineering standards, security, and governance … assisted development tools like GitHub Copilot, ChatGPT, and Fabric Copilot Essential Experience: Microsoft Fabric is essential Azure Data ecosystem Lakehouse architectures & Data Factory Python, PySpark, Spark SQL Proven hands-on delivery in this stack If you're an experienced Data Engineer with strong Microsoft Fabric and Azure experience, this ...

Level Required in Years: 10 - 15 years Mandatory Required Skills Hands-on experience with Quantexa platform/Decision Intelligence Strong in Apache Spark/PySpark Proficiency in Python/Scala Experience in ETL, Data Engineering, Data Warehousing Knowledge of entity resolution, graph analytics, link analysis Experience working with large … share the Detailed Job Description Design and develop solutions using Quantexa Decision Intelligence Platform Build data ingestion, transformation, and ETL pipelines using Spark/PySpark Work on entity resolution, network analytics, and graph-based data models Integrate multiple structured/unstructured data sources for analytics use cases Ensure data ...

modernisation of existing data estates. 🔧 Key Responsibilities Design and build scalable data pipelines using Databricks Develop and optimise ETL/ELT workflows using Spark (PySpark/SQL) Support migration and modernisation of legacy data platforms into Databricks Improve performance, reliability and cost efficiency across data workloads Work closely with … best practices across delivery ✅ What We’re Looking For Strong hands-on experience with Databricks in a Data Engineering capacity Solid experience with Spark (PySpark/Spark SQL) Proven track record building and optimising data pipelines (ETL/ELT) Experience working within cloud environments (AWS or Azure) Strong ...

dashboard content Identify and mitigate data quality risks and ensure data integrity Design, build, and maintain reusable data pipelines and datasets using Python and PySpark Develop and test datasets used in BI dashboards and reporting products Work with stakeholders to translate business needs into data solutions and insights Collaborate … DevSecOps Engage in pair programming and adhere to collaborative coding standards Essential Skills & Experience Strong hands-on experience with Python and PySpark Solid background in exploratory data analysis and data modelling Experience with Palantir platforms, ideally including the NHS Federated Data Platform (FDP) Proficiency with BI and data platforms ...

Lakes*** Experience with **Apache Airflow/MWAA** for workflow orchestration* Strong knowledge of **dbt (data build tool)** for data transformation* Experience with **EMR/PySpark** for large-scale data processing* Ability to create **Architecture Design Documents (ADDs)*** Deep understanding of **cloud-native design patterns*** Experience in **performance tuning** across ...

Technical Skills You'll have experience with some of the following: Cloud & Data Platforms: AWS, Azure, Google Cloud Platform, Databricks Programming Languages: Python, R, PySpark Agentic & GenAI Platforms: Microsoft Copilot Studio, OpenAI GPT-5 Agents, UiPath, Adept AI, Orby AI, Beam AI If you've held any of these … Specialist, Python Developer, AI Transformation Consultant, Databricks Consultant, Azure AI Consultant, AWS Data Scientist, GCP Data Consultant, Agentic AI Consultant, OpenAI GPT Specialist, PySpark Developer. Deerfoot Recruitment Solutions Ltd is a leading independent tech recruitment consultancy in the UK. For every CV sent to clients, we donate ...

New Permanent Opportunity for an eDV Cleared Data Engineer with AWS Cloud Experience for a leading National Security Consultancy in London. Up to £85,000 DoE plus bonuses Active eDV required London Location – full-time ...

Skills You’ll need to have experience with some of the following and be able to learn new technologies or concepts quickly: Python/PySpark - Experience using Python and PySpark to connect to APIs and extract data, with strong API design skills and familiarity with LLM orchestration frameworks ...