76 to 100 of 100 PySpark Jobs in England

Databricks, APIs, and integration platforms Strong understanding of MLOps and productionizing machine learning Experience leading and scaling engineering teams Tech stack: Azure, Databricks, Python, PySpark, SQL, JavaScript, Node, Bash THE ROLE: You’ll own the technical vision and delivery of our core platforms: Build world‐class platforms Architect cloud … technical leadership where required Define and own the engineering roadmap and align to resource needs Work with: Azure, Databricks, Bash, JavaScript, Node, Python, PySpark, SQL Integration & Ecosystem Enablement Own the strategy and delivery of integration services, including public and private APIs, event‐driven and asynchronous integrations, and authentication/ ...

Job Title: Technical Delivery Lead - Data Engineering Lead (Hands-on Python/Databricks) Contract: 6 months - Potentially extension. Location: London Hybrid (5 days in 10 in the office) Rate: Highly competitive Status: Inside IR35 Working ...

Azure platform — including infrastructure-as-code (Pulumi), CI/CD automation, monitoring, security, and Databricks workspace administration. The ideal candidate combines strong Python/PySpark engineering skills with deep Azure platform knowledge and a service-excellence mindset. Duties and Responsibilities Lead solution design activities, collaborating with peers and mentoring … Databricks: hands-on experience building and optimizing pipelines, managing Delta Lake, and administering workspaces (cluster policies, Unity Catalog, Secret Scopes, Workflows). Python/PySpark: strong programming skills for data processing, automation, and scripting. Azure data stack: Synapse, Databricks, ADLS Gen2, Key Vault — including Linked Services, Managed Identity ...

Senior Data Engineer, Lead Data Engineer, Data Engineer, ETL, AWS, AWS Glue, AWS Lambda, AWS S3, AWS Redshift, Amazon EMR, Python, SQL, Pipeline Orchestration, PySpark, data warehousing, data lakehouse, data architecture, Agile, Senior Data Engineer, Lead Data Engineer, Data Engineer) Our client is a global innovator and world leader … with AWS services such as AWS Glue, AWS Lambda, AWS S3, AWS Redshift, and Amazon EMR. Proven expertise in Python, SQL, Pipeline Orchestration and PySpark is also expected, as is experience with data lakehouse architectures and data warehousing solutions. Agile experience is also key, as are excellent communication skills ...

contribute to building scalable data pipelines and improving engineering standards across the platform. Responsibilities include: Building and maintaining robust data pipelines using Python and PySpark Supporting CI/CD processes and applying DevOps best practices Contributing to code deployment, testing, and release processes Collaborating with engineers, QA specialists … exposure to data modelling and machine learning workflows over time Your Skills & Experience Hands-on experience with Python and data engineering concepts Exposure to PySpark and large-scale data processing Familiarity with Databricks or similar modern data platforms Understanding of software engineering principles and version control Interest ...

Data Architect (Databricks or Microsoft Fabric) £110,000–£125,000 Hybrid – 2 days a week in London, Manchester, or Edinburgh We’re looking for a twoexperienced Data Architects to design and deliver modern, scalable data ...

Data Engineering Lead Certain Advantage are hiring for a Data Engineering Lead based in London. This role is on an initial contract till the end of the year and is required to be onsite 3 ...

salary of up to £120,000. The successful candidate will engage with clients, focusing on architecture and data solutions while leveraging expertise in Spark, PySpark, and Delta Lake. You'll lead workshops, translate complex requirements, and contribute to modernizing enterprise data platforms. #J-18808-Ljbffr ...

maintain ETL/ELT pipelines and data models in Fabric (Data Factory, Notebooks, Spark) Write high-performance Spark SQL, T-SQL, Python/PySpark Manage ingestion, transformation, and loading from multiple sources Translate stakeholder requirements into scalable technical solutions Mentor team members and establish engineering standards, security, and governance … assisted development tools like GitHub Copilot, ChatGPT, and Fabric Copilot Essential Experience: Microsoft Fabric is essential Azure Data ecosystem Lakehouse architectures & Data Factory Python, PySpark, Spark SQL Proven hands-on delivery in this stack If you're an experienced Data Engineer with strong Microsoft Fabric and Azure experience, this ...

join a leading organisation on a contract basis. This role will suit someone with strong experience across the Azure data platform , particularly Databricks and PySpark , who can take ownership of data engineering delivery from design through to implementation. This is a hybrid role requiring the candidate … data processing in a cloud environment. Key Responsibilities Design and build scalable data pipelines using Azure technologies Develop and optimise solutions in Databricks using PySpark Work with large and complex datasets, including streaming and batch data Build and maintain data platforms and data warehouses Implement ETL/ELT processes ...

dashboard content Identify and mitigate data quality risks and ensure data integrity Design, build, and maintain reusable data pipelines and datasets using Python and PySpark Develop and test datasets used in BI dashboards and reporting products Work with stakeholders to translate business needs into data solutions and insights Collaborate … DevSecOps Engage in pair programming and adhere to collaborative coding standards Essential Skills & Experience Strong hands-on experience with Python and PySpark Solid background in exploratory data analysis and data modelling Experience with Palantir platforms, ideally including the NHS Federated Data Platform (FDP) Proficiency with BI and data platforms ...

Lakes*** Experience with **Apache Airflow/MWAA** for workflow orchestration* Strong knowledge of **dbt (data build tool)** for data transformation* Experience with **EMR/PySpark** for large-scale data processing* Ability to create **Architecture Design Documents (ADDs)*** Deep understanding of **cloud-native design patterns*** Experience in **performance tuning** across ...

with Infrastructure as Code (Terraform) Experience working with GitLab and implementing CI/CD pipelines Practical experience building data pipelines in AWS (e.g. EMR, PySpark) Solid understanding of ETL tooling (e.g. Informatica or similar) Strong background in data processing and transformation Proficiency in Python and PySpark Essential Requirements ...

Technical Skills You'll have experience with some of the following: Cloud & Data Platforms: AWS, Azure, Google Cloud Platform, Databricks Programming Languages: Python, R, PySpark Agentic & GenAI Platforms: Microsoft Copilot Studio, OpenAI GPT-5 Agents, UiPath, Adept AI, Orby AI, Beam AI If you've held any of these … Specialist, Python Developer, AI Transformation Consultant, Databricks Consultant, Azure AI Consultant, AWS Data Scientist, GCP Data Consultant, Agentic AI Consultant, OpenAI GPT Specialist, PySpark Developer. Deerfoot Recruitment Solutions Ltd is a leading independent tech recruitment consultancy in the UK. For every CV sent to clients, we donate ...

Skills You’ll need to have experience with some of the following and be able to learn new technologies or concepts quickly: Python/PySpark - Experience using Python and PySpark to connect to APIs and extract data, with strong API design skills and familiarity with LLM orchestration frameworks ...

reporting perspective *Design and build data pipelines and datasets for dashboards *Identify and mitigate data quality risks *Develop reusable code in Python and PySpark for data prep and analysis *Collaborate closely with Data Engineers, Analysts and UCD professionals *Work within modern engineering practices including CI/… strong engineering culture What I need from you (essential) To be considered, your CV must clearly show experience with the following: *Palantir (FDP) *Python *PySpark *Exploratory Data Analysis *Data pipeline design/data engineering concepts *Dashboard/reporting dataset development *BI tools (Redshift, Databricks or similar) *Agile delivery environments ...

challenges Communicate findings and recommendations to stakeholders in a clear and structured manner Learning & Professional Development Build practical expertise in React, TypeScript, Python, and PySpark within a Palantir Foundry environment Apply and develop best practices in data engineering, visualisation, and business intelligence Engage with mentorship and guidance from senior … appreciation for how data can be presented in ways that are both intuitive and impactful Working knowledge of SQL, with exposure to Python or PySpark desirable but not essential Strong analytical and problem-solving skills, with a passion for extracting meaning from complex datasets Effective communication skills, both written ...

looks like from a reporting perspectiveDesign and build data pipelines and datasets for dashboardsIdentify and mitigate data quality risksDevelop reusable code in Python and PySpark for data prep and analysisCollaborate closely with Data Engineers, Analysts and UCD professionalsWork within modern engineering practices including CI/CD, TDD and DevSecOpsPair …/BA-style roleNot a role where you'll just "pick up tickets" with no ownershipNot suitable if your experience is light on Python, PySpark or PalantirKey Details£560 per day (Inside IR35)Start ASAPRunning until November 2026 (extensions very likely)Remote with occasional travel to LondonMust ...

digital Screening service. You will build and maintain data pipelines that ingest, transform and prepare data for reporting, developing ETL processes using PySpark and platform tooling. You will work within an Agile multidisciplinary team alongside Data Scientists, Analysts and UCD professionals. This is a largely remote role. Essential skills … experience Strong experience in data pipeline engineering, ETL development and PySpark Experience with Palantir products, including the NHS Federated Data Platform (FDP) Ability to ensure data quality, reliability and performance across reporting pipelines Experience supporting testing and validation of dashboards and underlying datasets Proven ability to work independently ...

Robert Half Technology are assisting UK's market leading legal organisation to recruit a Data Engineer on a 12 month contract basis - Hybrid working - London based. A Data Engineer is required to join a growing ...

development. Key Responsibilities Design and develop end-to-end data pipelines using reusable frameworks Build scalable data solutions using Python, SQL, and PySpark Create modular frameworks separating configuration from orchestration Implement robust automated testing (unit, integration, regression) Establish and enforce data quality controls (schema, nulls, row counts, integrity checks … based workflows Troubleshoot and optimise data pipelines for reliability and performance Required Experience Strong Data Engineering background Advanced Python and SQL skills Experience with PySpark or distributed processing tools Proven track record building reusable data frameworks Strong testing experience (pytest, mock data, data validation techniques) Solid understanding of data ...

models. This isn't just a 'high-level' role-we need someone who can design the blueprint and then dive into the Python/PySpark code to ensure the foundation is rock solid. Key Responsibilities: Architect & Design: Lead the implementation of on-premises, cloud, and hybrid AI solutions. GenAI … certification. AI Specialization: Proven experience with Prompt Engineering, Azure OpenAI, Document Intelligence (Form Recognizer), and Cognitive Search. Data Engineering: Expert proficiency in Python and PySpark for large-scale data processing. Operations: Familiarity with MLOps or LLMOps frameworks to support production-grade AI. Seniority: 15+ years of total IT experience ...

insurers. This is a hands-on role focused on building and enhancing a Databricks -based data engineering framework. You will design and develop scalable PySpark pipelines covering data ingestion, validation, transformation, and mapping, while creating reusable libraries and ensuring high engineering standards. You will also provide technical leadership, conducting … code reviews, guiding offshore teams, and ensuring delivery quality aligns with architectural and regulatory requirements. Key Requirements: Strong experience with Databricks and PySpark Solid Azure data platform experience Proven ability to build reusable frameworks and libraries Experience leading code quality and working with offshore teams Desirable: Experience in regulatory ...

Amex GBT is a place where colleagues find inspiration in travel as a force for good and – through their work – can make an impact on our industry. We’re here to help our colleagues achieve ...

JPMorgan Chase & Co. is seeking a Site Reliability Engineer for their AI/ML Data Platforms team in Greater London. In this role, you will develop scalable and resilient data solutions, manage incident resolutions, and ...