101 to 125 of 127 PySpark Jobs in the UK

Job Advertisement: Software Developer/Senior Data Engineer Location: London Contract Length: 12 Months Daily Rate: £700 (Umbrella) Are you ready to be part of a transformative journey? Our client is embarking on a major ...

salary of up to £120,000. The successful candidate will engage with clients, focusing on architecture and data solutions while leveraging expertise in Spark, PySpark, and Delta Lake. You'll lead workshops, translate complex requirements, and contribute to modernizing enterprise data platforms. #J-18808-Ljbffr ...

Significant personal lines pricing experience • Strong portfolio-management and commercial-pricing knowledge • Experience leading complex projects and influencing senior stakeholders • Advanced R, Python, PySpark, SAS or SQL capability • Experience using WTW Radar or Emblem • Knowledge of predictive modelling, including GLMs, GBMs, GAMs and other machine-learning approaches • The ability ...

existing pipelines. Working within a growing data team, you'll design scalable data models, build robust ELT pipelines in Databricks using SQL, Python and PySpark, and integrate data from multiple internal and third-party systems. You'll collaborate closely with Analytics Engineers to ensure data is reliable, well-governed … architecture and help define long-term data standards, rather than simply delivering against predefined requirements. You'll need experience with: Databricks, SQL, Python and PySpark Building and optimising ELT/data pipelines Data modelling, lakehouse architecture and modern data platforms Azure data services API and third-party system integration ...

Engineer (AWS) - Manchester (Data Engineer, Senior Data Engineer, ETL, AWS, AWS Glue, AWS Lambda, AWS S3, AWS Redshift, Amazon EMR, Python, SQL, Pipeline Orchestration, PySpark, data warehousing, data lakehouse, data architecture, Agile, Senior Data Engineer, Data Engineer) Our client is a global innovator and world leader with … with AWS services such as AWS Glue, AWS Lambda, AWS S3, AWS Redshift, and Amazon EMR. Proven expertise in Python, SQL, Pipeline Orchestration and PySpark is also expected, as is experience with data lakehouse architectures and data warehousing solutions. Agile experience is also key, as are excellent communication skills ...

Senior Databricks Architect Technology: Databricks (Spark, PySpark/SQL) and data lake architectures. Location: London, UK. Compensation: Competitive (including bonus). Job Description We are looking for a Data Architect to design, build, and operate scalable data pipelines and data products, enabling a modern data mesh architecture. The role … integrating legacy systems with cloud-native platforms to support analytics, reporting, and domain-owned data products. Primary Skills Strong experience with Databricks (Spark, PySpark/SQL) and data lake architectures. Hands‐on experience with streaming technologies, especially Kafka/Confluent Kafka. Experience integrating legacy data sources such ...

days holiday, pension, healthCareer Progression + Modern Tech StackPyspark | ML | AI | Databricks | Kafka | AWSAre you a Data Engineer with strong Python, SQL and PySpark skills who is looking for the opportunity to step up and accelerate your career We're partnering with a market-leading technology business that … Build and optimise large-scale data pipelines Work with billions of data points generated from IoT devices Develop solutions using Python, SQL and PySpark Gain exposure to Databricks, Kafka and streaming technologies Contribute to Machine Learning and AI initiatives Work alongside Data Scientists and Software Engineers Influence architecture ...

Senior Databricks Architect - London (Hybrid) - Databricks - Spark - PySpark - AWS - Up to £115,000 + Bonus Our client, a large-scale organisation undergoing significant investment in data and cloud technologies, is seeking an experienced Senior Databricks Architect to join its expanding Data & AI practice. This is an exciting opportunity … time data pipelines from legacy and modern source systems Implementing real-time and event-driven data processing solutions Developing data transformation frameworks using Spark, PySpark and SQL Designing Lakehouse architectures using Databricks and Medallion Architecture principles Creating scalable, reusable data products to support modern data operating models Integrating data ...

**Take the lead on meaningful, high-impact projects as a Senior Data Engineer.** We are seeking a Senior Data Engineer to join a growing organisation based in London, offering a hybrid work model. This role ...

Principal Consultant - Lead Data Analyst/Data Modeller Company: Transform Together Contract: 6-month Fixed Term Contract, with option to extend or become a permanent member of the team Compensation: OTE £90,000 Location: Hybrid ...

This job is with Capgemini, an inclusive employer and a member of myGwork – the largest global platform for the LGBTQ+ business community. Please do not contact the recruiter directly. CAPGEMINI INVENT At Capgemini Invent, we ...

adoption of modern data mesh principles across complex enterprise environments. What You'll Be Doing Architect and deliver modern data solutions using Databricks, Spark, PySpark and SQL Design scalable batch and real-time data pipelines that ingest data from legacy and modern platforms Lead the development of cloud-based … making Provide technical leadership and architectural direction on large-scale transformation programmes Essential Experience Strong hands-on expertise with Databricks Extensive experience using Spark, PySpark and SQL Proven track record designing and implementing data lake architectures Experience building both batch and streaming data pipelines Strong knowledge of Kafka Good ...

data processing solutions Work with modern data platforms such as Microsoft Fabric, Databricks, or similar technologies Build and optimise data solutions using Python, Spark, PySpark, and/or SQL Support data integration, transformation, and performance optimisation Collaborate with stakeholders to ensure high-quality, reliable data architecture Essential Skills & Experience … Hands-on experience with Microsoft Fabric, Databricks, or equivalent modern data platforms Strong coding experience in Python and/or Spark/PySpark Solid SQL skills and experience working with structured datasets Desirable Experience Knowledge of relational database design , normalisation , and data modelling Experience designing efficient, scalable data architectures ...

Staff Data Engineer | AWS | PySpark | Data Modernisation | London | Contract | Inside IR35 We're supporting a global, technology‐led organisation undertaking a large‐scale cloud data transformation programme. This is not a traditional data engineering role. The organisation is modernising hundreds of complex data pipelines, establishing reusable engineering patterns … culture. Responsibilities Defining technical standards and engineering patterns for a large‐scale data modernisation programme Re‐engineering complex legacy data pipelines into production‐grade PySpark Building scalable, reusable data movement frameworks across multiple engineering teams Driving best practice around AWS Glue, Apache Spark and Apache Airflow Designing high‐quality ...

Staff Data Engineer | AWS | PySpark | Data Modernisation | London | Contract | Inside IR35 We're supporting a global, technology-led organisation undertaking a large-scale cloud data transformation programme. This isn't a traditional Data Engineering role. The organisation is modernising hundreds of complex data pipelines, establishing reusable engineering patterns … responsible for Defining technical standards and engineering patterns for a large-scale Data Modernisation programme Re-engineering complex legacy data pipelines into production-grade PySpark Building scalable, reusable data movement frameworks across multiple engineering teams Driving best practice around AWS Glue, Apache Spark and Apache Airflow Designing high-quality ...

within the Lloyd’s marketplace Good understanding of tools and platforms used across the Lloyd’s insurance ecosystem Architect and optimize data solutions using PySpark and Datamart (GOS – Oracle as a Service). Develop and maintain data platforms leveraging Snowflake and Azure services (Blob Storage, Data Lake, Synapse Analytics … standards and best practices. Requirements: Proven experience in data engineering and architecture with strong expertise in ETL/ELT processes. Hands-on experience with PySpark, Snowflake, and Oracle-based data solutions. Proficiency in Azure ecosystem: Blob Storage, Data Lake, Synapse Analytics, Databricks, Functions, and Monitor. Strong understanding of data ...

required for the domain's data products, using the tools and guardrails provided by the Enablement team. Expected Skills : Advanced coding in Python and PySpark, strong SQL, expertise in building declarative pipelines (eg; Delta Live Tables), version control (Git), automated testing (eg; Pytest), and CI/CD deployment practices … automate data pipelines to ingest, transform, and model data specific to the squad's use cases. Write structured, scalable code (eg; Python/PySpark/SQL) and apply software engineering best practices, including version control (Git) and automated testing. Work closely with the Platform Enablement team to package their ...

Data Scientist (Python, PySpark, Healthcare Analytics) Contract: Initial 2 Months (Mid-September – Mid-November) Location: Remote (UK) with 1 day per week in London (2–3 days preferred) We are seeking an experienced Data Scientist to join a high-profile digital transformation programme focused on building innovative products … Work with large and complex datasets to generate actionable insights. Required Skills Strong commercial experience as a Data Scientist. Expert knowledge of Python and PySpark (essential). Experience designing and building analytics/data pipelines. Strong SQL skills and experience working with large datasets. Experience developing and deploying machine ...

Logging, tracking and managing defects through to resolution Contributing to testing strategies, quality frameworks and continuous improvement initiatives Supporting testing activities within Databricks and PySpark environments SKILLS AND EXPERIENCE The successful Data QA Engineer will have the following skills and experience: Experience in a Data QA, Software Testing … Quality Assurance role Strong experience testing data platforms, pipelines and data products Hands-on experience with Databricks, SQL, Python and PySpark Strong stakeholder communication and collaboration skills Able to work independently and take ownership of testing activities and quality standards BENEFITS The successful Data QA Engineer will receive ...

Logging, tracking and managing defects through to resolution Contributing to testing strategies, quality frameworks and continuous improvement initiatives Supporting testing activities within Databricks and PySpark environments SKILLS AND EXPERIENCE The successful Data QA Engineer will have the following skills and experience: Experience in a Data QA, Software Testing … Quality Assurance role Strong experience testing data platforms, pipelines and data products Hands-on experience with Databricks, SQL, Python and PySpark Strong stakeholder communication and collaboration skills Able to work independently and take ownership of testing activities and quality standards BENEFITS The successful Data QA Engineer will receive ...

Logging, tracking and managing defects through to resolution Contributing to testing strategies, quality frameworks and continuous improvement initiatives Supporting testing activities within Databricks and PySpark environments SKILLS AND EXPERIENCE The successful Data QA Engineer will have the following skills and experience: Experience in a Data QA, Software Testing … Quality Assurance role Strong experience testing data platforms, pipelines and data products Hands-on experience with Databricks, SQL, Python and PySpark Strong stakeholder communication and collaboration skills Able to work independently and take ownership of testing activities and quality standards BENEFITS The successful Data QA Engineer will receive ...

Logging, tracking and managing defects through to resolution Contributing to testing strategies, quality frameworks and continuous improvement initiatives Supporting testing activities within Databricks and PySpark environments SKILLS AND EXPERIENCE The successful Data QA Engineer will have the following skills and experience: Experience in a Data QA, Software Testing … Quality Assurance role Strong experience testing data platforms, pipelines and data products Hands-on experience with Databricks, SQL, Python and PySpark Strong stakeholder communication and collaboration skills Able to work independently and take ownership of testing activities and quality standards BENEFITS The successful Data QA Engineer will receive ...

prototypes. Curiosity and a learning mindset - tracking state-of-the-art advancements and evaluating their relevance. Strong data manipulation and engineering skills (e.g., PySpark, SQL, cloud-based data tooling). Strong communication skills, capable of explaining complex ideas to technical and non-technical audiences. Collaborative approach to working with … industry collaborations). Familiarity with vector databases, embeddings, and retrieval techniques. Experience Working on large datasets (efficient loading, batching, and streaming) and related tooling (PySpark preferred). What you can expect from us We won't just meet your expectations. We'll defy them. So you'll enjoy ...

prototypes. Curiosity and a learning mindset - tracking state-of-the-art advancements and evaluating their relevance. Strong data manipulation and engineering skills (e.g., PySpark, SQL, cloud-based data tooling). Strong communication skills, capable of explaining complex ideas to technical and non-technical audiences. Collaborative approach to working with … industry collaborations). Familiarity with vector databases, embeddings, and retrieval techniques. Experience Working on large datasets (efficient loading, batching, and streaming) and related tooling (PySpark preferred). What you can expect from us We won't just meet your expectations. We'll defy them. So you'll enjoy ...

champion best practice methods; upskill the team through coaching and code reviews. Query and engineer large datasets (e.g., Python/R/SQL/PySpark) on modern platforms (e.g., Azure Databricks). Communicate pricing proposals and insights clearly and persuasively to stakeholders at all levels. Collaborate with Modelling, Optimisation … insurance pricing. Strong technical expertise in predictive modelling techniques (e.g. GLMs and machine learning models). Proficiency in programming languages such as R, Python, PySpark, SAS, or SQL. Good knowledge of Azure Databricks and modern data technologies. Experience with WTW Radar software and optimisation tools. A quantitative degree ...