26 to 50 of 53 PySpark Jobs in the UK excluding London

complex data led organisation Deep understanding & hands-on leverage of Modern, Cloud scale Data Technologies, with specific expertise in Distributed Data Processing architectures (Spark, PySpark, Python, Airflow), ELT data processing underpinning Data LakeHouses and Data Products for scaled use within ML & AI Expertise in Streaming Data, Event Driven Design ...

Azure Data Platform and Databricks ecosystem. The ideal candidate should have strong hands-on experience with Structured Streaming, Delta Live Tables (DLT), PySpark architecture, Python, and SQL, and a proven track record of delivering large-scale enterprise data platforms. Youll be responsible for: Leading the design, development, and implementation ...

datasets for modelling and analysis. Strong background in NLP and GenAI techniques, including text classification, summarization, and embeddings. Advanced proficiency in SQL, Python, and PySpark, with a focus on software development best practises. Experience deploying data science models in a cloud environment (Azure preferred). Familiarity with database management ...

description and main responsibilities. Person Specification Qualifications Essential Relevant degree/apprenticeship or equivalent foundational experience. Knowledge Essential Basic programming awareness - Python, SQL, PySpark Understanding of data concepts (datasets, structures, basic pipelines). Skills & Experience Essential Works under close supervision Follows defined processes and standards Demonstrates problem-solving mindset ...

experience using Pandas. Experience working with SQL and cloud-based data warehousing technologies. Knowledge of Google BigQuery would be highly beneficial. Exposure to PySpark and distributed data processing environments. Ability to analyse, transform, and optimise complex datasets. Strong problem-solving and stakeholder communication skills. How to Apply ...

Kubernetes Experience in creating production grade coding and SOLID programming principles, including test-driven development (TDD) approaches Experience in programming languages (e.g. Python, PySpark, R, SAS, SQL) Experience in source-control software, e.g., GitHub Proficient at communicating results in a concise manner both verbally and written Experience in data ...

office, location options Welwyn Garden City or Slough. You will design, build and deploy enterprise-scale data science solutions using Python, SQL, PySpark and Azure Databricks. The role focuses on customer analytics, ML modelling, time series forecasting and production-ready analytics that inform business decisions and improve commercial outcomes. ...

architecture experience. Dimensional data modelling experience. Kimball/Inmon Azure Data engineering with exposure to Synapse, Databricks, Data Factory Azure SQL. Python Scripting for Pyspark C# Experience Experience with TDD and Unit Testing. Desirables - Experience working on Multi Cloud Solutions Azure Integration experience with Azure Function Apps/Logic ...

promote best practice across the analytics function. Independently own projects from discovery through to delivery. Skills & Experience Strong SQL skills. Python and/or PySpark experience. Experience within Product Analytics or Digital Analytics. Strong understanding of experimentation and A/B testing. Good grounding in statistics and analytical methodologies. ...

What we expect from you Degree in Statistics, Mathematics, Physics, Economics, or a related quantitative field Strong programming skills - Python and SQL are essential; PySpark experience is highly advantageous Experience with version control tools (e.g. GIT) Solid understanding of analytical technologies, tools, and techniques Excellent logical thinking and problem ...

visible role where you'll combine strong technical skills with commercial thinking to drive meaningful product improvements. Your Skills & Experience Strong SQL skills Python (PySpark experience is beneficial) Experience in Product Analytics Strong knowledge of A/B testing and experimentation Excellent statistical analysis skills, with exposure to modelling ...

visible role where you'll combine strong technical skills with commercial thinking to drive meaningful product improvements. Your Skills & Experience Strong SQL skills Python (PySpark experience is beneficial) Experience in Product Analytics Strong knowledge of A/B testing and experimentation Excellent statistical analysis skills, with exposure to modelling ...

ecosystem. This role will focus on building enterprise-grade data pipelines, migration frameworks and transformation processes using Azure Synapse Analytics, Azure Data Factory (ADF), PySpark, Python and SQL. You will work closely with technical and business stakeholders to ensure successful delivery of data migration and integration initiatives. Key Responsibilities … Data Factory (ADF). Develop Change Data Capture (CDC) solutions to support incremental and delta data loads. Create data transformation and cleansing processes using PySpark and Spark DataFrames. Develop Python-based utilities and automation scripts to support migration and validation activities. Design and optimise data models, SQL queries, stored ...

structured and unstructured datasets Validate large-scale data ingestion, transformation, and processing workflows Ensure data integrity, transformation accuracy, reconciliation, and performance under load Oversee PySpark-based data validation and interrogation activities on AWS EMR Support testing and monitoring of workflows orchestrated through Apache Airflow Drive root cause analysis … DynamoDB, AWS CloudWatch, AWS IAM, Python, SQL, Java, Bash, GitLab CI/CD, Jupyter Notebooks, Apache Spark/PySpark, Terraform, Docker Key Skills Strong proficiency in Python, SQL, Java, and Scripting languages such as Bash Hands-on experience testing Big Data platforms and Data Lake solutions Strong experience with ...

accurate documentation, lineage and operational run books to support ongoing platform support Key Responsibilities Develop and maintain data pipelines using Azure Databricks (Python, PySpark and SQL) Implement ingestion and transformation across Bronze, Silver and Gold layers using Delta Lake Support and operate Azure Databricks and Azure Synapse platforms, including … pipelines and code reviews What you'll need to succeed Essential Proven experience as a Data Engineer Strong hands-on experience with AzureDatabricks, Python, PySpark and SQL Experience supporting Azure data platforms in production Understanding of medallion architecture and Delta Lake Desirable Experience with Azure Synapse Analytics Experience with ...

Senior Data Scientist Welwyn Garden City, Hertfordshire or Slough, Berkshire | Hybrid (2 Days Office) | Python | SQL | PySpark | Azure | Databricks | Machine Learning | Statistics | Time Series Forecasting | Propensity Models Role: Senior Data Scientist, Lead Data Scientist, ML Scientist, AI Data Scientist, Statistical Data Scientist, Predictive Analytics Key Skills: Senior Data Scientist … Data Scientist, Machine Learning, Artificial Intelligence, Predictive Analytics, Statistical Modelling, Python, SQL, PySpark, Azure, Databricks, MLflow, GitHub Actions, CI/CD, Time Series Forecasting, Propensity Modelling, Feature Engineering, Data Engineering, Azure Machine Learning, Azure Data Platform, Experiment Design, A/B Testing, Hypothesis Testing, Predictive Modelling, Customer Analytics, Marketing ...

Engineer (AWS) - Manchester (Data Engineer, Senior Data Engineer, ETL, AWS, AWS Glue, AWS Lambda, AWS S3, AWS Redshift, Amazon EMR, Python, SQL, Pipeline Orchestration, PySpark, data warehousing, data lakehouse, data architecture, Agile, Senior Data Engineer, Data Engineer) Our client is a global innovator and world leader with … with AWS services such as AWS Glue, AWS Lambda, AWS S3, AWS Redshift, and Amazon EMR. Proven expertise in Python, SQL, Pipeline Orchestration and PySpark is also expected, as is experience with data lakehouse architectures and data warehousing solutions. Agile experience is also key, as are excellent communication skills ...

Senior Databricks Architect - London (Hybrid) - Databricks - Spark - PySpark - AWS - Up to £115,000 + Bonus Our client, a large-scale organisation undergoing significant investment in data and cloud technologies, is seeking an experienced Senior Databricks Architect to join its expanding Data & AI practice. This is an exciting opportunity … time data pipelines from legacy and modern source systems Implementing real-time and event-driven data processing solutions Developing data transformation frameworks using Spark, PySpark and SQL Designing Lakehouse architectures using Databricks and Medallion Architecture principles Creating scalable, reusable data products to support modern data operating models Integrating data ...

**Take the lead on meaningful, high-impact projects as a Senior Data Engineer.** We are seeking a Senior Data Engineer to join a growing organisation based in London, offering a hybrid work model. This role ...

adoption of modern data mesh principles across complex enterprise environments. What You'll Be Doing Architect and deliver modern data solutions using Databricks, Spark, PySpark and SQL Design scalable batch and real-time data pipelines that ingest data from legacy and modern platforms Lead the development of cloud-based … making Provide technical leadership and architectural direction on large-scale transformation programmes Essential Experience Strong hands-on expertise with Databricks Extensive experience using Spark, PySpark and SQL Proven track record designing and implementing data lake architectures Experience building both batch and streaming data pipelines Strong knowledge of Kafka Good ...

data processing solutions Work with modern data platforms such as Microsoft Fabric, Databricks, or similar technologies Build and optimise data solutions using Python, Spark, PySpark, and/or SQL Support data integration, transformation, and performance optimisation Collaborate with stakeholders to ensure high-quality, reliable data architecture Essential Skills & Experience … Hands-on experience with Microsoft Fabric, Databricks, or equivalent modern data platforms Strong coding experience in Python and/or Spark/PySpark Solid SQL skills and experience working with structured datasets Desirable Experience Knowledge of relational database design , normalisation , and data modelling Experience designing efficient, scalable data architectures ...

required for the domain's data products, using the tools and guardrails provided by the Enablement team. Expected Skills : Advanced coding in Python and PySpark, strong SQL, expertise in building declarative pipelines (eg; Delta Live Tables), version control (Git), automated testing (eg; Pytest), and CI/CD deployment practices … automate data pipelines to ingest, transform, and model data specific to the squad's use cases. Write structured, scalable code (eg; Python/PySpark/SQL) and apply software engineering best practices, including version control (Git) and automated testing. Work closely with the Platform Enablement team to package their ...

Data Scientist Slough - 2 days per week Up to £520 per day - inside IR35 Initial 6 month contract SQL/Python/PySpark/NLP/Gen AI/Azure This is a hands-on Data Scientist role within the Journey Insights team, focused on building and delivering data … techniques, including text classification, summarisation, entity extraction, embeddings, retrieval-augmented generation (RAG) and LLM-enabled workflows Strong programming expertise in SQL, Python and PySpark, applying software engineering best practices such as code readability, version control, testing and reproducibility Experience deploying data science models and products in cloud environments, ideally ...

Logging, tracking and managing defects through to resolution Contributing to testing strategies, quality frameworks and continuous improvement initiatives Supporting testing activities within Databricks and PySpark environments SKILLS AND EXPERIENCE The successful Data QA Engineer will have the following skills and experience: Experience in a Data QA, Software Testing … Quality Assurance role Strong experience testing data platforms, pipelines and data products Hands-on experience with Databricks, SQL, Python and PySpark Strong stakeholder communication and collaboration skills Able to work independently and take ownership of testing activities and quality standards BENEFITS The successful Data QA Engineer will receive ...

Logging, tracking and managing defects through to resolution Contributing to testing strategies, quality frameworks and continuous improvement initiatives Supporting testing activities within Databricks and PySpark environments SKILLS AND EXPERIENCE The successful Data QA Engineer will have the following skills and experience: Experience in a Data QA, Software Testing … Quality Assurance role Strong experience testing data platforms, pipelines and data products Hands-on experience with Databricks, SQL, Python and PySpark Strong stakeholder communication and collaboration skills Able to work independently and take ownership of testing activities and quality standards BENEFITS The successful Data QA Engineer will receive ...