276 to 300 of 335 PySpark Jobs in the UK

Job Title Senior Data Engineer Job Description Summary Senior Data Engineer | Databricks | Agentic AI | EMEA & APAC We are the team leading the data transformation of our EMEA and APAC business, and we are hiring a ...

engineering, or a related field Experience analysing gambling marketing activities The ability to write clear, well-structured code Hands‐on experience with Python, SQL, PySpark, or similar tools A flair for problem‐solving and storytelling with data — and the confidence to present findings to clients Sound like ...

facing team in London. You will mentor data scientists, oversee complex data products, and ensure analytical excellence. The ideal candidate has strong SQL and PySpark experience, excellent communication skills, and a Bachelor’s degree in a related field. This position offers a collaborative environment, competitive benefits, and a chance ...

Engineer role. They aren't looking for someone who wants to move away from development into pure management. You'll work across: Python/PySpark Microsoft Fabric Spark and notebook engineering Lakehouse/Medallion architecture Delta SQL Azure Git/CI/CD Automated testing Data quality and observability … Reusable engineering patterns They're particularly interested in people coming from Databricks/PySpark environments, so extensive Fabric experience isn't essential. At Senior level, you'll also have the opportunity to influence architecture, technical design, coding standards, CI/CD, testing and engineering best practice, while supporting other ...

Lead the design and implementation of scalable data architectures on the Databricks Lakehouse platform. Oversee the development of efficient ETL/ELT pipelines using PySpark, SQL, and Delta Lake. Mentor and guide junior data engineers, promoting best practices in code quality, CI/CD, and performance tuning. Collaborate with … Requirements & Skills Databricks Expertise: Extensive hands-on experience with Databricks, Delta Lake, Unity Catalog, and Spark optimization. Programming Languages: Advanced proficiency in Python/PySpark and SQL. Cloud & Infrastructure: Strong background in cloud data platforms (Azure, AWS, or GCP) and cloud-native architectures. Engineering Best Practices: Experience with ...

data exploration skills Experience of working in Agile methodology Good knowledge and hands-on experience of developing Stored Procedures Experience of working with PySpark Notebooks for data ingestion and provision Practical experience of improving data quality and efficiency in data pipelines Solid understanding of DevOps and CI/… Understanding of compliance and lifecycle of data management Desirable Skills: Coding experience in any relevant language (Python, Pyspark etc.) Experience working with other Azure technologies such as Fabric, Power Automate Experience of working with non-relational databases (CosmosDB, MongoDB, etc.) and handling unstructured data in ETL pipelines Our values ...

maintainable solutions. What you'll be doing: Take ownership of designing, building, and maintaining scalable, production-ready data pipelines using Microsoft Fabric, Python, and PySpark, with minimal supervision Make sound technical decisions on pipeline architecture, data modelling, and tooling, weighing up trade-offs in performance, cost, and maintainability Mentor … domain-driven modelling) - this is the load‐bearing, provider‐agnostic skill underneath almost all of our pipeline work Have strong proficiency in Python and PySpark for batch data processing, with a track record of writing production‐grade, well‐tested code (this role is focused on batch/ELT pipeline ...

operational decisions. Applying statistical and machine learning techniques to build predictive and prescriptive models, test hypotheses and solve real business problems. Using Python, PySpark and SQL to analyse data and develop high-quality, maintainable solutions. Creating and maintaining data pipelines and automated workflows, following agreed ETL and data governance … domain. Proven expertise in using data, statistical analysis and machine learning to solve business challenges and generate actionable insight. Strong technical capability across Python, PySpark and SQL, with experience working with complex datasets. Strong knowledge of machine learning algorithms, including classification, regression, supervised learning and unsupervised learning. Proven expertise ...

operational decisions. Applying statistical and machine learning techniques to build predictive and prescriptive models, test hypotheses and solve real business problems. Using Python, PySpark and SQL to analyse data and develop high‐quality, maintainable solutions. Creating and maintaining data pipelines and automated workflows, following agreed ETL and data governance … domain. Proven expertise in using data, statistical analysis and machine learning to solve business challenges and generate actionable insight. Strong technical capability across Python, PySpark and SQL, with experience working with complex datasets. Strong knowledge of machine learning algorithms, including classification, regression, supervised learning and unsupervised learning. Proven expertise ...

Lakes*** Experience with **Apache Airflow/MWAA** for workflow orchestration* Strong knowledge of **dbt (data build tool)** for data transformation* Experience with **EMR/PySpark** for large-scale data processing* Ability to create **Architecture Design Documents (ADDs)*** Deep understanding of **cloud-native design patterns*** Experience in **performance tuning** across ...

best practices and CI/CD within an agile, hybrid setup that supports collaboration across Haywards Heath, Salford Quays, and Guernsey. You’ll use PySpark, Python, SQL, and Azure-based tools to implement end-to-end data solutions, mentor junior engineers, and #J-18808-Ljbffr ...

Data Engineer - Python/PySpark/Microsoft Fabric £55,000 - £65,000 + Benefits St Helens - 1-2 days per week This is an opportunity to join an organisation at a particularly interesting point in the development of its data capability. The business has invested heavily in modernising … procedures and drag-and-drop development. The team is deliberately moving towards a more code-led engineering approach, with a strong emphasis on Python, PySpark and notebook-based development. What you’ll be working with The environment includes: Python/PySpark Microsoft Fabric Spark and notebook development Lakehouse ...

Data Engineer (Databricks) Leeds (Hybrid) 6 Month Contract £(Apply online only)/day (Inside IR35) Data Engineer needed with strong hands-on Databricks, PySpark, AWS, SQL + Python experience. 6 Month Contract based in Leeds (Hybrid). Start ASAP in October 2026. Hybrid Working Model - primarily working from home … month. A chance to work with a leading global IT transformation on a large-scale Government project: Data Engineer with strong Databricks, AWS, Python, PySpark + SQL skills. Hands-on Data Engineer to run + maintain production data pipelines on Databricks and AWS. Keeping scheduled data processing reliable, resolve ...

2026. A chance to work with a leading global IT transformation on a large-scale Government project: Data Engineer with strong Databricks, AWS, Python, PySpark and SQL skills. Hands-on Data Engineer to run + maintain production data pipelines on Databricks and AWS. Keeping scheduled data processing reliable, resolve … understanding of role-based access, KMS encryption + diagnosing data access failures. Owning incidents from investigation through recovery and closure. Diagnosing issues in Python, PySpark, SQL, Databricks jobs and AWS integrations + providing clear progress updates and timely escalation. Working knowledge of Git, code reviews, CI/ ...

team. Nice to have: A PhD or equivalent research training in a quantitative field, and/or familiarity with Databricks, dbt, Spark/PySpark, or similar large-scale data platforms. Experience modifying infrastructure-as-code (e.g. Terraform) or contributing to production data pipelines. Experience building shared tools or platforms … them. Experience with UTXO, EVM, or other blockchain data, graph computation, and/or the cryptocurrency ecosystem. Technologies we use: Python SQL Databricks Dbt PySpark/Spark Terraform AWS Postgres AI at Chainalysis AI is not a feature at Chainalysis - it is a new way of working. One that ...

scalable, secure, and high-performance data platforms.Key Responsibilities Design and implement scalable AWS Data Lakehouse solutions Develop and maintain data pipelines using Python and PySpark Work with AWS services including: S3, EMR, Lambda, IAM, EKS, MWAA Build and manage Apache Iceberg tables for efficient data storage and querying Implement … engineersRequired Skills & Experience Strong experience in AWS Data Engineering Hands-on expertise with: AWS services: IAM, Lambda, EKS, S3, EMR, MWAA Apache Iceberg Python & PySpark Solid understanding of data lakehouse architecture principles Experience with CI/CD pipelines and Terraform Knowledge of data quality frameworks and data security standards ...

orchestration, experiment tracking, hyper-parameter tuning, and deploying automated model retraining pipelines. Programming & Data Engineering: Proficiency in object-oriented Python, advanced dataframes (Polars/Pyspark), and data versioning (DVC). Experience designing data storage solutions and using object-oriented SQL interfaces. Cloud & DevOps: Hands-on experience with at least … Desirable experienceAdvanced Education: A Master's degree or PhD in a relevant field is a strong plus. Parallelisation & Performance: Experience with parallelisation frameworks like Pyspark or Ray. Advanced Cloud & Infrastructure: Familiarity with serverless deployments (e.g., Fargate, Lambdas), infrastructure automation with Terratest or Ansible. ...

star schema and other best practices to optimise data usability. Sets best practices and standards on how we use SQL & dbt/Python/Pyspark and/or other languages to solve business problems. Becomes an expert in the business subject area the team are working on, holding expertise … team click. Were a group that chooses to lift each other up and think smart every day. Extensive experience in SQL/Python/Pyspark and/or other languages relevant to data processing Extensive experience in dbt and ideally Airflow (or other orchestration tools) Extensive experience in Entity ...

The PureGym Way The PureGym Group is a global gym business with a community of more than 700 gyms and over 2 million members. At PureGym, everything we do comes back to one simple idea ...

seeking an experienced Senior Data Engineer to design, build, and maintain scalable data pipelines and distributed data processing solutions. Mandatory/Primary Skills: Python PySpark AWS SQL Experience 4 to 7 years of hands‐on experience in Data Engineering and building scalable data solutions. Strong experience working with … based data engineering services and cloud data platforms . Technical Skills Strong hands‐on expertise in Python and PySpark for large-scale data processing. Advanced proficiency in SQL , including writing complex queries, query optimization, and data analysis. Strong experience with AWS services relevant to data engineering and cloud-based ...

pipelines to ingest, transform, and prepare data Develop and optimise datasets for reporting and analytics Implement data transformations using SQL and Python (incl. PySpark) Monitor, troubleshoot, and improve performance of data pipelines and queries Apply data quality controls and maintain clear documentation Contribute to ongoing improvements in data architecture … with AWS data services (e.g. Redshift, Athena, S3, Glue, Lambda) Strong SQL skills Exposure to SnapLogic or similar integration platforms Proficiency in Python (including PySpark) Familiarity with modern data tooling (e.g. open table formats, DuckDB beneficial) Experience with version control (Git/CodeCommit preferred) Awareness of BI tools (e.g. ...

Snowflake – Warehouses, Streams/Tasks, Snowpipe, Snowpark, UDFs/Stored Procedures, Time Travel and Cloning. Snowpark and end‐to‐end pipeline development. Experience with PySpark, Polars or Dask is also valuable. Python, including testing, build tooling and engineering best practices. SQL, including Snowflake‐optimised SQL. dbt, including Jinja templating … champion better ways of working. Nice to Have Experience with Docker, Java/Scala, Kafka, Pulsar, Azure Event Hub, Collibra, Apache Iceberg, Apache Arrow, PySpark or AI engineering technologies such as Snowflake AI, LangChain, LlamaIndex or Pinecone would be advantageous. This is a great opportunity to join a collaborative ...

Snowflake – Warehouses, Streams/Tasks, Snowpipe, Snowpark, UDFs/Stored Procedures, Time Travel and Cloning. Snowpark and end-to-end pipeline development. Experience with PySpark, Polars or Dask is also valuable. Python, including testing, build tooling and engineering best practices. SQL, including Snowflake-optimised SQL. dbt, including Jinja templating … champion better ways of working. Nice to Have Experience with Docker, Java/Scala, Kafka, Pulsar, Azure Event Hub, Collibra, Apache Iceberg, Apache Arrow, PySpark or AI engineering technologies such as Snowflake AI, LangChain, LlamaIndex or Pinecone would be advantageous. This is a great opportunity to join a collaborative ...

Remote - United Kingdom JOB ID: FEQ427R333 Location: The customer is based in Leeds or Manchester, United Kingdom Industry: Enterprise Retail or CPG (UKI Enterprise) - The customer is based in Leeds. You will be visiting the ...

Tria is seeking an experienced Senior Data Engineer to design and deliver scalable data pipelines using Python, PySpark and SQL in a cloud-based environment. The role focuses on Airflow orchestration, Kubernetes deployment on Azure, data modeling and building a modern data warehouse to support analytics and product insights. ...