51 to 75 of 119 PySpark Jobs in the UK

lifecycle Collaborating across squads and upskilling within the Databricks ecosystem YOUR SKILLS AND EXPERIENCE: You will bring strong capability in: Python development Databricks Azure PySpark Understanding of data engineering principles A proactive, growth-mindset approach with strong collaboration skills THE BENEFITS: You will receive a salary ...

lifecycle Collaborating across squads and upskilling within the Databricks ecosystem YOUR SKILLS AND EXPERIENCE: You will bring strong capability in: Python development Databricks Azure PySpark Understanding of data engineering principles A proactive, growth-mindset approach with strong collaboration skills THE BENEFITS: You will receive a salary ...

developers, architects and product teams to support analytics, machine learning and wider data-driven projects.Relevant experience includes:• Databricks or Snowflake, ideally within AWS• Python, PySpark and SQL• ETL pipelines and data transformation• End-to-end data modelling• DBT, SparkSQL, Airflow, Git, Jenkins or similar tools• Data quality, governance ...

methodology. Platforms & Tools Experience with Databricks (preferred) or similar platforms: Snowflake, BigQuery, Redshift, Synapse, or equivalent Strong proficiency with: SQL (advanced level) Spark/PySpark Data pipeline orchestration tools (e.g., Airflow, ADF, etc.) Data Modelling Extensive experience designing: Fact and dimension tables Star and snowflake schemas Aggregation layers ...

apply Experience working as a Data Engineer Hands-on experience with Microsoft Fabric Experience with Azure Strong SQL skills Experience using Python, ideally PySpark Experience building and maintaining ETL or ELT pipelines Experience working directly with stakeholders Ability to gather requirements and translate them into technical solutions Experience with ...

experience in CRM Analytics, Data Analytics, or related roles, ideally within the fashion, retail, or luxury sector Strong proficiency in Databricks (SQL, Python, or PySpark) for data transformation and analysis Solid understanding of CRM principles, customer segmentation, and campaign measurement. Experience working with customer databases, CDPs, and marketing automation ...

experience with Azure AI Search or other vector/hybrid search solutions, including vector search, hybrid search, and retrieval evaluation. Experience with Databricks and PySpark for large‐scale data processing. Experience designing evaluation frameworks for LLM systems, including offline metrics, hallucination detection, and production monitoring. Experience implementing responsible ...

Platforms: Google Cloud Platform, Amazon Web Services, Microsoft Azure, and Databricks. Experience in one or more of the listed Languages such as: Python, R, PySpark Experience in one or more of the following Agentic & Gen AI Platforms: such as Microsoft Copilot Studio, Adept AI, UiPath, OpenAI GPT-5 Agents ...

expertise with strategic thinking. Essential Skills & Experience Strong Microsoft Fabric expertise (Lakehouse, Warehouse, Pipelines, Notebooks) OneLake architecture and management Advanced SQL/T-SQL PySpark and Spark optimisation Data warehousing, modelling and ETL/ELT development Power BI semantic modelling Experience delivering enterprise cloud data platforms Experience integrating multiple … deliver like a Senior Data Engineer . You'll be equally comfortable discussing strategic transformation plans with senior leadership as you are building PySpark pipelines, optimising Fabric workloads and solving complex data challenges. This is not a reporting role — it's a platform ownership role that will define ...

This job is with Capgemini, an inclusive employer and a member of myGwork – the largest global platform for the LGBTQ+ business community. Please do not contact the recruiter directly. Who You Will Be Working With ...

assure enterprise-aligned data integration architectures Optimise cloud platforms (Azure, AWS, Databricks) for performance and cost Build and own ETL/ELT pipelines using PySpark, SQL and orchestration frameworks Automate and modernise legacy data processes with reusable patterns Design robust data models (Delta Lake, schema evolution, metadata, lineage) Ensure … reviews Collaborate with stakeholders to translate complex technical decisions clearly Support strategic roadmap, tooling and continuous improvement initiatives Core Skills & Expertise Data Engineering Advanced PySpark and SQL, performance tuning and best practice coding Data modelling (conceptual, logical, physical) and integration design Data quality, testing frameworks and automated validation Strong ...

outputs Set up monitoring and ensure data health for outputs What we are looking for Proficiency in Python, with experience in Apache Spark and PySpark Previous experience with data analytics softwares Ability to scope new integrations and translate user requirements into technical specifications What ...

everyone downstream, including governance Who you are Strong in SQL and Python, with real pipeline experience, not just querying Comfortable with Spark or PySpark, dbt and Airflow for orchestration Experience with at least one major cloud platform (AWS, GCP or Azure) You've worked with vector databases or feature ...

understanding of DAG (Directed Acyclic Graphs) and their role in pipeline orchestration It would be nice if you could bring (desirables): Professional experience with PySpark, MySQL, Docker, Kubernetes Experience with best-practice pipeline orchestration software (Airflow, Dagster). Experience migrating code from prototype notebooks to .py, while maintaining logic ...

Azure Data Platform and Databricks ecosystem. The ideal candidate should have strong hands-on experience with Structured Streaming, Delta Live Tables (DLT), PySpark architecture, Python, and SQL, and a proven track record of delivering large-scale enterprise data platforms. Youll be responsible for: Leading the design, development, and implementation ...

ongoing basis. Contribute to the planning, research, recommendation, and implementation of current and future IT strategies. PERSON SPECIFICATION: Essential Demonstrable experience of Spark (PySpark) and SQL. Demonstrable experience of Fabric components. Demonstrable experience of working on enterprise environments. Experience of designing architectural Fabric processes. Experience of Azure Data Factory. ...

experiment design and conducting A/B tests Bonus Points – Experience building shared or platform‐style ML systems Bonus Points – Experience with Databricks and PySpark Bonus Points – Experience working with AWS or another cloud platform (GCP/Azure) Additional Information Health + Mental Wellbeing: PMI and cash plan healthcare ...

through best practices, mentoring, and thoughtful technical direction. \n Job responsibilities \n \n Design scalable, reusable data processing and data quality frameworks using Python, PySpark, and dbt \nBuild and optimize batch and streaming data pipelines with strong performance, fault tolerance, and observabilityDevelop and operate workflow orchestration (e.g., Apache Airflow … data movement and transformationsModel and transform data for analytics using SQL and dbt to support business intelligence and reporting workloadsWrite production-grade Python/PySpark code with disciplined testing, performance tuning, and maintainable object-oriented designImplement infrastructure-as-code (e.g., Terraform) to provision and manage cloud-based data platform ...

considering: We are seeking an experienced Azure Data Engineering Lead with deep expertise in Azure Databricks, Azure Data Factory (ADF), PySpark, SQL, Delta Lake, and Azure cloud-native data services to drive the design and delivery of enterprise-scale data platforms for leading banking and financial services clients. … with Azure Databricks, Azure Data Factory (ADF), and ADLS Gen2. Expertise in designing scalable, resilient, and enterprise-grade Azure data solutions. Advanced proficiency in PySpark, Python, Spark SQL, and SQL. Strong experience with SQL optimization, performance tuning, joins, window functions, and stored procedures. Extensive experience in developing and optimizing ...

/GCP (based on organization need) integrated with Databricks. Data Engineering & Platform Build Build and optimize large-scale ETL/ELT pipelines using PySpark, SQL, and Delta. Oversee data quality frameworks, metadata management, lineage, and monitoring. Leadership & Stakeholder Engagement Work closely with business leaders and product teams to translate … years overall experience in data engineering or data architecture. 5+ years of hands-on architecture experience specifically in Databricks . Expert in Apache Spark , PySpark , Databricks SQL , Delta Lake, and distributed systems. Strong understanding of Lakehouse architecture , BI ecosystems, and modern data platforms. Cloud expertise in Azure/ ...

Machine Learning – Understanding of AI/ML concepts, experience with LLMs, prompt engineering, or integrating AI capabilities into applications • Data Technologies – Experience with Kafka, PySpark, Parquet and Avro data formats, Apache Airflow for workflow management, or Streamlit for building interactive data applications • Advanced Data Management – Strong SQL design, query ...

practices. Strong AWS knowledge across IAM, S3, KMS, and CI/CD tooling such as CodePipeline, CodeBuild, or equivalent. Expert Python development skills, with PySpark experience highly desirable. Proven experience designing enterprise MLOps frameworks, including model registries, monitoring, governance, and deployment automation. Strong understanding of statistical validation and model ...

recommendation models Experience with deep learning & large language models Experience with experiment design and conducting A/B tests Experience with Databricks and PySpark Experience working with AWS or another cloud platform (GCP/Azure) Additional Information Health + Mental Wellbeing PMI and cash plan healthcare access with Bupa ...

machine learning operations. Excellent communication skills with the ability to influence cross-functional teams and stakeholders. Experience with Python, SQL, Java, HQL, Hive, Pyspark, or similar programming languages preferred Well-versed in design thinking principles Great problem solving and analytical skills The total cash range for this position ...

between security and broader technology teams to demonstrably improve the overall security posture. Desirable skills and experience: Scripting and automation capabilities, particularly utilising Python, PySpark, and SQL to streamline analytical workflows. Experience navigating cloud-native security environments (AWS, Azure, or GCP) alongside familiarity with the Microsoft 365 ecosystem. Advanced ...