1 to 25 of 105 PySpark Jobs in the UK

tools (e.g. Power BI) Deep understanding of data warehousing concepts, ETL/ELT pipelines and dimensional modelling Proficiency in advanced programming languages (Python/PySpark, SQL) Experience in data pipeline orchestration (e.g. Airflow, Data Factory) Familiarity with DevOps and CI/CD practices (Git, Azure DevOps etc) Ability ...

enhance customer engagement, service delivery, and operational efficiencies. What you'll need to succeed Knowledge/Qualifications - Expert-level SQL and Python (ideally PySpark).- A strong understanding of DataSecOps; experience with PII handling, data obfuscation, and RBAC (Role-Based Access Control).- Proven ability to design data systems ...

analytical use cases Ability to work closely with both technical and non-technical stakeholders, translating business requirements into data solutions Experience with Python or PySpark, particularly within Fabric Notebooks Exposure to Azure data services such as Data Factory, Synapse or Data Lake Power BI experience, including supporting reporting ...

assurance processes Excellent problem-solving skills with the ability to diagnose and resolve complex data issues Strong communication and stakeholder management experience Desirable Experience PySpark and/or Python development Azure Data Factory (ADF) Azure Synapse Analytics Azure Data Lake Storage Power BI development and data modelling Exposure ...

engineering tools and storage solutions (e.g., ADLS, Spark, Databricks, SQL/NoSQL databases) Experience with distributed computing and big data processing frameworks such as PySpark Knowledge of infrastructure‐as‐code tools such as Terraform and Helm Advanced/Specialist Expertise Experience building and deploying GenAI solutions using frameworks such ...

deliver practical, value-driven data solutions. Drive continuous improvement across data engineering, reporting, and analytics capabilities. Essential Requirements Strong expertise in SQL and Python (PySpark desirable). Proven experience designing and building cloud-based data platforms from inception. Hands-on experience with Databricks, Spark, modern data lake/lakehouse ...

+ environment configurations. Analysing current Data Engineering solutions + identifying data migration requirements. Onboarding datasets, users + workloads onto target platform. Data Engineering: Python, PySpark, SQL, Data Pipeline Development, ETL/ELT Design, Data Modelling, Data Quality, Data Validation. AWS Cloud Platform, S3, IAM, CI/CD concepts, DevOps ...

Azure. Experience with other platforms like AWS, Snowflake, or Databricks is also beneficial. Hands-on expertise with Data Factory (Fabric or Azure), notebooks (Python, PySpark), advanced SQL, and robust data modelling and performance optimisation skills. Proven track record in building scalable data pipelines and implementing modern architectures such ...

working with flexibility and autonomy Key Experience Deep end-to-end experience delivering enterprise Databricks analytics solutions Expertise in Lakehouse Architecture, Delta Lake, Spark, PySpark Strong data modelling skills (3NF, Kimball, Data Vault) Experience exposing Delta Lakehouse to Power BI or Tableau Skilled in Python, Scala ...

Experience managing multi-engine compute environments CI/CD experience with Azure DevOps, GitHub, and Databricks Asset Bundles Strong programming skills in PySpark, Databricks SQL, and T-SQL Understanding of Medallion Architecture and modern data modelling approaches (Star Schema, Data Vault 2.0) Experience with performance optimisation in Databricks ...

contribute to long-term platform innovation Required Experience Strong experience designing enterprise data platforms within Microsoft Azure environments Deep expertise with Azure Databricks, Spark (PySpark/Scala), Delta Lake, and Lakehouse architecture Strong hands-on experience with Snowflake architecture and optimisation Experience with Azure Data Factory, ADLS Gen2, Synapse ...

Data Lake and Azure Synapse Analytics Solid understanding of ETL processes, data integration and pipeline development Advanced SQL skills as well as Python/PySpark experience for data processing Exposure to Power BI or other reporting tools is beneficial What's on Offer Salary up to £65,000 Fully ...

Advanced SQL/T-SQL development skills Experience designing and implementing data models for reporting and analytical requirements Experience using Python and/or PySpark, particularly within Fabric Notebooks Exposure to Azure data technologies such as Azure Data Factory, Synapse Analytics or Data Lake What's on Offer Salary ...

hands-on experience with modern cloud data platforms (e.g. Databricks or Snowflake), ideally within AWS environments. Advanced programming and data engineering skills in Python, PySpark and SQL, with a solid understanding of ETL pipelines and data transformation. Proven experience in data modelling (end-to-end), including performance optimisation using ...

years of proven experience delivering robust data engineering solutions. The Fabric Ecosystem: Demonstrable, hands-on expertise with Microsoft Fabric , SQL , and Python/PySpark Notebooks . Core Competencies: Proven skill in dimensional modeling, data governance, and data extraction from online platforms via SaaS and APIs. Analytical Mindset: Excellent ability ...

design decisions. Required Skills & Experience Strong analytical skills with experience working with large-scale and unstructured datasets. Python development experience, including technologies such as PySpark, Pandas and PyArrow. Expertise in distributed data processing frameworks, particularly Apache Spark. Experience building and maintaining ETL pipelines using technologies such as Apache Airflow ...

Significant hands-on experience in data engineering, analytics engineering, or related disciplines delivering enterprise data solutions \nStrong proficiency in Databricks, SQL, and Python/PySpark, including building, optimizing, and troubleshooting ETL pipelinesExperience designing scalable, maintainable data architectures and dimensional models for BI, reporting, and analytical use casesProven ability ...

emerging data technologies Cloud experience across Azure, AWS or similar. The client is technology agnostic and values transferable skills over specific stack experience Python, PySpark or equivalent modern data engineering tooling Comfortable working at pace in a collaborative, fast-moving environment If you are an experienced Data Engineer with ...

wider data community Promote high coding and data practice standards Experiment with emerging data, ML and AI technologies What you'll need: Strong Python, PySpark, SQL and Databricks or Snowflake experience AWS cloud data engineering expertise Experience building scalable batch and real-time data pipelines Airflow, CI/ ...

ADLS). Solid understanding of Scrum practices and a strong Agile mindset. Advanced proficiency in Python and its key data and ML libraries (NumPy, PySpark, Scikit learn, TensorFlow/PyTorch) Working knowledge of generative AI models (e.g., ChatGPT, Claude), Databricks GenAI tooling (including embedding models), and modern data structures ...

Qlik Cloud) Ability to use data science techniques and statistics, including use of Python/R Ability to manipulate data in data lakes using PySpark/Experience with cloud-based data architectures and scalable data solutions Personal Qualities Essential Excellent stakeholder management and interpersonal skills Highly developed negotiation ...

performance, scalability, and cost efficiency. Implement data quality controls, validation processes, and monitoring solutions. Data Transformation & Processing Develop complex data transformation logic using Spark (PySpark or Scala). Build data cleansing, enrichment, aggregation, and processing solutions. Ensure data accuracy, consistency, and reliability throughout the data life cycle. Azure Databricks … Experience 8+ years of Data Engineering experience (10+ years preferred). Minimum 3 years of hands-on Azure Databricks experience. Strong expertise in Python, PySpark, and/or Scala. Extensive experience with: Azure Databricks Azure Data Factory Azure Blob Storage Azure SQL Database Delta Lake Spark SQL Strong understanding ...

improve performance and minimize RU consumption. Leverage Cosmos DB Change Data Feed (CDF) capabilities to support data integration and processing solutions. Design and implement PySpark data pipelines using Microsoft Fabric. Establish and maintain CI/CD pipelines using GitLab to support automated deployment and delivery. Develop and maintain unit ...

looking for: Strong background in data engineering and software development Proven experience working with Databricks (essential) Hands-on expertise with tools such as Python, PySpark, Spark, and SQL Experience building and maintaining data pipelines and data platforms Solid understanding of data modelling and data architecture principles Experience working with ...

looking for: Strong background in data engineering and software development Proven experience working with Databricks (essential) Hands-on expertise with tools such as Python, PySpark, Spark, and SQL Experience building and maintaining data pipelines and data platforms Solid understanding of data modelling and data architecture principles Experience working with ...