1 to 25 of 39 Remote/Hybrid PySpark Jobs in the UK

enhance customer engagement, service delivery, and operational efficiencies. What you'll need to succeed Knowledge/Qualifications - Expert-level SQL and Python (ideally PySpark).- A strong understanding of DataSecOps; experience with PII handling, data obfuscation, and RBAC (Role-Based Access Control).- Proven ability to design data systems ...

data engineering solutions using Python . Extensive hands-on experience with Databricks , including Workflows, Notebooks and Delta Lake. Strong experience with Apache Spark/PySpark . Proven experience building scalable ETL/ELT pipelines. Strong SQL and data modelling skills. Experience designing modern Lakehouse architectures. Experience working with cloud ...

environments. Strong hands-on expertise in Python Proven experience with Databricks, including notebooks, workflows, jobs and Delta Lake Strong knowledge of Apache Spark/PySpark Experience designing and delivering large-scale ETL/ELT pipelines Advanced SQL skills and experience with relational databases Experience working with cloud technologies such ...

assurance processes Excellent problem-solving skills with the ability to diagnose and resolve complex data issues Strong communication and stakeholder management experience Desirable Experience PySpark and/or Python development Azure Data Factory (ADF) Azure Synapse Analytics Azure Data Lake Storage Power BI development and data modelling Exposure ...

architects. Together, you will design and build modern machine learning and AI services that support analytics and improve products across the business. Experience with PySpark, Python, SQL and Databricks (or Snowflake) is required for these roles. Its a chance to work on a very modern tech stack and learn ...

+ environment configurations. Analysing current Data Engineering solutions + identifying data migration requirements. Onboarding datasets, users + workloads onto target platform. Data Engineering: Python, PySpark, SQL, Data Pipeline Development, ETL/ELT Design, Data Modelling, Data Quality, Data Validation. AWS Cloud Platform, S3, IAM, CI/CD concepts, DevOps ...

Azure. Experience with other platforms like AWS, Snowflake, or Databricks is also beneficial. Hands-on expertise with Data Factory (Fabric or Azure), notebooks (Python, PySpark), advanced SQL, and robust data modelling and performance optimisation skills. Proven track record in building scalable data pipelines and implementing modern architectures such ...

working with flexibility and autonomy Key Experience Deep end-to-end experience delivering enterprise Databricks analytics solutions Expertise in Lakehouse Architecture, Delta Lake, Spark, PySpark Strong data modelling skills (3NF, Kimball, Data Vault) Experience exposing Delta Lakehouse to Power BI or Tableau Skilled in Python, Scala ...

contribute to long-term platform innovation Required Experience Strong experience designing enterprise data platforms within Microsoft Azure environments Deep expertise with Azure Databricks, Spark (PySpark/Scala), Delta Lake, and Lakehouse architecture Strong hands-on experience with Snowflake architecture and optimisation Experience with Azure Data Factory, ADLS Gen2, Synapse ...

Data Lake and Azure Synapse Analytics Solid understanding of ETL processes, data integration and pipeline development Advanced SQL skills as well as Python/PySpark experience for data processing Exposure to Power BI or other reporting tools is beneficial What's on Offer Salary up to £65,000 Fully ...

Advanced SQL/T-SQL development skills Experience designing and implementing data models for reporting and analytical requirements Experience using Python and/or PySpark, particularly within Fabric Notebooks Exposure to Azure data technologies such as Azure Data Factory, Synapse Analytics or Data Lake What's on Offer Salary ...

years of proven experience delivering robust data engineering solutions. The Fabric Ecosystem: Demonstrable, hands-on expertise with Microsoft Fabric , SQL , and Python/PySpark Notebooks . Core Competencies: Proven skill in dimensional modeling, data governance, and data extraction from online platforms via SaaS and APIs. Analytical Mindset: Excellent ability ...

Data Engineering Foundation: Practical experience with data modelling, ETL/ELT, and various database technologies. Technical Versatility: Fluency in data processing languages (Python, SQL, PySpark) and openness to tech stacks such as SSIS, Airflow, Spark, Databricks, Fabric. Cloud Capability: Experience with at least one major cloud provider (AWS, Azure ...

wider data community Promote high coding and data practice standards Experiment with emerging data, ML and AI technologies What you'll need: Strong Python, PySpark, SQL and Databricks or Snowflake experience AWS cloud data engineering expertise Experience building scalable batch and real-time data pipelines Airflow, CI/ ...

with various databases e.g. MS SQL, Azure Cosmos DB. Skilled at optimizing large and more complicated SQL statements. Proficiency in Python and experience with PySpark Experience of REST API standards. Proficiency in writing clean, readable, and secure code for Azure backend services, i.e. functions, utility functions etc. Experience ...

semantic model design Hands-on experience with Microsoft Fabric, particularly lakehouse architecture, notebooks and data pipelines Strong understanding of Apache Spark/PySpark and large-scale data processing Experience designing and operating modern data platforms Experience developing or enabling AI and machine learning capabilities using business data Strong understanding ...

hands-on experience with modern cloud data platforms (e.g. Databricks or Snowflake), ideally within AWS environments Advanced programming and data engineering skills in Python, PySpark and SQL, with a solid understanding of ETL pipelines and data transformation Proven experience in data modelling (end-to-end), including performance optimisation using ...

developers, architects and product teams to support analytics, machine learning and wider data-driven projects.Relevant experience includes:• Databricks or Snowflake, ideally within AWS• Python, PySpark and SQL• ETL pipelines and data transformation• End-to-end data modelling• DBT, SparkSQL, Airflow, Git, Jenkins or similar tools• Data quality, governance ...

apply Experience working as a Data Engineer Hands-on experience with Microsoft Fabric Experience with Azure Strong SQL skills Experience using Python, ideally PySpark Experience building and maintaining ETL or ELT pipelines Experience working directly with stakeholders Ability to gather requirements and translate them into technical solutions Experience with ...

experience with Azure AI Search or other vector/hybrid search solutions, including vector search, hybrid search, and retrieval evaluation. Experience with Databricks and PySpark for large‐scale data processing. Experience designing evaluation frameworks for LLM systems, including offline metrics, hallucination detection, and production monitoring. Experience implementing responsible ...

expertise with strategic thinking. Essential Skills & Experience Strong Microsoft Fabric expertise (Lakehouse, Warehouse, Pipelines, Notebooks) OneLake architecture and management Advanced SQL/T-SQL PySpark and Spark optimisation Data warehousing, modelling and ETL/ELT development Power BI semantic modelling Experience delivering enterprise cloud data platforms Experience integrating multiple … deliver like a Senior Data Engineer . You'll be equally comfortable discussing strategic transformation plans with senior leadership as you are building PySpark pipelines, optimising Fabric workloads and solving complex data challenges. This is not a reporting role — it's a platform ownership role that will define ...

assure enterprise-aligned data integration architectures Optimise cloud platforms (Azure, AWS, Databricks) for performance and cost Build and own ETL/ELT pipelines using PySpark, SQL and orchestration frameworks Automate and modernise legacy data processes with reusable patterns Design robust data models (Delta Lake, schema evolution, metadata, lineage) Ensure … reviews Collaborate with stakeholders to translate complex technical decisions clearly Support strategic roadmap, tooling and continuous improvement initiatives Core Skills & Expertise Data Engineering Advanced PySpark and SQL, performance tuning and best practice coding Data modelling (conceptual, logical, physical) and integration design Data quality, testing frameworks and automated validation Strong ...

understanding of DAG (Directed Acyclic Graphs) and their role in pipeline orchestration It would be nice if you could bring (desirables): Professional experience with PySpark, MySQL, Docker, Kubernetes Experience with best-practice pipeline orchestration software (Airflow, Dagster). Experience migrating code from prototype notebooks to .py, while maintaining logic ...

language models Experience with experiment design and conducting A/B tests Experience building shared or platform-style ML systems Experience with Databricks and PySpark Experience working with AWS or another cloud platform (GCP/Azure) Additional Information Health + Mental Wellbeing PMI and cash plan healthcare access with ...

hands-on data engineering role within a large-scale cloud data programme, responsible for building, maintaining, and troubleshooting data pipelines using Apache Spark, PySpark, Apache Airflow, and a broad suite of AWS services. You will apply strong analytical and engineering skills to deliver trusted, well-governed data assets … data programme - designing, building, and maintaining data pipelines that process large volumes of data across a modern AWS-native stack. Using Apache Spark and PySpark for distributed data processing, Apache Airflow for orchestration, and a range of AWS services for storage, compute, and analytics, you will help deliver reliable ...