1 to 25 of 88 PySpark Jobs in the UK excluding London

data engineering team while partnering closely with product, BI, data science, and business stakeholders Experience required: Strong experience with Microsoft Fabric, Delta Lake, PySpark, Python, SQL, and modern ETL/ELT frameworks, alongside CI/CD practices using Azure DevOps or GitHub Hands-on expertise integrating Azure AI/ ...

Warehouses, and real-time analytics solutions in Fabric Strong SQL/T-SQL capability, including optimisation and development of reusable objects. Proficiency in Python (PySpark, Spark SQL beneficial). Experience working with structured/unstructured data, delta lake patterns, partitioning, and scalable pipeline design. Strong understanding of data modelling ...

tooling Key skills Strong experience with Microsoft Fabric (Data Factory, Lakehouse, OneLake, Pipelines, SQL Warehouse) Solid data engineering background using SQL, Python, PySpark, Delta Lake E xperience building and optimising end-to-end data pipelines CI/CD experience for analytics and data platforms (Azure DevOps or GitHub) Exposure ...

technical architecture and detailed design. •Expertise in building scalable ingestion frameworks (batch/streaming) using ADF, Databricks or Fabric pipelines. •Strong programming skills in PySpark, SQL, Python. •Experience implementing medallion architecture, delta lakes, and metadata‐driven pipelines. •Hands-on experience with DQ frameworks, validation rules and reconciliation logic. •Ability ...

pipelines and working with lakehouse architectures Knowledge of tools such as Azure Data Factory and/or Fabric Data Pipelines Experience with Python or PySpark for data transformation Understanding of data modelling (dimensional modelling, medallion architecture) Strong communication skills and ability to work with non-technical stakeholders Desirable: Experience ...

added flexibility for diverse migration and integration projects. Prior experience with tools such as MuleSoft, Boomi, Informatica, Talend, SSIS, or custom scripting languages (Python, PySpark, SQL) for data extraction and transformation. Prior experience with Data warehousing and Data modelling (Star Schema or Snowflake Schema). Skilled in security frameworks ...

backgrounds. The technology stack is flexible, but experience in some of the following are nice to have: Data Engineering & Programming: • Python • SQL • Spark/PySpark • Scala • Data modelling and warehousing concepts Cloud & Platforms: • AWS (S3, Redshift, Glue, EMR, Lambda) • Other cloud platforms (Azure/GCP considered) • Data lake ...

automation and optimization of data workflows Requirements: Experience in data engineering and pipeline development Strong expertise in Azure (Synapse, Data Lake) Proficiency in Python, PySpark, and T-SQL Knowledge of data warehousing, CDC, and incremental processing Familiarity with modern data architectures (e.g., medallion) Preferred Experience with Azure DevOps/ ...

data pipelines using Databricks and Apache Spark Build and optimize data workflows for batch and real-time processing Transform and process large datasets using PySpark, SQL, and Delta Lake Implement data ingestion from multiple sources (APIs, streaming, databases) Your Skills: Python proficiency DBT AWS : Cloud formation , Service Catalog ...

best practices are followed Required Skills & Experience: Strong experience with Azure data platform (e.g., Data Factory, Data Lake, Synapse) Hands-on expertise with Databricks (PySpark or Spark) Solid SQL development skills Experience working with Kafka for streaming data pipelines Good understanding of lakehouse architecture, including bronze/silver/ ...

modern data platform development. Deep hands-on experience with Microsoft Fabric, including Lakehouse, Data Warehouse, OneLake, Pipelines and Data Factory. Strong SQL, Spark/PySpark and data modelling capability across analytical and operational datasets. Proven experience delivering complex end-to-end data platforms in Azure or Fabric environments. Strong ...

Strong experience designing and delivering data solutions in Databricks Proficient with SQL and Python Experience using Big Data technologies such as Apache Spark or PySpark Great communication skills, effectively participating with Senior Stakeholders Nice to haves: Azure/AWS Data Engineering certifications Databricks certifications What ...

Strong experience designing and delivering data solutions in Databricks Proficient with SQL and Python Experience using Big Data technologies such as Apache Spark or PySpark Great communication skills, effectively participating with Senior Stakeholders Nice to haves: Azure/AWS Data Engineering certifications Databricks certifications What ...

Azure ML. Key skills and responsibilities, Demonstrated hands-on experience with Microsoft Fabric and Power BI. Extensive knowledge in data modelling, DAX, Python/PySpark, SQL/KQL. Proven record of technical leadership and effective stakeholder engagement. Design and deliver comprehensive Fabric solutions, including OneLake, Lakehouse, Warehouses, and Power ...

business needs Ensure data quality, governance and best practice Key Requirements Strong leadership experience (5+ engineers) Solid Data Engineering/DataOps background Python/PySpark for data pipelines AWS experience (design & deployment) Strong data modelling & warehousing knowledge Agile (Jira/Azure DevOps) experience Get in touch for more details ...

Financial Services experience is preffered, but not essential) Comfortable challenging existing approaches and offering fresh technical perspectives Nice to Have Exposure to Databricks, PySpark or Python-based data processing Experience optimising legacy or sub-optimal Azure implementations Familiarity with Agile delivery frameworks Please reach out to Adam Akhtar ...

Support production systems and troubleshoot issues Write clean, tested, and maintainable code Tech Stack Python & SQL AWS (EMR, Athena, Lambda) Apache Spark (Scala or PySpark) Big data/distributed processing tools What They’re Looking For Strong experience as a Data Engineer using Python and SQL Experience working with ...

Lloyd's of London or London Market insurance sector Deep understanding of Delta Lake, Medallion architecture, Lakehouse patterns and data modelling. Strong Python, PySpark and Spark SQL capability Solid understanding of data governance Proven track record of ownership, delivery and operating effectively in fast-paced environments. What ...

Data Warehouse capabilities. Design, manage, and optimise data storage within OneLake , ensuring well-governed, well-structured, and shareable data assets. Develop scalable Spark (PySpark/Spark SQL) data transformations through Fabric notebooks to support high-volume processing. Create, orchestrate, and monitor robust data ingestion and transformation pipelines using Fabric ...

Power BI Optimise ELT, modelling, and workspace governance Skills Needed Strong experience with Azure & Microsoft Fabric Solid Synapse, ADF, ADLS, SQL, Python/PySpark Hands-on Power BI modelling & dataset expertise Experience in BI modernisation/migration projects If this is a role that suits your skillset, can work ...

strong data governance and GDPR compliance Skills Experience in data engineering within enterprise environments Strong Azure/Microsoft Fabric experience SQL and Python/PySpark ETL/ELT (ADF/Fabric) Knowledge of data modelling and governance ...

data projects Key Requirements Proven leadership of multiple engineering teams Strong Data Engineering/DataOps background Experience in large, dataheavy environments Python/PySpark for endto-end pipelines Strong AWS experience (design & deployment) Deep knowledge of data modelling, warehousing & lakehouse concepts Strong stakeholder management and communication skills Agile delivery ...

similar Demonstrable experience in programming full scale data and analytic pipelines, storage solutions and process automation functions Highly proficient coding skills in SQL, Python & PySpark Proven knowledge in at least one of the following big data technologies – HIVE, Hadoop, MongoDB, Spark Experience within the Life Science, Medical Research ...

production. Key skills and experience: Strong experience as a Data Engineer Advanced Python and SQL skills Experience with cloud platforms (AWS preferred) Proficiency in PySpark and/or Pandas Familiarity with data quality tools (e.g., Great Expectations) Experience with infrastructure as code (Terraform or CloudFormation) Understanding of modern development ...

will have the following skills and experience: Deep expertise with Azure Databricks, including development, deployment, and platform-level engineering. Strong software engineering capability across Pyspark, Spark SQL and Azure Data Factory. Solid understanding of DevOps practices including CI/CD and automated testing. Experience with monitoring, logging, and alerting ...