1 to 25 of 98 Remote/Hybrid PySpark Jobs in the UK

Good understanding of modern software platform architecture and the role of the data platform within the broader platform ecosystem Good Scala skills (or strong PySpark plus a genuine willingness to work primarily in Scala) Strong SQL skills for transformations, data modelling, and debugging data issues Experience building and supporting ...

using Azure Databricks. Build robust ETL/ELT solutions using Azure Data Factory and Microsoft Fabric. Develop high-performance data processing solutions using Python, PySpark, and SQL. Implement Delta Lake and Lakehouse architectures following industry best practices. Work with Azure Data Lake Storage (ADLS Gen2) to manage large-scale ...

standards and best practices. About you Extensive experience designing and delivering enterprise-scale data engineering solutions Strong hands-on experience with Azure Databricks, including PySpark, Delta Live Tables and Databricks Workflows and orchestration Microsoft Fabric experience, including Lakehouse, Data Factory, Warehouse and Spark Knowledge of Azure Data Platform services ...

understanding on Master Data Services, DQ profiling, cleansing, update, DQ reports, Purview. Strong experience with at least one of the programming languages (Python, Scala, PySpark). Knowledge and experience using the Microsoft technology stack and associated development tools. Experience working with SharePoint, PowerPivot, PowerBI and deep knowledge ...

e.g. DAMA) • Strong expertise in Microsoft Azure, Fabric, Microsoft 365 platform • Fluency in one or more of the following programming languages (SQL, Python (pandas, PySpark, REST API integration), M Power Query, DAX, JavaScript) • Proven experience as a Data Architect, Data Engineer, or similar role. • Extensive experience with database management ...

data quality best practice.Strong problem-solving skills and attention to detail.The ability to communicate effectively with both technical and business stakeholders.Desirable ExperiencePython or PySpark development within Microsoft Fabric.CI/CD practices for data engineering and data product delivery.Experience within legal services or another professional services environment.Exposure to systems such ...

best practice. Strong problem-solving skills and attention to detail. The ability to communicate effectively with both technical and business stakeholders. Desirable ExperiencePython or PySpark development within Microsoft Fabric. CI/CD practices for data engineering and data product delivery. Experience within legal services or another professional services environment. ...

skills (Conceptual, Logical, Physical and Dimensional)Extensive Azure Databricks experience within large-scale enterprise environmentsDatabricks Lakehouse Architecture and Medallion Design PatternsDatabricks Data Engineering using PySpark and Spark SQLDelta Live Tables (DLT)Unity Catalog implementation and governanceDatabricks WorkflowsDatabricks Asset Bundles (DABs)Databricks Solution AcceleratorsMetadata-Driven Ingestion and Orchestration FrameworksMLflow, Feature ...

following: OneLake Fabric Lakehouse Fabric Warehouse Data Factory Notebooks Power BI Semantic Models Dataflows ✅ Advanced SQL development skills ✅ Python and/or PySpark experience ✅ Strong understanding of dimensional modelling and data warehousing concepts ✅ Experience building enterprise-scale data integration and pipeline solutions ✅ Strong Power BI development expertise, including ...

Skills Significant experience leading the design, development, and operation of enterprise-scale data engineering platforms and teams. Strong technical expertise in SQL and Python, PySpark, or equivalent technologies for data ingestion, transformation, and validation. Experience with Microsoft Fabric or similar modern cloud-based analytics and data platforms. Strong experience ...

Skills Significant experience leading the design, development, and operation of enterprise‐scale data engineering platforms and teams. Strong technical expertise in SQL and Python, PySpark, or equivalent technologies for data ingestion, transformation, and validation. Experience with Microsoft Fabric or similar modern cloud‐based analytics and data platforms. Strong experience ...

analytics strategy AI & Automation Agentic AI patterns and intelligent automation solutions Document intelligence and AI-driven business use cases SQL Python and/or PySpark Data pipelines and transformation frameworks CI/CD concepts and deployment automation Azure DevOps and/or GitHub Infrastructure as Code awareness Environment promotion ...

contribute to long-term platform innovation Required Experience Strong experience designing enterprise data platforms within Microsoft Azure environments Deep expertise with Azure Databricks, Spark (PySpark/Scala), Delta Lake, and Lakehouse architecture Strong hands-on experience with Snowflake architecture and optimisation Experience with Azure Data Factory, ADLS Gen2, Synapse ...

APIs, Prometheus, or Azure Monitor.• Maintain best practices for data engineering performance, testing, and documentation.Technical Skills Required• Strong proficiency with Databricks, Apache Spark (SQL, PySpark, Scala), and Delta Lake.• Deep understanding of Spark execution plans, shuffle optimization, and data partitioning strategies.• Experience tuning job configurations (e.g., memory management, parallelism ...

Looking ForProven hands-on experience with the Databricks platform, including orchestration and enterprise-scale Azure data engineering solutions.Strong technical skills in Python, PySpark, and SQL, with experience working with distributed data processing frameworks.Expertise delivering end-to-end data pipelines across ingestion, transformation, and serving layers within modern Data Lakehouse ...

Looking ForProven hands-on experience with the Databricks platform, including orchestration and enterprise-scale Azure data engineering solutions. Strong technical skills in Python, PySpark, and SQL, with experience working with distributed data processing frameworks. Expertise delivering end-to-end data pipelines across ingestion, transformation, and serving layers within modern ...

storage Proficiency with Terraform (ideally the Databricks provider) and Gitbased CI/CD workflows Solid programming skills in Python and SQL; comfortable with PySpark and shell scripting Experience tuning Spark jobs and Delta tables (partitioning, Working knowledge of SSO/SCIM, identity providers and secretsmanagement Understanding of data security ...

Exposure to metadata management, data governance, and data quality frameworks Experience with CI/CD (Azure DevOps, Gi t) and ideally scripting (Python/PySpark) Job Offer £700-£800 per day (Inside IR35) Hybrid working (2-3 days in London) Initial 6-month contract with extension potential ...

OneLake, pipelines and notebooks, and how these can be used to build scalable enterprise data solutions. Strong experience with SQL, Python and/or PySpark and modern data engineering approaches. Strong understanding of data architecture and data modelling, including designing reusable and scalable data structures. Experience implementing modern software ...

Data Stack, critically ADF and Synapse (experience with Microsoft Fabric is a plus) Highly developed python and data pipeline development knowledge, must include substantial PySpark experience Demonstrable DevOps and DataOps experience with an understanding of best practices for engineering, test and ongoing service delivery An understanding of Infrastructure ...

Strong hands-on experience with Quantexa implementation and development. Proven experience building and supporting big data platforms . Advanced knowledge of Apache Spark/PySpark . Strong programming skills in Scala and/or Python . Expertise in SQL and large-scale data modelling. Strong cloud experience across ...

particularly interested in engineers with strong hands-on Databricks experience. Essential Skills Commercial experience building solutions within Databricks. Strong hands-on expertise with PySpark, Spark SQL, and Delta Lake. Experience designing and supporting production-grade ETL/ELT pipelines. Strong Python and SQL development skills. Experience working with large ...

particularly interested in engineers with strong hands-on Databricks experience. Essential Skills Commercial experience building solutions within Databricks. Strong hands-on expertise with PySpark, Spark SQL, and Delta Lake. Experience designing and supporting production-grade ETL/ELT pipelines. Strong Python and SQL development skills. Experience working with large ...

Data Engineering Foundation: Practical experience with data modelling, ETL/ELT, and various database technologies. Technical Versatility: Fluency in data processing languages (Python, SQL, PySpark) and openness to tech stacks such as SSIS, Airflow, Spark, Databricks, Fabric. Cloud Capability: Experience with at least one major cloud provider (AWS, Azure ...

Looking ForProven expertise with the Databricks platform, including Unity Catalog, Delta Lake, orchestration, and enterprise-scale Azure deployments.Strong technical skills in Python, PySpark, and distributed data processing frameworks, with experience building production-grade data platforms.Extensive experience implementing CI/CD pipelines using Azure DevOps, Jenkins, GitHub Actions, or equivalent ...