22 of 22 Permanent Spark SQL Jobs in the UK excluding London

including machine learning and their application in financial services. Proficiency in data tools, programming languages, and database technologies, in particular in Azure Databricks, Spark, Spark SQL and Python/pyspark. Understanding of data governance, security, and data quality management principles. Experience working with large … producing audit evidence Essential Skills: Data pipeline development and optimisation Data modelling and architecture (in Azure Databricks) Programming and scripting for data processing (Spark, Spark SQL, Python, pyspark) Data science and machine learning Data governance and quality assurance Problem-solving and analytical thinkingStakeholder engagement ...

including machine learning and their application in financial services.* Proficiency in data tools, programming languages, and database technologies, in particular in Azure Databricks, Spark, Spark SQL and Python/pyspark.* Understanding of data governance, security, and data quality management principles.* Experience working with large … producing audit evidence**Essential Skills:*** Data pipeline development and optimisation* Data modelling and architecture (in Azure Databricks)* Programming and scripting for data processing (Spark, Spark SQL, Python, pyspark)* Data science and machine learning* Data governance and quality assurance* Problem-solving and analytical thinking* Stakeholder ...

using Microsoft Fabric • Build scalable data pipelines using Dataflow Gen2 and Fabric Notebooks • Develop and maintain Lakehouse solutions using Medallion architecture • Work with SQL, Python, Spark SQL and Delta Lake • Integrate data from APIs, databases, CRM, ERP and third party platforms • Support both batch … wider business stakeholders Essential experience • Strong commercial experience with Microsoft Fabric • Hands on experience with Dataflow Gen2, Fabric Notebooks and Lakehouse architecture • Strong SQL and Python skills • Experience with Spark and Spark SQL • Proven experience designing scalable ETL and ELT pipelines • Data ...

using Microsoft Fabric • Build scalable data pipelines using Dataflow Gen2 and Fabric Notebooks • Develop and maintain Lakehouse solutions using Medallion architecture • Work with SQL, Python, Spark SQL and Delta Lake • Integrate data from APIs, databases, CRM, ERP and third party platforms • Support both batch … wider business stakeholders Essential experience • Strong commercial experience with Microsoft Fabric • Hands on experience with Dataflow Gen2, Fabric Notebooks and Lakehouse architecture • Strong SQL and Python skills • Experience with Spark and Spark SQL • Proven experience designing scalable ETL and ELT pipelines • Data ...

reporting, analytics, and operational decision-making across the project programme. Working within a multidisciplinary Data & Analytics team, the successful candidate will use Databricks, SQL, Python, and Microsoft technologies to deliver robust, scalable, and maintainable data solutions. They will collaborate closely with Data Architects and Performance Analysts to transform … business intelligence solutions. Strong proven commercial experience with Databricks. Strong experience developing ETL/ELT pipelines and data integration solutions. Experience using Apache Spark, PySpark, and Spark SQL. Strong SQL development and optimisation skills. Experience working with large and complex datasets from multiple source ...

scalable Data Engineering solutions. Experience of core MS Fabric features including: OneLake, Lakehouses, Warehouses, Data Pipelines, Dataflows, Gen2, Notebooks, Eventstreams + Eventhouses. Using SQL, T-SQL, Spark SQL, PySpark + Python for data ingestion, data transformation, data optimisation, data modelling, and converting ...

scalable Data Engineering solutions. Experience of core MS Fabric features including: OneLake, Lakehouses, Warehouses, Data Pipelines, Dataflows, Gen2, Notebooks, Eventstreams + Eventhouses. Using SQL, T-SQL, Spark SQL, PySpark + Python for data ingestion, data transformation, data optimisation, data modelling, and converting ...

extensive benefits package. What you'll be doing Design, build and maintain scalable data pipelines and data products using Databricks , PySpark, Spark SQL and Delta Lake. Develop and optimise ETL/ELT processes to ingest, transform and curate data from internal and external systems. Shape enterprise … mature a growing data function. What you need to bring “Senior” level experience in data engineering, with strong hands-on Databricks skills (PySpark, Spark SQL, Delta Lake, etc) Advanced SQL and Python coding skills. A track record of building and optimising enterprise-scale data ...

extensive benefits package. What you'll be doing Design, build and maintain scalable data pipelines and data products using Databricks , PySpark, Spark SQL and Delta Lake. Develop and optimise ETL/ELT processes to ingest, transform and curate data from internal and external systems. Shape enterprise … mature a growing data function. What you need to bring “Senior” level experience in data engineering, with strong hands-on Databricks skills (PySpark, Spark SQL, Delta Lake, etc) Advanced SQL and Python coding skills. A track record of building and optimising enterprise-scale data ...

extensive benefits package. What you'll be doing Design, build and maintain scalable data pipelines and data products using Databricks , PySpark, Spark SQL and Delta Lake. Develop and optimise ETL/ELT processes to ingest, transform and curate data from internal and external systems. Shape enterprise … mature a growing data function. What you need to bring “Senior” level experience in data engineering, with strong hands-on Databricks skills (PySpark, Spark SQL, Delta Lake, etc) Advanced SQL and Python coding skills. A track record of building and optimising enterprise-scale data ...

layers of the medallion architecture. Onboard new source systems by setting up the team’s reusable ingestion patterns (Fabric mirroring from Snowflake and SQL Server, change data capture, file and API loads). Do this through configuration rather than one-off code. Write and tune PySpark … SQL for Lakehouse and Warehouse workloads, including fact and dimension models in the enterprise data warehouse. Implement data quality rules and validation pipelines, and help investigate and fix data issues found by DQ checks and reconciliations. Work through the team’s engineering workflow: Git-based development in Azure ...

data-driven culturewithin the organisation, mentoring team members, and contributing to the continuous improvement of the Engineering Practice You Will Demonstrate: Proficiencyin SQL and Python:You are highly proficient in SQL and Python, enabling you to handle complex data problems with ease. Understanding of Data Lakehouse … Architecture:You have a strong grasp of the principles and implementation of Data Lakehouse architecture. Hands-On Experience with Spark-Based Solutions:Youpossessexperience with Spark-based platforms like Azure Synapse, Databricks, Microsoft Fabric, or evenon-premiseSpark clusters, usingPySparkor Spark SQL to manage ...

3+ years with strong Foundry depth). Proven, hands-on experience working with Palantir Foundry in a production environment. Strong proficiency in Python, SQL, PySpark and Spark SQL Experience delivering production pipelines in AWS, Azure, or GCP environments. Solid understanding of data modeling, schema ...

lakehouses and data warehouses. Data modelling and dimensional modelling techniques such as Kimball. ETL/ELT development and large-scale data integration. PySpark, Spark SQL and modern transformation techniques. Advanced SQL and relational databases including SQL Server, PostgreSQL and Oracle. Integrating structured … semi-structured and unstructured data sources. Migration and modernisation of legacy platforms such as Oracle, SQL Server/SSIS or Synapse. Power BI semantic models, curated datasets and supporting analytics platforms. Data quality, governance, metadata management and lineage. Data security, classification and access control approaches. Automated testing, validation ...

3+ years with strong Foundry depth). Proven, hands‐on experience working with Palantir Foundry in a production environment. Strong proficiency in Python, SQL, PySpark and Spark SQL Experience delivering production pipelines in AWS, Azure, or GCP environments. Solid understanding of data modeling, schema ...

team. This role is central to delivering Canopius’ data strategy through development of new data capabilities and supporting the final transition from legacy SQL Server-based environments. Alongside hands-on delivery, the role provides technical leadership across data engineering initiatives, helping to establish a future-ready, cloud-native … secure, scalable, maintainable, and aligned with organisational standards and governance requirements. Design, build, and optimise data pipelines on Databricks using PySpark and Spark SQL, applying Delta Lake and a medallion/Lakehouse architecture. Implement data validation and reconciliation checks, and automated testing across pipelines to ensure ...

design and deliver cutting-edge data solutions for customers operating in some of the UK's most complex and challenging environments. Using Databricks, Apache Spark and cloud-native technologies, you’ll develop scalable Lakehouse architectures and production-grade data pipelines that enable organisations to trust, share and exploit … grow your career while helping customers unlock the power of their data. Responsibilities Design, build and optimise scalable data pipelines using Databricks, Apache Spark, PySpark and Delta Lake. Develop and implement modern Lakehouse architectures, applying Medallion (Bronze, Silver, Gold) design principles to support trusted and reusable data products. ...

your duties, you will be responsible for: Design, build and deploy scalable data pipelines using Azure Databricks , Azure Data Factory and Azure SQL Database . Optimise Databricks environments for performance, cost and security. Develop and maintain data integration , ETL/ELT processes and data quality frameworks. Translate retail … environments. Deep expertise in Azure Databricks, Azure Data Factory, Azure Synapse and Azure Data Lake. Hands-on experience with Spark (PySpark/SparkSQL), ETL/ELT design and data pipeline orchestration. Strong data modelling skills (Dimensional, ODS, Data Vault) and experience with data warehousing concepts. Experience working with ...

related field. Applied experience in data science, machine learning, or related areas. Strong programming skills in Python with experience in PySpark, Spark SQL, and manipulating data. Experience LLM Orchestration and Agentic AI libraries Proficiency in using GenAI models (OpenAI or similar) to solve business problems. Experience ...

/CD pipelines, version control, and DevOps practices related to data engineering. Proficiency in SQL (T-SQL/SparkSQL) for data manipulation, transformation, and querying. Solid understanding and practical application of Data Vault and Kimball data modelling methodologies. Experience designing and optimising ETL/ELT workflows ...

will lead a small team, mentor junior engineers, and collaborate with stakeholders to deliver scalable data pipelines on Databricks using PySpark and Spark SQL. The role involves guiding data strategy, migrating legacy processes, and delivering enterprise data solutions in a cloud-native environment, with hybrid #J-18808-Ljbffr ...

shape our core modeling practices. Your qualifications and experience ESSENTIAL Pro at using SQL for data manipulation (PostgreSQL, MSSQL, Google BigQuery, SparkSQL, etc.) Strong coding skills in Python, R, Scala, C, Java, or JS Proven track record of using data manipulation and machine learning libraries Practical modelling & statistical … apprenticeship/work experience in a subject such as Mathematics, Statistics, Computer Science, or Geography DESIRABLE Exposure to Big Data Analytics (BigQuery, Apache Spark), Parallel Computing (Kubernetes, Databricks), Cloud Engineering (AWS, GCP, Azure), Spatial Query Optimisation, Graph Computing, or Microservices Relevant industry sector knowledge across retail, property ...