14 of 14 Spark SQL Jobs in the UK

one. What You'll Be Doing Lakehouse Engineering (Azure + Databricks) Engineer scalable ELT pipelines using Lakeflow Declarative Pipelines , PySpark , and Spark SQL across a full Medallion Architecture (Bronze Silver Gold) . Implement ingestion patterns for files, APIs, SaaS platforms (e.g. subscription billing), SQL … SLIs, runbooks and cost-optimisation across the platform. DevOps & Platform Engineering Build CI/CD pipelines in Azure DevOps for notebooks, Lakeflow pipelines, SQL models and ADF artefacts. Ensure secure, enterprise-grade platform operation across Dev Prod , using private endpoints, managed identities and Key Vault. Contribute to platform ...

firefighting? You will design, build and maintain ETL and ELT pipelines , lakehouse structures and semantic models using Microsoft Fabric's Dataflows Gen2, Notebooks, Spark SQL and Python. Your work will ensure data is accessible, trusted and well governed across hybrid on‐prem and cloud environments. … Essential Strong experience designing and operating scalable ETL/ELT pipelines. Hands‐on Microsoft Fabric experience (Dataflows Gen2, Notebooks, semantic models). SQL and Python proficiency, with Spark/Spark SQL exposure. Practical understanding of data quality, observability and troubleshooting. Ability ...

combat travel-based issues and advice on expected interview topics/questions. Main Duties and Responsibilities Write clean and testable code using SQL and Python scripting languages, to enable our customer data products and business applications Build and manage data pipelines and notebooks, deploying code in a structured … Promote an innovative thinking process and encourage it in others. Working within the agile framework at Chambers Skills and Experience Strong proficiency in SQL , including Spark SQL and MS SQL Server , for querying, data manipulation, and performance optimization. Hands-on experience with ...

through to modelling and optimisation — working closely with analytics, reporting, and business stakeholders. Key Responsibilities: Design, build, and maintain data pipelines using Databricks (Spark/PySpark/SQL). Support the migration of on-premise data sources to a cloud-based Databricks platform. Develop and optimise … planning, delivery, and issue tracking using Jira. Help establish and promote data engineering standards, governance, and documentation. Tech Stack & Tools: Core Technologies: Databricks Apache Spark (PySpark/Spark SQL) Delta Lake SQL Focus Areas Databricks platform development Cloud data migration Data modelling ...

Your Profile Essential Skills/Knowledge/Experience Strong working and technical knowledge of GCP (GCS Buckets, BigQuery, Dataproc, Pub/Sub), Airflow, Spark, PySpark, Spark SQL, Hive. Good knowledge of Python (Pandas, NumPy, Wiresafe), SQL, Linux CLI, BigQuery. Experience with DBMS … Delta Tables, Oracle, MS Access, SQL Server. Skills in Data Modelling, ETL/ELT Data Pipelines, Python scripting, Data Warehousing. Desirable Skills/Knowledge/Experience Experience in production support, troubleshooting data pipeline issues, and ensuring timely and accurate business reporting. Expertise in Data Modelling, ETL/ ...

domain experience is beneficial but not essential. Key Responsibilities Design and deliver scalable data pipelines using Azure Databricks, Azure Data Factory, and Azure SQL Build and maintain ETL/ELT processes and data quality frameworks Develop consistent data models and analytics-ready datasets Partner with stakeholders to deliver … years’ experience in data engineering within enterprise environments Strong expertise in Azure Databricks, ADF, Synapse, and Azure Data Lake Hands-on experience with Spark (PySpark/Spark SQL) and orchestration Strong data modelling and data warehousing knowledge Proficiency in SQL ...

products go to production. What are the key skills/experience you'll already have? 3+ years commercial experience in Data Engineering SQL Server Spark SQL Strong Python coding skills PySpark Azure experience including Azure SQL, Data Factory and Functions Experience collaborating ...

using Delta Lake principles Ingest, transform, and curate data from multiple sources (APIs, databases, files, streaming) Develop scalable data transformations using PySpark and Spark SQL Implement data models optimized for analytics and reporting (e.g. star schemas) Monitor, troubleshoot, and optimize performance and cost of data workloads … Experience Strong experience with Microsoft Fabric (Lakehouse, Pipelines, Notebooks, Dataflows, OneLake) Hands-on experience with Databricks in production environments Proficiency in PySpark and SQL Solid understanding of data engineering concepts (ETL/ELT, orchestration, partitioning) Experience working with Delta Lake Familiarity with cloud platforms (Azure preferred) Experience integrating ...

Experience 5+ years' experience working as a Data Engineer or in a similar role Proven experience with cloud-based data platforms (Azure, AWS, SQL, Snowflake, Springserv); Microsoft Fabric experience is a strong plus Strong proficiency in Spark SQL and PySpark , including complex transformations Experience ...

Analytics Team with an exciting plan and long-term vision. We are seeking an experienced Data Engineer, with skills in Microsoft Azure and SQL, to take the contribute to the design, build and deployment of our Azure data platform in line with our enterprise architecture. Key Outputs Transform …/or Databricks. Strong foundation in data modelling and data warehousing concepts (star/snowflake schemas, SCD2, Dimensions/Facts) Proficient in SQL and Python, with hands-on experience using PySpark, Spark SQL, and/or Pandas. Experience in technical documentation of migration processes ...

platform. You'll play a pivotal role in designing, building, and optimising their Azure-based Data Lakehouse, with a focus on Databricks, PySpark, Spark SQL, and Azure Data Factory. This isn't just about coding - you'll also provide architectural guidance, mentor engineers, and ensure solutions … management, integration runtimes, and networking Lead architectural design and ensure platform scalability, reliability, and governance Write efficient, maintainable code in PySpark, Python, and SQL Implement CI/CD pipelines and cloud infrastructure via Terraform/ARM Collaborate with data engineers, architects, and business stakeholders Mentor and coach engineers ...

curve. Skills and Experience Proven experience as a Senior or Lead Data Engineer. Hands-on experience with Databricks and PySpark/Spark SQL. Strong expertise with the Azure data platform (Data Factory, Synapse, etc.). Solid understanding of data architecture and performance optimisation. What is on offer Salary ...

processes. Skills and Experience Proven experience as a Senior or Lead Data Engineer. Strong hands-on expertise with Databricks and PySpark/Spark SQL. Deep knowledge of the Azure data platform (Data Factory, Synapse, etc.). Solid understanding of data architecture principles and performance optimisation. Experience managing ...

workflows. From a technical standpoint you will ideally possess/or aspire to learn: Knowledge or experience of Databricks including Unity Catolog and Spark SQL. Programming skills, preferably in Python and SQL. Knowledge and experience in Azure, including working with Azure Data Factory and Azure Storage Accounts. Knowledge ...