19 of 19 Spark SQL Jobs in the UK excluding London

British Citizenship, or ILR (Indefinite Leave to Remain), or UK Settlement Visa. We do not provide visa sponsorship. Key Requirements:Must have skills:-Spark & Scala Nice to have skills:-Spark Streaming, Hadoop,-Hive,-SQL,-Sqoop,- Impala Detailed Job Description: At least 8+ years … fundamental Data Structures and their usage At least 8+ years of experience in designing and developing large scale, distributed data processing pipelines using Apache Spark and related technologies. Having expertise in Spark Core, Spark SQL and Spark Streaming. Experience with ...

Developing and deploying the Lakehouse platform using Databricks Building automation for cluster management, integration runtimes, and infrastructure components Leading implementation with Databricks, PySpark, Spark SQL, and Azure Data Factory Creating APIs for integration and automation Driving CI/CD, automated testing, and DevOps best practice Your … Skills and Experience You will have strong capability in: Azure and Databricks engineering PySpark, SQL, Python, and Spark SQL DevOps best practices including CI/CD, automation, and testing Infrastructure as code such as Terraform Stakeholder communication and cross-team collaboration How to Apply ...

data services across the organisation. Key Responsibilities Design, build, and maintain data pipelines using Databricks Develop ETL/ELT processes with PySpark and Spark SQL Transform and model structured and semi-structured datasets Improve performance, reliability, and cost efficiency of data workloads Ensure compliance with data … routes Required Skills & Experience Active SC Clearance - mandatory and non-negotiable Strong hands-on experience with Databricks in production environments Expert knowledge of Apache Spark , including PySpark and Spark SQL Proficiency with Python for data engineering Experience delivering solutions on cloud data platforms ...

built to last. What You'll Be Responsible For Developing and deploying scalable data engineering solutions on the Databricks Lakehouse platform using PySpark, Spark SQL, and Python Building batch pipelines that feed our Discovery Analytics platform, powered by Power BI, with accurate and reliable data Contributing … understands how good data engineering creates real value, not just technical output Solid hands-on experience with Databricks, Delta Lake, PySpark, and Spark SQL Strong Python and SQL skills with a focus on clean, maintainable, production-ready code Working knowledge of CI/ ...

Logistics. What You'll Do Lakehouse Engineering (Azure + Databricks) Build and maintain scalable ELT pipelines using Lakeflow Declarative Pipelines, PySpark and Spark SQL. Work within a Medallion architecture (Bronze Silver Gold) to deliver reliable, high-quality datasets. Ingest data from multiple sources including ChargeBee, legacy operational files … SharePoint, SFTP, SQL, REST and GraphQL APIs using Azure Data Factory and metadata-driven patterns. Apply data quality and validation rules using Lakeflow Declarative Pipelines expectations. Curated Layers & Data Modelling Develop clean and conforming Silver & Gold layers aligned to enterprise subject areas. Contribute to dimensional modelling (star schemas ...

combat travel-based issues and advice on expected interview topics/questions. Main Duties and Responsibilities Write clean and testable code using SQL and Python scripting languages, to enable our customer data products and business applications Build and manage data pipelines and notebooks, deploying code in a structured … Promote an innovative thinking process and encourage it in others. Working within the agile framework at Chambers Skills and Experience Strong proficiency in SQL , including Spark SQL and MS SQL Server , for querying, data manipulation, and performance optimization. Hands-on experience with ...

Data Warehouse capabilities. Design, manage, and optimise data storage within OneLake , ensuring well-governed, well-structured, and shareable data assets. Develop scalable Spark (PySpark/Spark SQL) data transformations through Fabric notebooks to support high-volume processing. Create, orchestrate, and monitor robust data ingestion … expecting to see: Required Skills Proven expertise across core Microsoft Fabric workloads and the unified analytics environment. Strong proficiency in Python, SQL , and Spark-based processing frameworks. Solid understanding of lakehouse architecture , including Delta/Parquet formats and data modelling. Experience designing secure, compliant, and well ...

Microsoft Fabric Knowledge: A working understanding of Lakehouse concepts (Bronze/Silver/Gold), data pipelines, and OneLake storage. Querying & Modelling: Exposure to SQL or Spark SQL for querying data models, alongside a familiarity with Star Schema principles. Platform Concepts: An understanding ...

experience with Palantir Foundry and/or Foundry Data Platform (FDP) Strong proficiency in: Python SQL Spark (PySpark or SparkSQL) Experience building production data pipelines in cloud environments (AWS, Azure, or GCP) Understanding of data modeling, schema design, and metadata management Familiarity with CI/ ...

platform operations. • Architect and design end‐to‐end data platform solutions focused on scalability, security and reliability. • Lead delivery using Databricks, PySpark, Spark SQL and Azure Data Factory. • Drive DevOps best practices including CI/CD, automated testing and infrastructure as code. • Collaborate closely with data … Skills and Experience You will bring: • Strong commercial experience in Azure and Databricks, with deep knowledge of Lakehouse architecture. • Proficiency in PySpark, Python, SQL and Spark SQL. • Strong DevOps engineering skills including CI/CD, automated testing and deployment pipelines. • Experience with monitoring, logging and alerting ...

formal training, in-house development sessions, mentoring, and opportunities to attend external gatherings such as User Groups, and of course FabCon and SQL Bits. We cover the cost of study for Microsoft certification exams to make sure you reach your full potential, and you will have personal development … based Production Data & Analytics ETL/integration solutions (Fabric, Databricks, Synapse, Data Factory, Function Apps, Power BI etc) against SLAs Python, PySpark, Scala, Spark SQL Ability to identify and investigate complex problems and implement resolutions Direct contact with customers/end-users on the phone ...

Databricks** You'll play a pivotal role in designing, building, and optimising their Azure-based Data Lakehouse, with a focus on Databricks, PySpark, Spark SQL, and Azure Data Factory. This isn't just about coding - you'll also provide architectural guidance, mentor engineers, and ensure solutions … management, integration runtimes, and networking Lead architectural design and ensure platform scalability, reliability, and governance Write efficient, maintainable code in PySpark, Python, and SQL Implement CI/CD pipelines and cloud infrastructure via Terraform/ARM Collaborate with data engineers, architects, and business stakeholders Mentor and coach engineers ...

Databricks** You'll play a pivotal role in designing, building, and optimising their Azure-based Data Lakehouse, with a focus on Databricks, PySpark, Spark SQL, and Azure Data Factory. This isn't just about coding - you'll also provide architectural guidance, mentor engineers, and ensure solutions … management, integration runtimes, and networking Lead architectural design and ensure platform scalability, reliability, and governance Write efficient, maintainable code in PySpark, Python, and SQL Implement CI/CD pipelines and cloud infrastructure via Terraform/ARM Collaborate with data engineers, architects, and business stakeholders Mentor and coach engineers ...

London or London Market insurance sector Deep understanding of Delta Lake, Medallion architecture, Lakehouse patterns and data modelling. Strong Python, PySpark and Spark SQL capability Solid understanding of data governance Proven track record of ownership, delivery and operating effectively in fast-paced environments. What ...

experience with Microsoft Fabric Experience building end-to-end Lakehouse solutions Good data modelling skills (e.g. star schema) Experience using PySpark or Spark SQL Building data pipelines from APIs, ERP or SaaS systems Working with Fabric pipelines, notebooks and Dataflows Understanding of data governance and security ...

following skills and experience: Deep expertise with Azure Databricks, including development, deployment, and platform-level engineering. Strong software engineering capability across Pyspark, Spark SQL and Azure Data Factory. Solid understanding of DevOps practices including CI/CD and automated testing. Experience with monitoring, logging, and alerting ...

following skills and experience: Deep expertise with Azure Databricks, including development, deployment, and platform-level engineering. Strong software engineering capability across Pyspark, Spark SQL and Azure Data Factory. Solid understanding of DevOps practices including CI/CD and automated testing. Experience with monitoring, logging, and alerting ...

following skills and experience: Deep expertise with Azure Databricks, including development, deployment, and platform-level engineering. Strong software engineering capability across Pyspark, Spark SQL and Azure Data Factory. Solid understanding of DevOps practices including CI/CD and automated testing. Experience with monitoring, logging, and alerting ...

SAS2PY) and manual refactoring. Pipeline Engineering: Design, build, and troubleshoot complex ETL/ELT workflows and data marts on AWS. Performance Tuning: Optimise Spark workloads for execution efficiency, partitioning, and cost-effectiveness. Quality Assurance: Implement clean coding principles, modular design, and robust unit/comparative testing to ensure …/CD integration, and comprehensive technical documentation. Technical Requirements PySpark (P3): 5+ years of hands-on experience writing scalable, production-grade PySpark/Spark SQL. AWS Data Stack (P3): Strong proficiency in EMR, Glue, S3, Athena, and Glue Workflows. SAS Knowledge (P1): Solid foundation in SAS to enable ...