7 of 7 Permanent Spark SQL Jobs in the UK excluding London

forward-thinking data team What You'll Do: Build and maintain ETL/ELT pipelines and data models in Fabric (Data Factory, Notebooks, Spark) Write high-performance Spark SQL, T-SQL, Python/PySpark Manage ingestion, transformation, and loading from multiple sources … tools like GitHub Copilot, ChatGPT, and Fabric Copilot Essential Experience: Microsoft Fabric is essential Azure Data ecosystem Lakehouse architectures & Data Factory Python, PySpark, Spark SQL Proven hands-on delivery in this stack If you're an experienced Data Engineer with strong Microsoft Fabric and Azure experience ...

data warehousing, OLTP systems, data integration, and the software development lifecycle (SDLC). Hands-on experience with big data frameworks such as Spark, Hadoop, or Hive, including derivatives like PySpark (preferred), Spark Scala, or Spark SQL, along with familiarity with development-accelerating … libraries and frameworks. Experience using data modelling tools such as ERwin, ER/Studio, PowerDesigner, or similar. Experience with major database platforms, including SQL Server, Oracle, Azure Data Lake, Hadoop, Azure Synapse (SQL Data Warehouse), Snowflake, Redshift, and others. Strong experience with orchestration tools such ...

performance, efficiency, and cost-effectiveness. Implement data quality checks and validation rules within data pipelines. Data Transformation & Processing: Implement complex data transformations using Spark (PySpark or Scala) and other relevant technologies. Develop and maintain data processing logic for cleaning, enriching, and aggregating data. Ensure data consistency and accuracy … throughout the data lifecycle. Azure Databricks Implementation: Work extensively with Azure Databricks Unity Catalog, including Delta Lake, Spark SQL, and other relevant services. Implement best practices for Databricks development and deployment. Optimise Databricks workloads for performance and cost. Need to program using the languages such ...

specified, and tracked — not just worked around QA coverage expands naturally with each deployment increment rather than lagging behind Competencies & Behaviours Technical Strong SQL skills — comfortable writing and reading complex analytical queries (window functions, CTEs, aggregations) to interrogate data and verify correctness Hands-on experience with Databricks — running … queries, navigating Unity Catalog, reading Spark job outputs and understanding what they mean for data quality Working knowledge of PySpark or Spark SQL — enough to read pipeline code, understand transformations, and trace where data issues originate Understanding of Lakehouse/medallion architecture (bronze-silver ...

support technical decision-making * Troubleshoot complex system and data issues What you'll bring: * Deep experience building production grade data systems * Strong SQL and hands-on experience with modern data tooling and cloud platforms * Ability to work beyond data into backend engineering (APIs, services, system design) * Experience with … someone who combines deep data expertise with broader engineering capability, get in touch with Justin Donaldson for a confidential chat. Skills: dbt spark sql snowflake apache airflow cloud JBLE1 ...

with modern cloud data platforms (e.g. Databricks or Snowflake), ideally within AWS environments Advanced programming and data engineering skills in Python, PySpark and SQL, with a solid understanding of ETL pipelines … data transformation Proven experience in data modelling (end-to-end), including performance optimisation using tools such as DBT and distributed processing frameworks like SparkSQL Experience with CI/CD, version control (e.g. Git, Jenkins) and pipeline orchestration tools such as Airflow A collaborative mindset with excellent interpersonal skills ...

analytics; exposure to predictive modelling and machine learning is a plus. Proficient in SQL and Python, with bonus points for PySpark, SparkSQL, and Git. Skilled in data visualisation with tools such as Tableau or Power BI. Confident writing efficient code and troubleshooting sophisticated queries. Clear and adaptable communicator ...