8 of 8 Permanent Spark SQL Jobs in the North of England

using Microsoft Fabric • Build scalable data pipelines using Dataflow Gen2 and Fabric Notebooks • Develop and maintain Lakehouse solutions using Medallion architecture • Work with SQL, Python, Spark SQL and Delta Lake • Integrate data from APIs, databases, CRM, ERP and third party platforms • Support both batch … wider business stakeholders Essential experience • Strong commercial experience with Microsoft Fabric • Hands on experience with Dataflow Gen2, Fabric Notebooks and Lakehouse architecture • Strong SQL and Python skills • Experience with Spark and Spark SQL • Proven experience designing scalable ETL and ELT pipelines • Data ...

extensive benefits package. What you'll be doing Design, build and maintain scalable data pipelines and data products using Databricks , PySpark, Spark SQL and Delta Lake. Develop and optimise ETL/ELT processes to ingest, transform and curate data from internal and external systems. Shape enterprise … mature a growing data function. What you need to bring “Senior” level experience in data engineering, with strong hands-on Databricks skills (PySpark, Spark SQL, Delta Lake, etc) Advanced SQL and Python coding skills. A track record of building and optimising enterprise-scale data ...

platforms,lakehousesand data warehouses. Data modelling and dimensional modelling techniques such as Kimball. ETL/ELT development and large-scale data integration. PySpark, Spark SQL and modern transformation techniques. Advanced SQL and relational databases including SQL Server, PostgreSQL and Oracle. Integrating structured … semi-structured and unstructured data sources. Migration and modernisation of legacy platforms such as Oracle, SQL Server/SSIS or Synapse. Power BI semantic models, curated datasets and supporting analytics platforms. Data quality, governance, metadata management and lineage. Data security, classification and access control approaches. Automated testing, validation ...

work with a leading global IT transformation on a large-scale Government project: Data Engineer with strong Databricks, AWS, Python, PySpark and SQL skills. Hands-on Data Engineer to run + maintain production data pipelines on Databricks and AWS. Keeping scheduled data processing reliable, resolve incidents, fixes + … role combines L2/L3 production support and Data Engineering tasks. Databricks experience covering notebooks, scheduled jobs, cluster configuration, driver/executor logs, Spark UI + performance troubleshooting. Supporting + maintaining production data pipelines, including incident investigation, safe recovery, root cause analysis + permanent remediation. Experience of Spark ...

your duties, you will be responsible for: Design, build and deploy scalable data pipelines using Azure Databricks , Azure Data Factory and Azure SQL Database . Optimise Databricks environments for performance, cost and security. Develop and maintain data integration , ETL/ELT processes and data quality frameworks. Translate retail … environments. Deep expertise in Azure Databricks, Azure Data Factory, Azure Synapse and Azure Data Lake. Hands-on experience with Spark (PySpark/SparkSQL), ETL/ELT design and data pipeline orchestration. Strong data modelling skills (Dimensional, ODS, Data Vault) and experience with data warehousing concepts. Experience working with ...

migrating and transforming data from legacy and on-premise systems to cloud platforms. Expertise in automating ETL/ELT pipelines using PySpark and Spark SQL across Azure-based data environments. You and your role As a Data Engineer, you will design build and enhance scalable ...

migrating and transforming data from legacy and on-premise systems to cloud platforms. Expertise in automating ETL/ELT pipelines using PySpark and Spark SQL across Azure-based data environments. You and your role As a Data Engineer, you will design build and enhance scalable ...

migrating and transforming data from legacy and on-premise systems to cloud platforms. Expertise in automating ETL/ELT pipelines using PySpark and Spark SQL across Azure-based data environments. You and your role As a Data Engineer, you will design build and enhance scalable ...