13 of 13 Spark SQL Jobs in England

Data Engineer (Python Spark SQL) *Newcastle Onsite* to £120k Do you have a first class education combined with Data Engineering skills? You could be progressing your career at a start-up Investment Management firm that have secure backing, an established Hedge Fund client as a partner … minimum A A B grades at A-level You have commercial Data Engineering experience working with technologies such as SQL, Apache Spark and Python including PySpark and Pandas You have a good understanding of modern data engineering best practices Ideally you will also have experience with ...

Data Engineer (Python Spark SQL) *Newcastle Onsite* to £120k Do you have a first class education combined with Data Engineering skills? You could be progressing your career at a start-up Investment Management firm that have secure backing, an established Hedge Fund client as a partner … minimum A A B grades at A-level You have commercial Data Engineering experience working with technologies such as SQL, Apache Spark and Python including PySpark and Pandas You have a good understanding of modern data engineering best practices Ideally you will also have experience with ...

hands-on, focused on building and optimising large-scale data pipelines, dataflows, semantic models, and lakehouse components. Key Responsibilities Design, build and optimise Spark-based data pipelines for batch and streaming workloads Develop Fabric dataflows, pipelines, and semantic models Implement complex transformations, joins, aggregations and performance tuning Build … translate requirements into scalable technical solutions Troubleshoot and improve reliability, latency and workload performance Essential Skills Strong hands-on experience with PySpark, Spark SQL, Spark Streaming, DataFrames Microsoft Fabric (Fabric Spark jobs, dataflows, pipelines, semantic models) Azure: ADLS, cloud data engineering, notebooks ...

Microsoft Fabric architectures using the Medallion pattern. Building metadata-driven ingestion pipelines and optimising Delta Lake performance. Writing advanced PySpark/Spark SQL notebooks for large-scale transformations. Developing semantic models for enterprise reporting and enabling Direct Lake. Leading client workshops and guiding technical decisions. Implementing … Looking For Hands-on experience with Microsoft Fabric workloads: Lakehouse, Data Factory, Pipelines, Notebooks, Delta Lake, Eventstreams, Semantic Models. Advanced PySpark/Spark SQL skills and strong data engineering fundamentals. Integration experience with Dynamics 365, Dataverse, Business Central. CI/CD expertise with Azure DevOps/ ...

data architectures using Medallion (Bronze/Silver/Gold) patterns Build metadata-driven ingestion pipelines and transformation frameworks Develop advanced PySpark/Spark SQL notebooks for data cleansing and modelling Create production-ready semantic models and support BI teams Implement governance, security, and CI/… business requirements into technical solutions Essential Skills Strong experience with Microsoft Fabric workloads (Lakehouse, Data Factory, Pipelines, Notebooks, Semantic Models) Advanced PySpark/Spark SQL for large-scale transformations Integration with Dynamics 365, Dataverse, and/or Business Central CI/CD implementation using Azure DevOps ...

modeling, ETL/ELT development, and collaborative engineering practices. Key Responsibilities * Design, develop, and maintain scalable data pipelines using Azure Databricks (Python, PySpark, SQL). * Build and optimize ETL/ELT workflows that ingest data from various on-prem and cloud-based sources. * Work with Azure services including … Azure Data Lake Storage, Azure Data Factory, Azure Synapse Analytics, Azure SQL, and Event Hub. * Implement data quality validation, monitoring, metadata management, and governance processes. * Collaborate closely with data architects, analysts, and business stakeholders to understand data requirements. * Optimize Databricks clusters, jobs, and runtimes for performance and cost ...

data sources into Azure Databricks. Optimize pipelines for performance, reliability, and cost, incorporating data quality checks. Develop complex transformations and processing logic using Spark (PySpark/Scala) for cleaning, enrichment, and aggregation, ensuring accuracy and consistency across the data lifecycle. Work extensively with Unity Catalog, Delta Lake, Spark SQL, and related services. Apply best practices for development, deployment, and workload optimization. Program in SQL, Python, R, YAML, and JavaScript. Integrate data from relational databases, APIs, and streaming sources using best-practice patterns. Collaborate with API developers for seamless data exchange. Utilize Azure ...

pipelines to ingest, transform, and load data from diverse sources (APIs, databases, files) into Azure Databricks. Implement data cleaning, validation, and enrichment using Spark (PySpark/Scala) and related tools to ensure quality and consistency. Utilize Unity Catalog, Delta Lake, Spark SQL, and best … practices for Databricks development, optimization, and deployment. Program in SQL, Python, R, YAML, and JavaScript. Integrate data from multiple sources and formats (CSV, JSON, Parquet, Delta) for downstream analytics, dashboards, and reporting. Apply Azure Purview for governance and quality checks. Monitor pipelines, resolve issues, and enhance data quality ...

Title: SQL Data Architect - Azure Stack Location: Remote with occasional travel to Birmingham Salary/Rate: Up to £600 per day inside IR35 Start Date: 12/01/2026 Job Type: Contract - 12 months Company Introduction: We have an exciting opportunity now available with … sector-leading consultancy clients! They are currently looking for a skilled SQL Data Architect to join their team for a twelve-month contract. Required Skills/Experience: The ideal candidate will have the following: Extensive experience in data engineering/DBA/architecture with a focus on Microsoft ...

plans and lead the development of data processing workflows. From a technical standpoint you must possess: Extensive Databricks experience including Unity Catolog and Spark SQL Strong programming skills, preferably in Python and SQL Strong knowledge and experience in Azure, including working with Azure Data ...

house team, this role could be the next step for you. Data Engineer Responsibilities: Design, build, and optimise data pipelines in Python, PySpark, SparkSQL, and Databricks. Ingest, transform, and enrich structured, semi structured, and unstructured data. Operate and support production grade data systems with strong observability and monitoring. Enable real ...

including pension scheme and PMI Role and Responsibilities Design, build, and deploy scalable data pipelines using Azure Databricks, Azure Data Factory, and Azure SQL Database Optimise Databricks environments for performance, cost efficiency, and security Develop and maintain ETL/ELT processes and data quality frameworks Translate business requirements … environments Deep expertise in Azure Databricks, Azure Data Factory, Azure Synapse, and Azure Data Lake Hands-on experience with Spark (PySpark/SparkSQL) and distributed data processing Strong data modelling skills (Dimensional, ODS, Data Vault) and data warehousing concepts Proficiency in SQL, version control ...

analytics; exposure to predictive modelling and machine learning is a plus. Proficient in SQL and Python, with bonus points for PySpark, SparkSQL, and Git. Skilled in data visualisation with tools such as Tableau or Power BI. Confident writing efficient code and troubleshooting sophisticated queries. Clear and adaptable communicator ...