15 of 15 Permanent Spark SQL Jobs in the UK

considering: We are looking for an experienced AWS Databricks Engineer with strong hands-on expertise in Databricks, AWS cloud services, PySpark, Spark SQL, Delta Lake, Python, SQL, and data engineering. The candidate will be responsible for designing, developing, optimizing, and supporting scalable data pipelines … support code reviews, design reviews, and engineering best practices. Your Skills: Strong hands-on experience with Databricks on AWS. Strong experience with Apache Spark/PySpark. Excellent programming skills in Python. Strong SQL skills including complex queries, joins, CTEs, window functions, and query optimization. Experience with ...

biggest investment banks. Key responsibilities: Design and Develop Data Pipelines Build scalable, high-performance data pipelines using Databricks, PySpark, and SQL . Develop and optimize ETL/ELT workflows for batch and streaming data processing. Lead Cloud Migration Initiatives Drive and support the migration of on-premises data … cloud environments (preferably Azure). Refactor legacy data pipelines and ensure seamless transition to modern cloud-based architectures. Convert and optimize legacy SQL dialects into Spark SQL for improved performance and scalability. Data Platform Engineering Develop and manage data solutions within the Databricks Lakehouse ...

forward-thinking data team What You'll Do: Build and maintain ETL/ELT pipelines and data models in Fabric (Data Factory, Notebooks, Spark) Write high-performance Spark SQL, T-SQL, Python/PySpark Manage ingestion, transformation, and loading from multiple sources … tools like GitHub Copilot, ChatGPT, and Fabric Copilot Essential Experience: Microsoft Fabric is essential Azure Data ecosystem Lakehouse architectures & Data Factory Python, PySpark, Spark SQL Proven hands-on delivery in this stack If you're an experienced Data Engineer with strong Microsoft Fabric and Azure experience ...

data warehousing, OLTP systems, data integration, and the software development lifecycle (SDLC). Hands-on experience with big data frameworks such as Spark, Hadoop, or Hive, including derivatives like PySpark (preferred), Spark Scala, or Spark SQL, along with familiarity with development-accelerating … libraries and frameworks. Experience using data modelling tools such as ERwin, ER/Studio, PowerDesigner, or similar. Experience with major database platforms, including SQL Server, Oracle, Azure Data Lake, Hadoop, Azure Synapse (SQL Data Warehouse), Snowflake, Redshift, and others. Strong experience with orchestration tools such ...

modernisation of existing data estates. 🔧 Key Responsibilities Design and build scalable data pipelines using Databricks Develop and optimise ETL/ELT workflows using Spark (PySpark/SQL) Support migration and modernisation of legacy data platforms into Databricks Improve performance, reliability and cost efficiency across data workloads … best practices across delivery ✅ What We’re Looking For Strong hands-on experience with Databricks in a Data Engineering capacity Solid experience with Spark (PySpark/Spark SQL) Proven track record building and optimising data pipelines (ETL/ELT) Experience working within cloud environments ...

considering: We are looking for an experienced AWS Databricks Engineer with strong hands-on expertise in Databricks, AWS cloud services, PySpark, Spark SQL, Delta Lake, Python, SQL, and data engineering. The candidate will be responsible for designing, developing, optimizing, and supporting scalable data pipelines … architecture review boards, design authority meetings, and client stakeholder workshops. Your Skills: Strong hands-on experience with Databricks on AWS. Strong experience with Apache Spark/PySpark. Excellent programming skills in Python. Strong SQL skills including complex queries, joins, CTEs, window functions, and query optimization. ...

performance, efficiency, and cost-effectiveness. Implement data quality checks and validation rules within data pipelines. Data Transformation & Processing: Implement complex data transformations using Spark (PySpark or Scala) and other relevant technologies. Develop and maintain data processing logic for cleaning, enriching, and aggregating data. Ensure data consistency and accuracy … throughout the data lifecycle. Azure Databricks Implementation: Work extensively with Azure Databricks Unity Catalog, including Delta Lake, Spark SQL, and other relevant services. Implement best practices for Databricks development and deployment. Optimise Databricks workloads for performance and cost. Need to program using the languages such ...

platforms (AWS, Azure, GCP). • Develop & Optimise robust data models, warehouses and 'big data' architectures. • Integrate & Transform data using tools such as Databricks, Spark, Kafka, Talend, Informatica, AWS Glue or Azure Data Factory. • Collaborate & Influence with cross-functional agile teams and client stakeholders. • Automate & Secure CI/… this role. Essential qualifications: • You should have hands-on experience with cloud platforms such as AWS, Azure or GCP. • Strong proficiency in Python, SQL, Scala, Spark-SQL, JavaScript or PowerShell. • Proven ability to design and build data pipelines and workflow solutions. • Experience with data ...

modelling, dimensional modelling, canonical data models, and enterprise data models. Strong experience in designing ETL/ELT data pipelines. Good understanding of PySpark, Spark SQL, Python, and SQL. Strong knowledge of data warehouse, data lake, data mart, and semantic layer architecture. Experience in defining data quality ...

track record of delivering impactful solutions • Proven experience developing and deploying machine learning models and statistical analysis • Strong proficiency in Python and SQL, with experience in Spark SQL • Familiarity with cloud platforms and distributed computing tools such as Azure Synapse, Snowflake, or Databricks • Ability ...

specified, and tracked — not just worked around QA coverage expands naturally with each deployment increment rather than lagging behind Competencies & Behaviours Technical Strong SQL skills — comfortable writing and reading complex analytical queries (window functions, CTEs, aggregations) to interrogate data and verify correctness Hands-on experience with Databricks — running … queries, navigating Unity Catalog, reading Spark job outputs and understanding what they mean for data quality Working knowledge of PySpark or Spark SQL — enough to read pipeline code, understand transformations, and trace where data issues originate Understanding of Lakehouse/medallion architecture (bronze-silver ...

support technical decision-making * Troubleshoot complex system and data issues What you'll bring: * Deep experience building production grade data systems * Strong SQL and hands-on experience with modern data tooling and cloud platforms * Ability to work beyond data into backend engineering (APIs, services, system design) * Experience with … someone who combines deep data expertise with broader engineering capability, get in touch with Justin Donaldson for a confidential chat. Skills: dbt spark sql snowflake apache airflow cloud JBLE1 ...

with modern cloud data platforms (e.g. Databricks or Snowflake), ideally within AWS environments Advanced programming and data engineering skills in Python, PySpark and SQL, with a solid understanding of ETL pipelines … data transformation Proven experience in data modelling (end-to-end), including performance optimisation using tools such as DBT and distributed processing frameworks like SparkSQL Experience with CI/CD, version control (e.g. Git, Jenkins) and pipeline orchestration tools such as Airflow A collaborative mindset with excellent interpersonal skills ...

analytics; exposure to predictive modelling and machine learning is a plus. Proficient in SQL and Python, with bonus points for PySpark, SparkSQL, and Git. Skilled in data visualisation with tools such as Tableau or Power BI. Confident writing efficient code and troubleshooting sophisticated queries. Clear and adaptable communicator ...

with how AWS services power scalable platforms. Python is second nature to you , whether for building pipelines, automating processes, or writing tests. Strong SQL skills with an interest in (or experience with) PostgreSQL/PostGIS for spatial data. Experience structuring data through Bronze, Silver, and Gold layers using … Python and Spark SQL. You should know, and have experience with, every keyword in the second half of this sentence: Lakeflow Spark Declarative Pipelines, ETL, Delta, Unity Catalog, Data Quality, Medallion Architecture, Change Data Capture, SCD Types, Python, Git, and GitHub. Preferred Skills Curious about geospatial ...