1 to 25 of 43 Spark SQL Jobs in the UK

considering: We are looking for an experienced AWS Databricks Engineer with strong hands-on expertise in Databricks, AWS cloud services, PySpark, Spark SQL, Delta Lake, Python, SQL, and data engineering. The candidate will be responsible for designing, developing, optimizing, and supporting scalable data pipelines … support code reviews, design reviews, and engineering best practices. Your Skills: Strong hands-on experience with Databricks on AWS. Strong experience with Apache Spark/PySpark. Excellent programming skills in Python. Strong SQL skills including complex queries, joins, CTEs, window functions, and query optimization. Experience with ...

more of: GCP, Azure, AWS. An intermediate or better level coding in one or more mainstream coding languages (i.e., Python, SQL, Java, Scala, and R), and critically review the code of other data engineers. Develop code for a range of data products including data matching, rule development, scans … date with these. Typical Data Engineering Experience required (5+ yrs): Knowledge and experience of Azure/AWS Cloud data solution provision. Proficient in SQL . Ability to develop and deliver complex visualisation, reporting and dashboard solutions using tools like Power BI . Enterprise-scale experience with ETL tools ...

more of: GCP, Azure, AWS. An intermediate or better level coding in one or more mainstream coding languages (i.e., Python, SQL, Java, Scala, and R), and critically review the code of other data engineers. Develop code for a range of data products including data matching, rule development, scans … date with these. Typical Data Engineering Experience required (5+ yrs): Knowledge and experience of Azure/AWS Cloud data solution provision. Proficient in SQL . Ability to develop and deliver complex visualisation, reporting and dashboard solutions using tools like Power BI . Enterprise-scale experience with ETL tools ...

more of: GCP, Azure, AWS. An intermediate or better level coding in one or more mainstream coding languages (i.e., Python, SQL, Java, Scala, and R), and critically review the code of other data engineers. Develop code for a range of data products including data matching, rule development, scans … date with these. Typical Data Engineering Experience required (5+ yrs): Knowledge and experience of Azure/AWS Cloud data solution provision. Proficient in SQL . Ability to develop and deliver complex visualisation, reporting and dashboard solutions using tools like Power BI . Enterprise-scale experience with ETL tools ...

more of: GCP, Azure, AWS. An intermediate or better level coding in one or more mainstream coding languages (i.e., Python, SQL, Java, Scala, and R), and critically review the code of other data engineers. Develop code for a range of data products including data matching, rule development, scans … date with these. Typical Data Engineering Experience required (5+ yrs): Knowledge and experience of Azure/AWS Cloud data solution provision. Proficient in SQL . Ability to develop and deliver complex visualisation, reporting and dashboard solutions using tools like Power BI . Enterprise-scale experience with ETL tools ...

more of: GCP, Azure, AWS. An intermediate or better level coding in one or more mainstream coding languages (i.e., Python, SQL, Java, Scala, and R), and critically review the code of other data engineers. Develop code for a range of data products including data matching, rule development, scans … date with these. Typical Data Engineering Experience required (5+ yrs): Knowledge and experience of Azure/AWS Cloud data solution provision. Proficient in SQL . Ability to develop and deliver complex visualisation, reporting and dashboard solutions using tools like Power BI . Enterprise-scale experience with ETL tools ...

more of: GCP, Azure, AWS. An intermediate or better level coding in one or more mainstream coding languages (i.e., Python, SQL, Java, Scala, and R), and critically review the code of other data engineers. Develop code for a range of data products including data matching, rule development, scans … date with these. Typical Data Engineering Experience required (5+ yrs): Knowledge and experience of Azure/AWS Cloud data solution provision. Proficient in SQL . Ability to develop and deliver complex visualisation, reporting and dashboard solutions using tools like Power BI . Enterprise-scale experience with ETL tools ...

more of: GCP, Azure, AWS. An intermediate or better level coding in one or more mainstream coding languages (i.e., Python, SQL, Java, Scala, and R), and critically review the code of other data engineers. Develop code for a range of data products including data matching, rule development, scans … date with these. Typical Data Engineering Experience required (5+ yrs): Knowledge and experience of Azure/AWS Cloud data solution provision. Proficient in SQL . Ability to develop and deliver complex visualisation, reporting and dashboard solutions using tools like Power BI . Enterprise-scale experience with ETL tools ...

more of: GCP, Azure, AWS. An intermediate or better level coding in one or more mainstream coding languages (i.e., Python, SQL, Java, Scala, and R), and critically review the code of other data engineers. Develop code for a range of data products including data matching, rule development, scans … date with these. Typical Data Engineering Experience required (5+ yrs): Knowledge and experience of Azure/AWS Cloud data solution provision. Proficient in SQL . Ability to develop and deliver complex visualisation, reporting and dashboard solutions using tools like Power BI . Enterprise-scale experience with ETL tools ...

growing, forward-thinking data teamWhat You'll Do: Build and maintain ETL/ELT pipelines and data models in Fabric (Data Factory, Notebooks, Spark) Write high-performance Spark SQL, T-SQL, Python/PySpark Manage ingestion, transformation, and loading from multiple sources … development tools like GitHub Copilot, ChatGPT, and Fabric CopilotEssential Experience: Microsoft Fabric is essential Azure Data ecosystem Lakehouse architectures & Data Factory Python, PySpark, Spark SQL Proven hands-on delivery in this stackIf you're an experienced Data Engineer with s... ...

forward-thinking data team What You'll Do: Build and maintain ETL/ELT pipelines and data models in Fabric (Data Factory, Notebooks, Spark) Write high-performance Spark SQL, T-SQL, Python/PySpark Manage ingestion, transformation, and loading from multiple sources … tools like GitHub Copilot, ChatGPT, and Fabric Copilot Essential Experience: Microsoft Fabric is essential Azure Data ecosystem Lakehouse architectures & Data Factory Python, PySpark, Spark SQL Proven hands-on delivery in this stack If you're an experienced Data Engineer with strong Microsoft Fabric and Azure experience ...

data warehousing, OLTP systems, data integration, and the software development lifecycle (SDLC). Hands-on experience with big data frameworks such as Spark, Hadoop, or Hive, including derivatives like PySpark (preferred), Spark Scala, or Spark SQL, along with familiarity with development-accelerating … libraries and frameworks. Experience using data modelling tools such as ERwin, ER/Studio, PowerDesigner, or similar. Experience with major database platforms, including SQL Server, Oracle, Azure Data Lake, Hadoop, Azure Synapse (SQL Data Warehouse), Snowflake, Redshift, and others. Strong experience with orchestration tools such ...

cloud-based data pipelines for ingesting, transforming, and storing large volumes of data. Implement scalable, fault-tolerant systems using modern frameworks such as Spark, Temporal, Airflow o r Dagster . Develop APIs and services that expose reliable data access, transformation, and orchestration capabilities. Contribute to geospatial data processing … applications. Desired Qualifications: 3-5years of experience in data engineering, ideally working with large-scale or geospatial systems. Proficiency in Python and SQL (Spark SQL preferred); experience with distributed data processing frameworks such as Spark, Flink, or Beam. Familiarity with orchestration ...

cloud-based data pipelines for ingesting, transforming, and storing large volumes of data. Implement scalable, fault-tolerant systems using modern frameworks such as Spark, Temporal, Airflow o r Dagster . Develop APIs and services that expose reliable data access, transformation, and orchestration capabilities. Contribute to geospatial data processing … applications. Desired Qualifications: 3-5years of experience in data engineering, ideally working with large-scale or geospatial systems. Proficiency in Python and SQL (Spark SQL preferred); experience with distributed data processing frameworks such as Spark, Flink, or Beam. Familiarity with orchestration ...

cloud-based data pipelines for ingesting, transforming, and storing large volumes of data. Implement scalable, fault-tolerant systems using modern frameworks such as Spark, Temporal, Airflow o r Dagster . Develop APIs and services that expose reliable data access, transformation, and orchestration capabilities. Contribute to geospatial data processing … applications. Desired Qualifications: 3-5years of experience in data engineering, ideally working with large-scale or geospatial systems. Proficiency in Python and SQL (Spark SQL preferred); experience with distributed data processing frameworks such as Spark, Flink, or Beam. Familiarity with orchestration ...

cloud-based data pipelines for ingesting, transforming, and storing large volumes of data. Implement scalable, fault-tolerant systems using modern frameworks such as Spark, Temporal, Airflow o r Dagster . Develop APIs and services that expose reliable data access, transformation, and orchestration capabilities. Contribute to geospatial data processing … applications. Desired Qualifications: 3-5years of experience in data engineering, ideally working with large-scale or geospatial systems. Proficiency in Python and SQL (Spark SQL preferred); experience with distributed data processing frameworks such as Spark, Flink, or Beam. Familiarity with orchestration ...

cloud-based data pipelines for ingesting, transforming, and storing large volumes of data. Implement scalable, fault-tolerant systems using modern frameworks such as Spark, Temporal, Airflow o r Dagster . Develop APIs and services that expose reliable data access, transformation, and orchestration capabilities. Contribute to geospatial data processing … applications. Desired Qualifications: 3-5years of experience in data engineering, ideally working with large-scale or geospatial systems. Proficiency in Python and SQL (Spark SQL preferred); experience with distributed data processing frameworks such as Spark, Flink, or Beam. Familiarity with orchestration ...

cloud-based data pipelines for ingesting, transforming, and storing large volumes of data. Implement scalable, fault-tolerant systems using modern frameworks such as Spark, Temporal, Airflow o r Dagster . Develop APIs and services that expose reliable data access, transformation, and orchestration capabilities. Contribute to geospatial data processing … applications. Desired Qualifications: 3-5years of experience in data engineering, ideally working with large-scale or geospatial systems. Proficiency in Python and SQL (Spark SQL preferred); experience with distributed data processing frameworks such as Spark, Flink, or Beam. Familiarity with orchestration ...

cloud-based data pipelines for ingesting, transforming, and storing large volumes of data. Implement scalable, fault-tolerant systems using modern frameworks such as Spark, Temporal, Airflow o r Dagster . Develop APIs and services that expose reliable data access, transformation, and orchestration capabilities. Contribute to geospatial data processing … applications. Desired Qualifications: 3-5years of experience in data engineering, ideally working with large-scale or geospatial systems. Proficiency in Python and SQL (Spark SQL preferred); experience with distributed data processing frameworks such as Spark, Flink, or Beam. Familiarity with orchestration ...

modernisation of existing data estates. 🔧 Key Responsibilities Design and build scalable data pipelines using Databricks Develop and optimise ETL/ELT workflows using Spark (PySpark/SQL) Support migration and modernisation of legacy data platforms into Databricks Improve performance, reliability and cost efficiency across data workloads … best practices across delivery ✅ What We’re Looking For Strong hands-on experience with Databricks in a Data Engineering capacity Solid experience with Spark (PySpark/Spark SQL) Proven track record building and optimising data pipelines (ETL/ELT) Experience working within cloud environments ...

modelling, dimensional modelling, canonical data models, and enterprise data models. Strong experience in designing ETL/ELT data pipelines. Good understanding of PySpark, Spark SQL, Python, and SQL. Strong knowledge of data warehouse, data lake, data mart, and semantic layer architecture. Experience in defining data quality ...

high-stakes and the upside of getting it right is visible at company level. You'll design and deploy solutions in Python, SQL and Spark SQL, working across Azure Synapse, Snowflake, Databricks and Azure ML. The environment is modern and evolving. You'll help … afterthought Strong candidates will come from a background in data science or applied statistics, with hands-on ML experience and the Python/SQL depth to match. Experience with Azure tooling is useful; the ability to think clearly about business problems and turn them into scalable analytical solutions ...

Engineer within Azure cloud environments. Proven hands-on expertise with Azure Databricks in enterprise-scale production environments. Strong programming skills using: Python/SQL/PySpark/Spark SQL. Experience designing and building scalable data pipelines and data processing frameworks. Strong understanding of modern Lakehouse architecture … understand data requirements and translate them into scalable engineering solutions. Design, develop and maintain high-performance data pipelines using Azure Databricks, PySpark, Spark SQL and Python. Contribute to the design and implementation of Lakehouse architecture using Delta Lake and Databricks best practices. Develop and manage data ...

designing and building production-grade ETL pipelines, driving DevOps practices across data systems and contributing to high-availability architectures using tools like Databricks, Spark and Airflow- all within a modern AWS ecosystem. Responsibilities Architect and build scalable, secure data pipelines using AWS, Databricks and PySpark. Design and implement … . Advanced PySpark and Databricks experience (Delta Lake, Unity Catalog, Databricks Jobs etc). Proficient in SQL (T-SQL/SparkSQL) and Python for data transformation and scripting. Hands-on experience with workflow orchestration tools such as Airflow. Strong version control and DevOps exposure (Git, GitHub ...

designing and building production-grade ETL pipelines, driving DevOps practices across data systems and contributing to high-availability architectures using tools like Databricks, Spark and Airflow- all within a modern AWS ecosystem. Responsibilities Architect and build scalable, secure data pipelines using AWS, Databricks and PySpark. Design and implement … . Advanced PySpark and Databricks experience (Delta Lake, Unity Catalog, Databricks Jobs etc). Proficient in SQL (T-SQL/SparkSQL) and Python for data transformation and scripting. Hands-on experience with workflow orchestration tools such as Airflow. Strong version control and DevOps exposure (Git, GitHub ...