21 of 21 Permanent Spark SQL Jobs

one. What You'll Be Doing Lakehouse Engineering (Azure + Databricks) Engineer scalable ELT pipelines using Lakeflow Declarative Pipelines , PySpark , and Spark SQL across a full Medallion Architecture (Bronze Silver Gold) . Implement ingestion patterns for files, APIs, SaaS platforms (e.g. subscription billing), SQL … SLIs, runbooks and cost-optimisation across the platform. DevOps & Platform Engineering Build CI/CD pipelines in Azure DevOps for notebooks, Lakeflow pipelines, SQL models and ADF artefacts. Ensure secure, enterprise-grade platform operation across Dev Prod , using private endpoints, managed identities and Key Vault. Contribute to platform ...

firefighting? You will design, build and maintain ETL and ELT pipelines , lakehouse structures and semantic models using Microsoft Fabric’s Dataflows Gen2, Notebooks, Spark SQL and Python. Your work will ensure data is accessible, trusted and well governed across hybrid on‑prem and cloud environments. … Essential Strong experience designing and operating scalable ETL/ELT pipelines. Hands‑on Microsoft Fabric experience (Dataflows Gen2, Notebooks, semantic models). SQL and Python proficiency, with Spark/Spark SQL exposure. Practical understanding of data quality, observability and troubleshooting. Ability ...

combat travel-based issues and advice on expected interview topics/questions. Main Duties and Responsibilities Write clean and testable code using SQL and Python scripting languages, to enable our customer data products and business applications Build and manage data pipelines and notebooks, deploying code in a structured … Promote an innovative thinking process and encourage it in others. Working within the agile framework at Chambers Skills and Experience Strong proficiency in SQL , including Spark SQL and MS SQL Server , for querying, data manipulation, and performance optimization. Hands-on experience with ...

platform operations. * Architect and design end-to-end data platform solutions focused on scalability, security and reliability. * Lead delivery using Databricks, PySpark, Spark SQL and Azure Data Factory. * Drive DevOps best practices including CI/CD, automated testing and infrastructure as code. * Collaborate closely with data … Skills and Experience You will bring: * Strong commercial experience in Azure and Databricks, with deep knowledge of Lakehouse architecture. * Proficiency in PySpark, Python, SQL and Spark SQL. * Strong DevOps engineering skills including CI/CD, automated testing and deployment pipelines. * Experience with monitoring, logging and alerting ...

platform operations. * Architect and design end-to-end data platform solutions focused on scalability, security and reliability. * Lead delivery using Databricks, PySpark, Spark SQL and Azure Data Factory. * Drive DevOps best practices including CI/CD, automated testing and infrastructure as code. * Collaborate closely with data … Skills and Experience You will bring: * Strong commercial experience in Azure and Databricks, with deep knowledge of Lakehouse architecture. * Proficiency in PySpark, Python, SQL and Spark SQL. * Strong DevOps engineering skills including CI/CD, automated testing and deployment pipelines. * Experience with monitoring, logging and alerting ...

Microsoft Fabric Knowledge: A working understanding of Lakehouse concepts (Bronze/Silver/Gold), data pipelines, and OneLake storage. Querying & Modelling: Exposure to SQL or Spark SQL for querying data models, alongside a familiarity with Star Schema principles. Platform Concepts: An understanding ...

Experience 5+ years' experience working as a Data Engineer or in a similar role Proven experience with cloud-based data platforms (Azure, AWS, SQL, Snowflake, Springserv); Microsoft Fabric experience is a strong plus Strong proficiency in Spark SQL and PySpark , including complex transformations Experience ...

Analytics Team with an exciting plan and long-term vision. We are seeking an experienced Data Engineer, with skills in Microsoft Azure and SQL, to take the contribute to the design, build and deployment of our Azure data platform in line with our enterprise architecture. Key Outputs Transform …/or Databricks. Strong foundation in data modelling and data warehousing concepts (star/snowflake schemas, SCD2, Dimensions/Facts) Proficient in SQL and Python, with hands-on experience using PySpark, Spark SQL, and/or Pandas. Experience in technical documentation of migration processes ...

formal training, in-house development sessions, mentoring, and opportunities to attend external gatherings such as User Groups, and of course FabCon and SQL Bits. We cover the cost of study for Microsoft certification exams to make sure you reach your full potential, and you will have personal development … based Production Data & Analytics ETL/integration solutions (Fabric, Databricks, Synapse, Data Factory, Function Apps, Power BI etc) against SLAs Python, PySpark, Scala, Spark SQL Ability to identify and investigate complex problems and implement resolutions Direct contact with customers/end-users on the phone ...

platform. You'll play a pivotal role in designing, building, and optimising their Azure-based Data Lakehouse, with a focus on Databricks, PySpark, Spark SQL, and Azure Data Factory. This isn't just about coding - you'll also provide architectural guidance, mentor engineers, and ensure solutions … management, integration runtimes, and networking Lead architectural design and ensure platform scalability, reliability, and governance Write efficient, maintainable code in PySpark, Python, and SQL Implement CI/CD pipelines and cloud infrastructure via Terraform/ARM Collaborate with data engineers, architects, and business stakeholders Mentor and coach engineers ...

Databricks** You'll play a pivotal role in designing, building, and optimising their Azure-based Data Lakehouse, with a focus on Databricks, PySpark, Spark SQL, and Azure Data Factory. This isn't just about coding - you'll also provide architectural guidance, mentor engineers, and ensure solutions … management, integration runtimes, and networking Lead architectural design and ensure platform scalability, reliability, and governance Write efficient, maintainable code in PySpark, Python, and SQL Implement CI/CD pipelines and cloud infrastructure via Terraform/ARM Collaborate with data engineers, architects, and business stakeholders Mentor and coach engineers ...

role, focused on building, optimising, and operating Databricks-based Lakehouse solutions across cloud environments. Responsibilities: Design, build, and maintain data pipelines using Databricks (Spark, Delta Lake). Develop batch and streaming workloads using PySpark/Spark SQL. Implement data ingestion, transformation, and orchestration pipelines. Optimise Databricks … Databricks with cloud-native services across Azure, AWS, or GCP. Skills: Strong hands-on expertise with Databricks in production environments. Advanced PySpark and Spark SQL development experience. Experience building lakehouse architectures using Delta Lake. Strong cloud experience in Azure (preferred), AWS, or GCP. Experience with orchestration ...

curve. Skills and Experience Proven experience as a Senior or Lead Data Engineer. Hands-on experience with Databricks and PySpark/Spark SQL. Strong expertise with the Azure data platform (Data Factory, Synapse, etc.). Solid understanding of data architecture and performance optimisation. What is on offer Salary ...

processes. Skills and Experience Proven experience as a Senior or Lead Data Engineer. Strong hands-on expertise with Databricks and PySpark/Spark SQL. Deep knowledge of the Azure data platform (Data Factory, Synapse, etc.). Solid understanding of data architecture principles and performance optimisation. Experience managing ...

workflows. From a technical standpoint you will ideally possess/or aspire to learn: Knowledge or experience of Databricks including Unity Catolog and Spark SQL. Programming skills, preferably in Python and SQL. Knowledge and experience in Azure, including working with Azure Data Factory and Azure Storage Accounts. Knowledge ...

Lakehouse platform in Azure. Owning Databricks engineering, administration and platform optimisation. Designing end-to-end data platform solutions Leading implementation using Databricks, PySpark, Spark SQL and Azure Data Factory. Creating and maintaining CI/CD pipelines and automated testing frameworks. Developing APIs to support integration Implementing ...

following skills and experience: Deep expertise with Azure Databricks, including development, deployment, and platform-level engineering. Strong software engineering capability across Pyspark, Spark SQL and Azure Data Factory. Solid understanding of DevOps practices including CI/CD and automated testing. Experience with monitoring, logging, and alerting ...

following skills and experience: Deep expertise with Azure Databricks, including development, deployment, and platform-level engineering. Strong software engineering capability across Pyspark, Spark SQL and Azure Data Factory. Solid understanding of DevOps practices including CI/CD and automated testing. Experience with monitoring, logging, and alerting ...

following skills and experience: Deep expertise with Azure Databricks, including development, deployment, and platform-level engineering. Strong software engineering capability across Pyspark, Spark SQL and Azure Data Factory. Solid understanding of DevOps practices including CI/CD and automated testing. Experience with monitoring, logging, and alerting ...

columnar databases. Experience in data integration with ETL techniques and frameworks Experience in Big Data querying tools, such as Hive, Impala and Spark SQL Work Schedule: MULTIPLE SHIFTS AVAILABLE Work Type: Full Time ...

Role You’ll design and deliver modern, cloud-based data solutions across large Insurance transformation programmes, working hands-on with: Azure Databricks & Spark Azure Data Factory, Synapse & Data Lake Large-scale ETL/ELT pipelines Enterprise data modelling & governance frameworks Technical leadership and delivery assurance What … experience (Policy, Claims, Regulatory Data) ✔ 8–10+ years in Data Engineering/Integration ✔ Deep expertise in Azure & Databricks ✔ Strong Spark (PySpark/SparkSQL) skills ✔ Experience designing scalable, high-performance data platforms ✔ Confident with SQL, CI/CD & DevOps practices Why Join? You’ll be part ...