16 of 16 PySpark Jobs in the South West

with Spark-Based Solutions: You possess experience with Spark-based platforms like Azure Synapse, Databricks, Microsoft Fabric, or even on-premise Spark clusters, using PySpark or Spark SQL to manage and process large datasets. Expertise in Building ETL and ELT Pipelines: You are skilled in building robust ...

with Spark-Based Solutions: You possess experience with Spark-based platforms like Azure Synapse, Databricks, Microsoft Fabric, or even on-premise Spark clusters, using PySpark or Spark SQL to manage and process large datasets. Expertise in Building ETL and ELT Pipelines: You are skilled in building robust ...

with Spark-Based Solutions: You possess experience with Spark-based platforms like Azure Synapse, Databricks, Microsoft Fabric, or even on-premise Spark clusters, using PySpark or Spark SQL to manage and process large datasets. Expertise in Building ETL and ELT Pipelines: You are skilled in building robust ...

e.g. Power BI) Deep understanding of data wareho using concepts, ETL/ELT pipelines and dimensional modelling Proficiency in advanced programming languages (Python/PySpark, SQL) Experience in data pipeline orchestration (e.g. Airflow, Data Factory) Familiarity with DevOps and CI/CD practices (Git, Azure DevOps etc) Ability ...

overview of many different sectors. What We're Looking For 10 + years, hands-on experience in AWS data engineering technologies, including Glue, PySpark, Athena, Iceberg, Databricks, Lake Formation, and other standard data engineering tools. Strong experience engineering in a front-office/capital markets environment. Previous experience ...

overview of many different sectors. What We're Looking For 10 + years, hands-on experience in AWS data engineering technologies, including Glue, PySpark, Athena, Iceberg, Databricks, Lake Formation, and other standard data engineering tools. Strong experience engineering in a front-office/capital markets environment. Previous experience ...

practices for data governance and security. Required Skills & Experience 5–10 years in data engineering roles, ideally within consultancy environments. Strong proficiency in: Databricks (PySpark) Azure Data Factory (ADF) SQL (complex queries and optimisation) Azure Data Services (Data Lake, Synapse, etc.) Excellent communication and stakeholder management skills. Nice ...

practices for data governance and security. Required Skills & Experience 5–10 years in data engineering roles, ideally within consultancy environments. Strong proficiency in: Databricks (PySpark) Azure Data Factory (ADF) SQL (complex queries and optimisation) Azure Data Services (Data Lake, Synapse, etc.) Excellent communication and stakeholder management skills. Nice ...

practices for data governance and security. Required Skills & Experience 5–10 years in data engineering roles, ideally within consultancy environments. Strong proficiency in: Databricks (PySpark) Azure Data Factory (ADF) SQL (complex queries and optimisation) Azure Data Services (Data Lake, Synapse, etc.) Excellent communication and stakeholder management skills. Nice ...

manipulate data and building additional process to support data processing activities Experience manipulating, processing and storing data using tools such as SQL, dbt, pyspark and SSIS Comfortable building out and administrating data models, ensuring that they are accessible and used appropriately, leveraging platforms such as Databricks and Power ...

quality processes are followed and continuously improved. About You •Strong working knowledge of data warehousing, ETL/ELT processes and programming languages (SQL, Python, PySpark). •Experience on Cloud based BI/MI methodology, such as use of Azure, DataBricks, Azure Data Factory and Fabric. •Proven experience of taking ...

manage Ontology Objects, Links, and Actions to support downstream applications and workflows. Pipeline Development & Automation: Build scalable, fault-tolerant ETL/ELT pipelines using PySpark, SQL, and Foundry's proprietary tools like Pipeline Builder and Code Workbook. Productization & Scaling: Create Prototypes and Convert them into production-grade solutions ...

Continuously improve data systems and maintain awareness of best practice and emerging approaches Essential Skills Experience with large-scale analytics engines (e.g. Spark/PySpark) and scripting languages such as Python or Scala. Hands-on use of cloud data stacks (e.g. SageMaker Notebooks, S3, Glue, Athena) and modern storage ...

build skills and knowledge relevant to the role. Essential Skills Experience of using a unified engine for large-scale data analytics (e.g. Spark/PySpark) Experience in writing, testing and implementing scripts (e.g. Python, Scala) Experience of cloud data stack use (e.g. SageMaker Notebooks, S3, Glue, Athena) Communicating technical ...

quality and structure of data Translate business needs into clear technical plans Experience required: Strong skills in data modelling SQL with either Python or PySpark Databricks or similar cloud data platform Experience communicating with both technical and non-technical users Leadership or mentoring experience Insurance or regulated sector experience ...

that form the backbone of our Lakehouse platform, enabling accurate analytics, reporting, and regulatory compliance. You'll work with cutting-edge technologies including Databricks , PySpark , and Azure Data Factory , applying best practices in data engineering and governance to support this critical programme. Lead Databricks Engineer: Key Responsibilities Build … maintain Databricks pipelines (batch and incremental) using PySpark and SQL. Orchestrate end-to-end workflows with Azure Data Factory . Develop and optimise Delta Lake tables (partitioning, schema evolution, vacuuming). Implement Medallion Architecture (Bronze, Silver, Gold) for transforming raw data into business-ready datasets. Apply robust monitoring, logging ...