51 to 75 of 132 PySpark Jobs in England

senior or consultant level. Strong hands‐on knowledge of Microsoft Fabric components: Notebooks, Delta tables, Fabric SQL Lakehouse/OneLake concepts Deep experience writing PySpark in Fabric notebooks, including: Reading/writing Delta, schema evolution, partitioning, optimisation Solid understanding and implementation experience of SCD2 and other slowly changing dimension ...

senior or consultant level. Strong hands‐on knowledge of Microsoft Fabric components: Notebooks, Delta tables, Fabric SQL Lakehouse/OneLake concepts Deep experience writing PySpark in Fabric notebooks, including: Reading/writing Delta, schema evolution, partitioning, optimisation Solid understanding and implementation experience of SCD2 and other slowly changing dimension ...

senior or consultant level. Strong hands‐on knowledge of Microsoft Fabric components: Notebooks, Delta tables, Fabric SQL Lakehouse/OneLake concepts Deep experience writing PySpark in Fabric notebooks, including: Reading/writing Delta, schema evolution, partitioning, optimisation Solid understanding and implementation experience of SCD2 and other slowly changing dimension ...

senior or consultant level. Strong hands‐on knowledge of Microsoft Fabric components: Notebooks, Delta tables, Fabric SQL Lakehouse/OneLake concepts Deep experience writing PySpark in Fabric notebooks, including: Reading/writing Delta, schema evolution, partitioning, optimisation Solid understanding and implementation experience of SCD2 and other slowly changing dimension ...

efficient deployments and robust system performance. Working within Azure, you'll integrate advanced analytics platforms like Databricks and build data pipelines using Python and PySpark to support large-scale processing. The position also involves optimising data models, delivering insights through Power BI, and managing relational data with SQL. Collaboration ...

office each week. The Role Key responsibilities: Design, build and maintain data pipelines using Azure Data Factory and Azure Databricks Develop and optimise PySpark and SQL transformations Model data using dimensional modelling principles Support ingestion of data from various data sources and formats Collaborate with BI developers, analysts ...

enquire about other roles, please contact Nick Mandella at Harnham. KEYWORDS Python, SQL, AWS, GCP, Azure, Cloud, Databricks, Docker, Kubernetes, CI/CD, Terraform, Pyspark, Spark, Kafka, machine learning, statistics, Data Science, Data Scientist, Big Data, Artificial Intelligence, private equity, finance. ...

organisation ahead of the curve. Skills and Experience Proven experience as a Senior or Lead Data Engineer. Hands-on experience with Databricks and PySpark/Spark SQL. Strong expertise with the Azure data platform (Data Factory, Synapse, etc.). Solid understanding of data architecture and performance optimisation. What ...

5+ years of data engineering experience Strong Python and SQL skills —essential for pipeline development and data transformation. Experience with Databricks/Snowflake, and PySpark ideally A track record of taking ownership —whether it’s revamping a data warehouse, improving a data lake, or delivering visible business impact. ...

Strong knowledge of data extraction, transformation, and loading (ETL) processes, leveraging tools such as Talend, Informatica, Matillion, Pentaho, MuleSoft, Boomi, or scripting languages (Python, PySpark, SQL). Understanding of data warehousing and data modelling techniques (Star Schema, Snowflake Schema). Familiarity with security frameworks (GDPR, HIPAA, ISO 27001, NIST ...

Salary range: £75,000-£85,000 + 10% bonus + benefits Purpose: Build and maintain large, scalable Data Lakes, processes and pipelines Tech: Python, PySpark, AWS, Iceberg/Kafka, Spark/Glue, CI/CD Industry: Financial services/securities trading Immersum continue to support a leading SaaS securities ...

Salary range: £75,000-£85,000 + 10% bonus + benefits Purpose: Build and maintain large, scalable Data Lakes, processes and pipelines Tech: Python, PySpark, AWS, Iceberg/Kafka, Spark/Glue, CI/CD Industry: Financial services/securities trading Immersum continue to support a leading SaaS securities ...

optimising large-scale data pipelines Practical expertise with Databricks and Azure services Confident communicator with strong problem-solving ability Core Technologies Databricks DBT Python PySpark SQL Azure If you are interested in this role then please apply via this platform or email me a copy of your most ...

data pipelines and ETL processes. Skills and Experience Proven experience as a Senior or Lead Data Engineer. Strong hands-on expertise with Databricks and PySpark/Spark SQL. Deep knowledge of the Azure data platform (Data Factory, Synapse, etc.). Solid understanding of data architecture principles and performance optimisation. ...

understanding of relational databases and data warehouse concepts (dimensions, facts, star schemas) and associated technologies (e.g., Databricks). Experience with Python (including Pandas/PySpark). Familiarity with software development and source control, in particular Git and CI/CD practices. Excellent communication and stakeholder management skills — able ...

understanding of relational databases and data warehouse concepts (dimensions, facts, star schemas) and associated technologies (e.g., Databricks). Experience with Python (including Pandas/PySpark). Familiarity with software development and source control, in particular Git and CI/CD practices. Excellent communication and stakeholder management skills — able ...

understanding of relational databases and data warehouse concepts (dimensions, facts, star schemas) and associated technologies (e.g., Databricks). Experience with Python (including Pandas/PySpark). Familiarity with software development and source control, in particular Git and CI/CD practices. Excellent communication and stakeholder management skills — able ...

enabling advanced analytics and data-driven decision making across the business. Key Responsibilities Design, develop, and maintain ETL/ELT pipelines using Azure Databricks, PySpark, and Delta Lake. Build and optimise data lakehouse architectures on Azure Data Lake Storage ( ADLS ) . Develop high-performance data solutions using Azure Synapse … Actions, or Azure DevOps Required Skills & Experience 3+ years' experience as a Data Engineer working in Azure environments. Strong hands-on experience with Databricks (PySpark, Delta Lake, cluster optimisation, job scheduling). Solid knowledge of Azure cloud services including: Azure Data Lake Storage Azure Data Factory Azure Synapse/ ...

optimize Lakehouse architectures using Delta Lake principles Ingest, transform, and curate data from multiple sources (APIs, databases, files, streaming) Develop scalable data transformations using PySpark and Spark SQL Implement data models optimized for analytics and reporting (e.g. star schemas) Monitor, troubleshoot, and optimize performance and cost of data workloads … practices Required Skills & Experience Strong experience with Microsoft Fabric (Lakehouse, Pipelines, Notebooks, Dataflows, OneLake) Hands-on experience with Databricks in production environments Proficiency in PySpark and SQL Solid understanding of data engineering concepts (ETL/ELT, orchestration, partitioning) Experience working with Delta Lake Familiarity with cloud platforms (Azure preferred ...

across multiple time zones, including offshore development teams Skills required: Proven track record building scalable data pipelines (batch and streaming) in production Expert Python, PySpark, Celery and RabbitMQ skills; deep experience with AWS data stack (Glue, OpenSearch, RDS) Expert skills within SQL with experience in both transactional RDBMS systems ...

Required Medical industry experience (essential) Strong knowledge of Azure Databricks, Palantir Foundry, and Azure DevOps Proficiency in Terraform, Git, and automation tools Experience with PySpark, Python, and SQL Familiarity with PowerBI, Azure Functions, and data streaming Understanding of cloud infrastructure and public cloud operations Excellent problem-solving and analytical ...

Required Medical industry experience (essential) Strong knowledge of Azure Databricks, Palantir Foundry, and Azure DevOps Proficiency in Terraform, Git, and automation tools Experience with PySpark, Python, and SQL Familiarity with PowerBI, Azure Functions, and data streaming Understanding of cloud infrastructure and public cloud operations Excellent problem-solving and analytical ...

. Maintain/implement data warehousing solutions and manage large-scale data storage systems (e.g. Microsoft Fabric) Build and optimise SQL queries, stored procedures, PySpark notebooks and database objects to ensure data performance and reliability. Migrate and modernise legacy databases to cloud-based architectures. Database Administration Administer, monitor … normalisation, indexing, query optimisation). Strong experience with ETL/ELT tools. E.g. Azure Data Factory, Databricks, Synapse Pipelines, SSIS, etc. Experience with Python, PySpark, or Scala for data processing. Familiarity with CI/CD practices. Experience with Data lake, Data warehouse and Medallion architectures. Understanding of API integrations ...

skills/experience you'll already have? 3+ years commercial experience in Data Engineering SQL Server Spark SQL Strong Python coding skills PySpark Azure experience including Azure SQL, Data Factory and Functions Experience collaborating with Software and Test Engineers Creating, documenting and maintaining database schemas To find out more ...

role focuses on designing, building and maintaining scalable data pipelines and products on a modern Databricks Lakehouse environment, using Delta Lake, Spark, Python (PySpark) and SQL to deliver high-quality, well-governed data across the business. You'll manage and optimise the cloud-based data lakehouse, leverage features like ...