1 to 25 of 62 PySpark Jobs in the UK excluding London

looking for an experienced Data Engineer with proven expertise in building pipelines in Databricks and a strong grasp of Apache Spark, whether in PySpark or Scala, including Structured Streaming. You should have experience with Kafka (MSK) and real-time data ingestion, as well as a deep understanding of Delta ...

would also be highly advantageous, particularly: Azure Data Factory Azure SQL Azure Data Lake/ADLS Azure DevOps Microsoft Fabric Azure Databricks Experience with PySpark, Airflow, Dagster, Prefect or other orchestration technologies would also be useful. We're particularly interested in people who have worked with modern data platforms ...

/AI services). Desirable Experience Big Data and Cloud Processing: Hands-on experience with AWS (EMR, S3, Lambda, Redshift, Athena, CloudWatch) and PySpark/Python for large-scale batch processing. DevOps and CI/CD: Familiarity with automated testing pipelines, Infrastructure as Code (Terraform), and Git/GitLab ...

data quality best practice.Strong problem-solving skills and attention to detail.The ability to communicate effectively with both technical and business stakeholders.Desirable ExperiencePython or PySpark development within Microsoft Fabric.CI/CD practices for data engineering and data product delivery.Experience within legal services or another professional services environment.Exposure to systems such ...

Python/Data Engineering professionals with a solid Azure foundation and the ability to learn and work across the wider platform. Desirable Databricks/PySpark NoSQL/Cosmos DB Infrastructure as Code/DevOps automation AI/ML data integration Automotive or insurance industry experience If this sounds ...

have To be considered for this role, we would love you to have: Extensive experience in programming/query language skills including Python/PySpark and SQL. Data engineering experience - via data preparation, transformation & conversion to allow for rapid processing of system data across a number of different formats ...

will bring strong expertise in Power BI, Microsoft Fabric and associated languages such as DAX, SQL and Power Query. Familiarity with Azure DevOps and PySpark/Python would be beneficial as would experience of working with cloud-based data solutions. Principal Responsibilities Report Development: Develop and deploy Power ...

contribute to long-term platform innovation Required Experience Strong experience designing enterprise data platforms within Microsoft Azure environments Deep expertise with Azure Databricks, Spark (PySpark/Scala), Delta Lake, and Lakehouse architecture Strong hands-on experience with Snowflake architecture and optimisation Experience with Azure Data Factory, ADLS Gen2, Synapse ...

deliver practical, value-driven data solutions. Drive continuous improvement across data engineering, reporting, and analytics capabilities. Essential Requirements Strong expertise in SQL and Python (PySpark desirable). Proven experience designing and building cloud-based data platforms from inception. Hands-on experience with Databricks, Spark, modern data lake/lakehouse ...

Knowledge of data warehousing and modern data architectures Strong problem-solving and stakeholder communication skills Desirable Skills Experience with Azure Data Factory Python or PySpark development experience Experience working within cloud data transformation programmes Understanding of data governance and security best practices Contract Details 6-Month Contract ...

pipelines. Experience working with cloud data platforms (AWS preferred). Exposure to data warehousing, data lakes and modern data architecture. Python experience (including PySpark) is beneficial but not essential. Exposure to BI platforms such as Power BI, Tableau or QuickSight, with an understanding of how backend data models support ...

hands-on experience with modern cloud data platforms (e.g. Databricks or Snowflake), ideally within AWS environments. Advanced programming and data engineering skills in Python, PySpark and SQL, with a solid understanding of ETL pipelines and data transformation. Proven experience in data modelling (end-to-end), including performance optimisation using ...

Data Platforms: Hands-on experience with Databricks and Azure Data Lake . ETL: Proven ability to build efficient pipelines (ADF, SSIS, APIs). Desirable: PySpark, Dimensional Modelling, DAX/SSAS, or an interest in AI/ML applications. Interested in learning more? Send me a DM or comment below ...

Databricks Azure Data Lake/Lakehouse Good understanding of data architecture and data flows . Practical SQL experience. Experience with Python and/or PySpark . Experience with CI/CD and DevOps . Experience with Power BI and/or Power Apps . Strong understanding of data ingestion ...

Microsoft Fabric, Azure Data Platform technologies, and modern cloud data architectures Hands-on experience with Databricks, Unity Catalog, and Delta Lake Advanced SQL and PySpark development skills Experience implementing Medallion Architecture principles Strong data modelling expertise, including Star Schema and Data Vault methodologies Solid understanding of data governance, security ...

data engineering/integration/migration experience Hands-on Azure Data Factory, Fabric Data Factory, or Synapse Pipelines Strong SQL - profiling, transformation, validation, reconciliation PySpark/Spark SQL notebook experience (Databricks or Fabric) Proven experience on a data migration project end-to-end (extraction through reconciliation) Microsoft Fabric experience ...

about its values Values | PA ConsultingWe use a range of different tools and technologies depending on the specific task, these include: Data Bricks, Python, PySpark, R, Excel, Palantir Foundry and PowerBI.QualificationsEssential requirements Even if you don’t meet every requirement below, feel free to still apply ...

present insights clearly to both technical and non-technical audiences.Some other highly valued skills include:Strong SQL and database experience.Knowledge of Python/PySpark and data analytics.Experience working in a Call Centre or Contact Centre environment.Strong communication and stakeholder management skills.Ability to lead technical teams and mentor junior developers.You ...

with Microsoft Fabric, Azure Data Platform technologies and modern Lakehouse architecture Hands-on experience with Databricks, Unity Catalog and Delta Lake Strong SQL and PySpark skills Experience implementing Medallion Architecture principles Knowledge of data modelling methodologies including Star Schema and Data Vault Strong understanding of data governance, security ...

solutions Requirements: Enterprise data warehouse architecture and design Microsoft Fabric, Azure Data Platform technologies and modern Lakehouse architectures Databricks, Unity Catalog and Delta Lake PySpark, Databricks, SQL Medallion Architecture implementation Data modelling approaches including Star Schema and Data Vault Data governance, security and access management Why This Opportunity Greenfield ...

solutions Requirements: Enterprise data warehouse architecture and design Microsoft Fabric, Azure Data Platform technologies and modern Lakehouse architectures Databricks, Unity Catalog and Delta Lake PySpark, Databricks, SQL Medallion Architecture implementation Data modelling approaches including Star Schema and Data Vault Data governance, security and access management Why This Opportunity Greenfield ...

that support operational, financial and client reporting. Key Responsibilities Design, develop and maintain Azure Data Factory pipelines Build and optimise Azure Databricks solutions using PySpark and SQL Support a large-scale enterprise data migration programme Develop scalable ETL/ELT processes and data ingestion frameworks Design data models using … working closely with both technical and non-technical stakeholders to deliver practical business solutions. Essential Skills & Experience Strong commercial experience with Azure Databricks and PySpark Azure Data Factory Advanced SQL, including query optimisation and complex transformations ETL/ELT pipeline development Dimensional modelling (Kimball, Star Schemas) Experience building modern ...

Logical, Physical and Dimensional) Extensive Azure Databricks experience within large-scale enterprise environments Databricks Lakehouse Architecture and Medallion Design Patterns Databricks Data Engineering using PySpark and Spark SQL Delta Live Tables (DLT) Unity Catalog implementation and governance Databricks Workflows Databricks Asset Bundles (DABs) Databricks Solution Accelerators Metadata-Driven Ingestion … Power BI semantic models and enterprise reporting environments Technical Environment Azure Databricks Delta Live Tables (DLT) Unity Catalog Databricks Asset Bundles (DABs) Databricks Workflows PySpark Spark SQL MLflow Mosaic AI Azure Data Factory Azure Synapse Azure Data Lake Storage Dynamics 365 Finance & Operations (D365 F&O) Azure DevOps Power ...

methodology. Platforms & Tools Experience with Databricks (preferred) or similar platforms: Snowflake, BigQuery, Redshift, Synapse, or equivalent Strong proficiency with: SQL (advanced level) Spark/PySpark Data pipeline orchestration tools (e.g., Airflow, ADF, etc.) Data Modelling Extensive experience designing: Fact and dimension tables Star and snowflake schemas Aggregation layers ...

actively building, modernising and optimising data workloads within a large-scale, complex environment. What we’re looking for: Strong Spark/PySpark engineering Large-scale data pipeline development & modernisation Python and strong software engineering fundamentals Performance, scalability & workload optimisation Cloud data engineering – ideally AWS CI/CD and modern ...