1 to 25 of 441 PySpark Jobs in the UK

optimisation techniques for large‐scale data processing Strong proficiency in SQL and Python for handling complex data problems Hands‐on experience with Apache Spark (PySpark or Spark SQL) Experience with the Azure data stack Knowledge of workflow orchestration tools like Azure Data Factory or Apache Airflow Experience with containerisation ...

Azure Databricks Azure Function Apps& Logic Apps Azure Stream Analytics Azure Resource Manager skills (Terraform, Azure Portal, Az CLI and Az PowerShell) Strong PySpark, Delta Lake, Unity Catalog and Python skills. Includes ability to write unit and integration tests in Python with unittest, pytest, etc. Strong understanding of software ...

transformation and orchestration pipelines, integrating data from cloud, on‐premises and third‐party business systems. Strong hands‐on capability with SQL, Python, Spark and PySpark for data engineering, analytics and automation workloads. Experience with Azure Databricks and/or Microsoft Fabric engineering workloads, including notebooks, Delta Lake, data transformation ...

Databricks ecosystem. What you'll need: Strong experience building data pipelines using Azure Data Factory, Databricks and Azure Data Lake. Advanced SQL and Python (PySpark) skills for data transformation, automation and optimisation. Hands-on experience with Delta Lake, Spark, Databricks Workflows and Structured Streaming. Knowledge of data warehousing, lakehouse ...

Databricks ecosystem. What you’ll need: Strong experience building data pipelines using Azure Data Factory, Databricks and Azure Data Lake. Advanced SQL and Python (PySpark) skills for data transformation, automation and optimisation. Hands-on experience with Delta Lake, Spark, Databricks Workflows and Structured Streaming. Knowledge of data warehousing, lakehouse ...

models in data warehouses, lakehouse environments or analytics tools such as Power BI. Familiarity with modern data engineering frameworks and tools (e.g. Spark/PySpark, Pandas, orchestration tools). Experience with cloud data platforms on AWS or Azure (e.g. Databricks, Redshift, Microsoft Fabric). Understanding of software engineering fundamentals ...

Data Quality, Data Integration, Data Analytics, Data Visualisation, Data Science and Business Intelligence.Confident in delivering solutions using Databricks & developing and optimising data pipelines using PySpark within Databricks environments.Proven experience building, publishing, and supporting dashboards and reports using Power BIStrong experience and programming skills in languages such as Python ...

construction, programme controls, PMO, or consultancy environments. Experience with cloud-native architectures, DevOps, CI/CD pipelines, and DataOps practices. Knowledge of Python, Spark, PySpark, Databricks, or equivalent data engineering technologies. Information Management in a complex construction or infrastructure project/programme Experience developing enterprise data products and supporting ...

Good understanding of modern software platform architecture and the role of the data platform within the broader platform ecosystem Good Scala skills (or strong PySpark plus a genuine willingness to work primarily in Scala) Strong SQL skills for transformations, data modelling, and debugging data issues Experience building and supporting ...

Spark/Azure Databricks for distributed compute, memory optimization, and Delta Lake formats.Data Engineering Languages: Advanced proficiency in SQL (T-SQL, SparkSQL) and Python (PySpark).Security & Compliance: Direct experience implementing security controls for sensitive or identifiable data, utilizing Azure Key Vault, Azure Monitor, Log Analytics, and Microsoft Purview.Data Integration ...

/Azure Databricks for distributed compute, memory optimization, and Delta Lake formats. Data Engineering Languages: Advanced proficiency in SQL (T-SQL, SparkSQL) and Python (PySpark). Security & Compliance: Direct experience implementing security controls for sensitive or identifiable data, utilizing Azure Key Vault, Azure Monitor, Log Analytics, and Microsoft Purview. ...

Quality, Data Integration, Data Analytics, Data Visualisation, Data Science and Business Intelligence. Confident in delivering solutions using Databricks & developing and optimising data pipelines using PySpark within Databricks environments. Proven experience building, publishing, and supporting dashboards and reports using Power BI Strong experience and programming skills in languages such ...

standards and best practices. About you Extensive experience designing and delivering enterprise-scale data engineering solutions Strong hands-on experience with Azure Databricks, including PySpark, Delta Live Tables and Databricks Workflows and orchestration Microsoft Fabric experience, including Lakehouse, Data Factory, Warehouse and Spark Knowledge of Azure Data Platform services ...

deployment automation and infrastructure-as-code practices. What We're Looking For Essential Strong commercial experience building production-grade data pipelines using Apache Spark (PySpark preferred). Experience delivering modern data platforms using Databricks or an equivalent Spark-based platform. Strong SQL skills and experience with data modelling. Good ...

complex data workflows Excellent communication and collaboration skills in agile, cross-functional teams Ability to mentor junior engineers Nice to have: Experience with Databricks, PySpark, and Delta Lake Условия Competitive salary based on location, experience, and skills Comprehensive benefits package for employees and their families Continuous learning, structured development ...

architectures. Data Platforms & Cloud Technologies Hands-on experience with: Azure Data Lake Databricks Snowflake Delta Lake Azure Synapse Analytics/Microsoft Fabric Spark/PySpark Strong understanding of: Data Lake and Lakehouse architecture Hybrid cloud environments Enterprise integration patterns Modern data formats including Parquet, Avro, and ORC DevOps & Platform ...

Technical Environment Experience with some of the following technologies would be beneficial: Python SQL Cloud platforms – AWS, Azure or GCP Databricks Apache Spark/PySpark Kafka/event-driven architectures Airflow dbt Git/GitHub CI/CD Infrastructure as Code Modern data platforms and distributed processing technologies Experience ...

pipelines, stored procedures and reporting solutions to cloud-based platforms. Exposure to Azure DevOps, Git and CI/CD practices. Experience with Python, PySpark, Power BI or real estate technology platforms would be advantageous. Previous exposure to commercial real estate, investment management or alternative asset management environments is highly ...

deep Azure Synapse/Databricks experience with clear ability to specialise quickly in Fabric. Expert SQL/T-SQL plus strong Python or PySpark, with a track record of building maintainable ELT/ETL pipelines and analytical data models. Strong Power BI and DAX experience, including semantic modelling, incremental ...

Azure Databricks, Databricks, Azure Data Lake, ADLS, Azure SQL, Azure Synapse, Synapse Analytics, Azure PaaS, ETL, ELT, Data Pipelines, Data Integration, SQL, Python, PySpark, Azure DevOps, CI/CD, Data Governance, Data Quality, Cloud Security, Microsoft Fabric, Power BI, Power Platform. ...

data engineering or integration environments. 8–10 years’ experience in data engineering and integration , ideally in enterprise‐scale environments. Hands‐on experience with Spark (PySpark/SparkSQL) , ETL/ELT design, and data pipeline orchestration . Strong data modelling skills (Dimensional, ODS, Data Vault) and experience with data warehousing ...

Lake Storage (ADLS). Azure SQL Database and SQL Server. Microsoft Fabric (preferred). Advanced SQL development and performance tuning skills. Experience with Python, PySpark, or Scala for data processing. Knowledge of data warehousing concepts and dimensional modelling. Experience with CI/CD pipelines using Azure DevOps or GitHub. ...

insurance domain processes, including policy, claims, underwriting, pricing, risk, and regulatory reporting. Hands-on expertise with: Azure Data Factory (ADF) for pipeline orchestration Databricks (PySpark, Delta Lake, MLflow) for data engineering and ML integration Azure ecosystem (Key Vault, Data Lake, Synapse, Functions, Logic Apps) Strong understanding of AI/ ...

understanding of modern data architectures including Data Lakes, Data Warehousing, Lakehouse Architecture, and Medallion Design Patterns. Strong hands-on knowledge of Databricks, Spark/PySpark, SQL, and data engineering best practices. Expertise in data modelling techniques including dimensional modelling, Data Vault, and canonical data models. Strong understanding of data ...

Data Lake, Lakehouse, Data Warehouse, Data Vault ETL/ELT processes and designing, building and testing data pipelines Building data transformations using Python/PySpark Azure Cloud Version control and CI/CD tools, specifically Azure DevOps Service Analytics and MI products including MS Power BI Data catalogue & governance ...