1 to 25 of 131 Permanent PySpark Jobs in London

with Lloyd’s, bordereaux, reinsurance, or regulatory and actuarial reporting. Hands‐on experience designing solutions on Azure Databricks, including Delta Lake, Unity Catalog, Spark (PySpark and Spark SQL) and Databricks SQL. Experience with core Azure data services and their integration with Databricks, such as ADLS Gen2, Azure Data Factory ...

Azure Databricks Azure Function Apps& Logic Apps Azure Stream Analytics Azure Resource Manager skills (Terraform, Azure Portal, Az CLI and Az PowerShell) Strong PySpark, Delta Lake, Unity Catalog and Python skills. Includes ability to write unit and integration tests in Python with unittest, pytest, etc. Strong understanding of software ...

with one or more cloud data ecosystems: Azure (Data Factory, Synapse, Fabric, ADLS, Databricks) AWS (Glue, Redshift, EMR) GCP (BigQuery, Dataflow) Databricks (Spark/PySpark) Snowflake (SQL, Snowpark, performance tuning) Strong SQL and/or Python capability, and familiarity with tools such as dbt, Git, CI/ ...

models in data warehouses, lakehouse environments or analytics tools such as Power BI. Familiarity with modern data engineering frameworks and tools (e.g. Spark/PySpark, Pandas, orchestration tools). Experience working with cloud data platforms on AWS or Azure (e.g. Databricks, Redshift, Microsoft Fabric). Understanding of software engineering ...

Lake Storage, Azure Data Factory, , MS Fabric ETL/ELT processes and designing, building and testing data pipelines Building data transformations using python/PySpark Azure Cloud Version control and CI/CD tools, specifically Azure DevOps Service Analytics and MI products including MS Power BI Data catalogue & governance ...

data warehouses or similar analytical platforms. Experience developing reliable ETL or ELT processes in the Medallion architecture. Experience with Microsoft Fabric data engineering, including Pyspark and dbt. Practical knowledge of data quality, testing, monitoring and operational support. Strong problem-solving skills and attention to detail. Clear communication ...

Good understanding of modern software platform architecture and the role of the data platform within the broader platform ecosystem Good Scala skills (or strong PySpark plus a genuine willingness to work primarily in Scala) Strong SQL skills for transformations, data modelling, and debugging data issues Experience building and supporting ...

deployment automation and infrastructure-as-code practices. What We're Looking For Essential Strong commercial experience building production-grade data pipelines using Apache Spark (PySpark preferred). Experience delivering modern data platforms using Databricks or an equivalent Spark-based platform. Strong SQL skills and experience with data modelling. Good ...

data-science role, including delivery of reliable pipelines, reusable data assets and useful analytical models in Microsoft Fabric and Databricks. Advanced Python, SQL and PySpark, including experience with data transformation, analysis and common machine-learning libraries. Hands-on experience designing and operating scalable data pipelines, lakehouse data models, Delta ...

deployment automation and infrastructure-as-code practices. What We're Looking For Essential Strong commercial experience building production-grade data pipelines using Apache Spark (PySpark preferred). Experience delivering modern data platforms using Databricks or an equivalent Spark-based platform. Strong SQL skills and experience with data modelling. Good ...

Data Lake, Lakehouse, Data Warehouse, Data Vault ETL/ELT processes and designing, building and testing data pipelines Building data transformations using Python/PySpark Azure Cloud Version control and CI/CD tools, specifically Azure DevOps Service Analytics and MI products including MS Power BI Data catalogue & governance ...

management. Experience integrating education or CRM systems such as an LMS, HubSpot or similar platforms. Knowledge of Power BI, including semantic models. Python or PySpark for data transformation. Microsoft Fabric or Boomi certification. Why join Corndel Work flexibly and stay connected – most of our people work remotely, supported ...

enterprise data source integration — MS SQL Server (on-premises and Azure SQL) and Oracle Database ingestion into Azure landing zones Strong Apache Spark skills — PySpark or Scala, optimisation, partitioning, and job scheduling in Databricks Experience assessing and remediating partially built or vendor-abandoned data platform implementations Proven ability ...

vs. centralized models, capacity (F-SKU) sizing and licensingDesign end-to-end Microsoft Fabric solutions: OneLake, Lakehouse, Warehouse, Data Factory pipelines, Dataflows Gen2, Notebooks (PySpark/Spark SQL), Real-Time Intelligence (Eventstreams, Eventhouse/KQL) and Power BI + Power BI CoPilotDesign governance, security and operating frameworks: data governance ...

Strong problem-solving and analytical skills. Desirable Experience with Microsoft Fabric. Experience with Azure Data Services. Working with multi-currency financial information. Knowledge of PySpark or Spark SQL. Understanding of Microsoft Entra ID and security frameworks. Experience with DevOps and automated deployment pipelines. Knowledge of financial, legal, or professional ...

data platform on Microsoft Fabric, Databricks, Snowflake or similar, using a medallion architecture, including managing ingestion performance and forecasting platform costs. Strong Python/PySpark and SQL, with a track record of production ETL/ELT pipelines integrating data from varied business systems, including less well-documented sources. Experience ...

Strong problem-solving and analytical skills. Desirable Experience with Microsoft Fabric. Experience with Azure Data Services. Working with multi-currency financial information Knowledge of PySpark or Spark SQL. Understanding of Microsoft Entra ID and security frameworks. Experience with DevOps and automated deployment pipelines. Knowledge of financial, legal, or professional ...

/CD deployment practices. Strong problem-solving and analytical skills.DesirableExperience with Microsoft Fabric. Experience with Azure Data Services.Working with multi-currency financial informationKnowledge of PySpark or Spark SQL.Understanding of Microsoft Entra ID and security frameworks.Experience with DevOps and automated deployment pipelines.Knowledge of financial, legal, or professional services data environments.Elite ...

diversity of views and opinions. Key Skills & Experience: Proven experience in a Data Engineering role within the insurance industry Advanced SQL and PySpark skills, with hands-on experience building data transformations and data products Demonstrable data modelling experience, including practical experience with medallion architecture, and comfortable contributing technical input ...

orchestrate the ingestion of data from disparate source systems into a centralised data analytics platform. Designing and implementingdata engineering solutions using T-SQL, Python, PySpark and DBT in the Azurecloud environment. Working with Data Analystsin promoting business logic into DBT data models, to support business reportsand dashboards. Maintaining ...

relevant discipline or equivalent practical experience. Desirable Experience Strong hands‐on SQL and Python skills, with experience in distributed data processing using PySpark and/or Databricks. Experience with AWS services, APIs, event‐driven integration and infrastructure as code. Exposure to AI/ML data preparation, data observability, metadata ...

degree in a relevant discipline or equivalent practical experience. Strong hands-on SQL and Python skills, with experience in distributed data processing using PySpark and/or Databricks. Experience with AWS services, APIs, event-driven integration and infrastructure as code. Exposure to AI/ML data preparation, data observability ...

/Azure Databricks for distributed compute, memory optimization, and Delta Lake formats. Data Engineering Languages: Advanced proficiency in SQL (T-SQL, SparkSQL) and Python (PySpark). Security & Compliance: Direct experience implementing security controls for sensitive or identifiable data, utilizing Azure Key Vault, Azure Monitor, Log Analytics, and Microsoft Purview. ...

Data Analytics/Warehousing & Business Intelligence Experience 5-10 years Location United Kingdom - London Skills Python & SQL: Advanced Python (PySpark, Advanced SQL (CTEs, Databricks We are looking for an experienced Principal Data Engineer to take technical ownership of our customer decisioning pipeline architecture, end-to-end orchestration, and production ...

only on ETL. However, we will consider candidates who bring Cloud + CI/CD experience alongside technologies such as: Proficient SQL; familiarity with PySpark is beneficial dbt Microsoft Fabric SQLMesh 5. Core Engineering Skills Strong Python, with good documentation and unit testing practices Comfortable in an agile delivery ...