1 to 25 of 164 Databricks Jobs in the City of London

Job Summary : We are seeking a highly skilled and experienced Senior Data Engineer to join our team and contribute to the development and maintenance of our cutting-edge Azure Databricks platform for economic data. This platform is critical for our Monetary Analysis, Forecasting, and Modelling activities. The Senior Data Engineer will be responsible for building and optimising data pipelines, implementing … Development & Optimisation : Design, develop, and maintain robust and scalable data pipelines for ingesting, transforming, and loading data from various sources (e.g., APIs, databases, financial data providers) into the Azure Databricks platform. Optimise data pipelines for performance, efficiency, and cost-effectiveness. Implement data quality checks and validation rules within data pipelines. Data Transformation & Processing: Implement complex data transformations using Spark (PySpark … or Scala) and other relevant technologies. Develop and maintain data processing logic for cleaning, enriching, and aggregating data. Ensure data consistency and accuracy throughout the data lifecycle. Azure Databricks Implementation: Work extensively with Azure Databricks Unity Catalog, including Delta Lake, Spark SQL, and other relevant services. Implement best practices for Databricks development and deployment. Optimise Databricks workloads for performance and More ❯

with BI/reporting tools such as Looker or PowerBI. Excellent communication and stakeholder management skills. Google Cloud Professional certifications. Experience in alternative cloud data platforms such as Snowflake, Databricks, Azure, or AWS. Understanding of DevOps/DataOps practices, CI/CD pipelines, and monitoring tools. Academic background in Computer Science, Mathematics, or a related technical field. What’s on More ❯

audiences. Problem-solving skills with a business focus and stakeholder management experience. Knowledge of change management and systems processes. Comfortable with cloud platforms and Microsoft stack tools like Synapse, Databricks, Azure Data Factory, and Azure DevOps. Programming experience in Python, SQL, T-SQL, and SSIS. Understanding of ELT/ETL processes, RESTful APIs, and integration concepts. What’s in it More ❯

relevant technologies to create and maintain data assets and reports for business insights. Assist in engineering and managing data models and pipelines within a cloud environment, utilizing technologies like Databricks, Spark, Delta Lake, and SQL. Contribute to the maintenance and enhancement of our progressive tech stack, which includes Python, PySpark, Logic Apps, Azure Functions, ADLS, Django, and ReactJs. Support the More ❯

greenfield MLOps pipelines that handle very large datasets. You will be responsible for building out a greenfield standaridised framework for Capital markets. The core platform is built on Azure Databricks Lakehouse, consolidating data from various front and Middle Office systems to support BI, MI, and advanced AI/ML analytics. As a lead, you will shape the MLOps framework and … data sources (orders, quotes, trades, risk, etc.). Essential Requirements: 2+ years of experience in MLOps and at least 3 years in AI/ML engineering. Knowledge in Azure Databricks and associated services. Proficiency with ML frameworks and libraries in Python. Proven experience deploying and maintaining LLM services and solutions. Expertise in Azure DevOps and GitHub Actions. Familiarity with Databricks … CLI and Databricks Job Bundle. Strong programming skills in Python and SQL; familiarity with Scala is a plus. Solid understanding of AI/ML algorithms, model training, evaluation (including hyperparameter tuning), deployment, monitoring, and governance. Experience in handling large datasets and performing data preparation and integration. Experience with Agile methodologies and SDLC practices. Strong problem-solving, analytical, and communication skills. More ❯

greenfield MLOps pipelines that handle very large datasets. You will be responsible for building out a greenfield standaridised framework for Capital markets. The core platform is built on Azure Databricks Lakehouse, consolidating data from various front and Middle Office systems to support BI, MI, and advanced AI/ML analytics. As a lead, you will shape the MLOps framework and … across various data sources (orders, quotes, trades, risk, etc.). 2+ years of experience in MLOps and at least 3 years in AI/ML engineering. Knowledge in Azure Databricks and associated services. Proficiency with ML frameworks and libraries in Python. Proven experience deploying and maintaining LLM services and solutions. Expertise in Azure DevOps and GitHub Actions. Familiarity with Databricks … CLI and Databricks Job Bundle. Strong programming skills in Python and SQL; familiarity with Scala is a plus. Solid understanding of AI/ML algorithms, model training, evaluation (including hyperparameter tuning), deployment, monitoring, and governance. Experience in handling large datasets and performing data preparation and integration. Experience with Agile methodologies and SDLC practices. Strong problem-solving, analytical, and communication skills. More ❯

detail, the ability to manage multiple projects, and a collaborative approach to working with regional and group teams. Key Responsibilities: Develop and maintain scalable data solutions on Azure (ADF, Databricks, Synapse, etc.) Maintain and enhance the integrity of the Data Lakehouse in partnership with regional and group data teams. Partner with analytical and reporting teams to ensure data is presented … evolution Support delivery of data models, APIs, ML integrations, and reporting tools Key Skills: Hands on experience designing and delivering solutions using Azure services including Azure Data Factory, Azure Databricks, Azure Synapse, Azure Storage, Azure DevOps. Proven track record in Data Engineering and supporting the business to gain true insight from data. Experience in data integration and modelling including ELT More ❯

ability to engage technical and non-technical stakeholders alike. The Head of Data Engineering & Insight will work within a modern, cloud-based BI ecosystem , including: Data Integration: Fivetran , HVR, Databricks , Apache Kafka, Google BigQuery , Google Analytics 4 Data Lake & Storage: Databricks Delta Lake, Amazon S3 Data Transformation: dbt Cloud Data Warehouse: Snowflake Analytics & Reporting: Power BI, Excel, Snowflake SQL REST … API Advanced Analytics: Databricks (AI & Machine Learning) Governance & Infrastructure: Centralised Data Catalogue & Access Control (Okta) Job Scheduling & Monitoring (AWS, Splunk) Agile Data Engineering with centralised code repositories BI Data Portal: Power BI However this coexists with a legacy tech stack , which theHead of Data Engineering & Insight will support the transition to the new architecture already in place. Qualifications/Education More ❯

translating these into technical deliverables. Motivated to expand technical skills. Desirable Experience with Microsoft BI Tools such as Power BI, SSRS & SQL Server. Experience with Azure Data Factory and Databricks (Python). Experience working with DevOps, IaC & CI/CD pipelines (e.g. Terraform and Databricks Asset Bundles). More ❯

analytical skills and attention to detail. Preferred/Bonus Skills: Experience with Microsoft Fabric (OneLake, DirectLake, Data Warehouse, Data Activator) is a significant plus. Knowledge of Azure Synapse Analytics, Databricks, or Power BI. Experience with automated data validation or scripting (e.g., Python, PowerShell). Familiarity with CI/CD processes in data environments. Relevant certifications (e.g., Microsoft Certified: Azure Data More ❯

infrastructure, improve data quality, and enable data-driven decision-making across the organization. Core Duties and Responsibilities Design, build, and maintain large-scale data pipelines using Microsoft Fabric and Databricks Develop and implement data architectures that meet business requirements and ensure data quality, security, and compliance Collaborate with wider Product & Engineering teams to integrate data pipelines with machine learning models … and cloud computing Skills Capabilities and Attributes Essential: Good experience in data engineering, with a focus on cloud-based data pipelines and architectures Strong expertise in Microsoft Fabric and Databricks, including data pipeline development, data warehousing, and data lake management Proficiency in Python, SQL, Scala, or Java Experience with data processing frameworks such as Apache Spark, Apache Beam, or Azure … with Azure Synapse Analytics, Azure Data Lake Storage, or other Azure data services Experience with agile development methodologies and version control systems such as Git Certification in Microsoft Azure, Databricks, or other relevant technologies What We Offer Save For Your Future - Equiniti Pension Plan; Equiniti matches your pension contributions up to 10% All Employee Long Term Incentive Plan (LTIP) - Gives More ❯

and brownfield projects. Comfortable working independently in a remote setup. Strong communication and stakeholder engagement skills. Nice to Have: Experience with other Azure services such as Azure Data Lake, Databricks, or Power BI. Familiarity with DevOps/DataOps processes and tools. Microsoft Certified: Fabric Analytics Engineer Associate Start Date: ASAP or July start acceptable Apply now to join a forward More ❯

drive technical innovation across client projects Document processes and contribute to internal knowledge repositories and best practice libraries Key Skills & Experience Strong hands-on experience with Azure tooling, including: Databricks, Data Factory, Data Lake, and Synapse (or similar data warehouse tools) Azure Analysis Services or comparable BI tooling Solid programming capability in: SQL, Python, Spark, and ideally DAX Familiarity with More ❯

aimed at maximizing value through data utilization. Knowled g e/Experience Expertise in Commercial/Procurement Analytics. Experience in SAP (S/4 Hana). Experience with Spark, Databricks, or similar data processing tools. Stron g technical proficiency in data modelin g , SQL, NoSQL databases, and data warehousing . Hands-on experience with data pipeline development, ETL processes, and More ❯

automate ETL pipelines using SQL, SSIS, and Azure Data Factory Support BI delivery, UAT testing, and project work across teams Contribute to our cloud shift using Azure Synapse and Databricks Maintain technical documentation and support data governance Tech Stack Platforms: SQL Server, Azure Synapse, Databricks ETL: SSIS (2014–2022), Azure Data Factory BI Tools: Power BI Other: Power Apps, Power More ❯

/item data sets for reporting and analytics Work with Informatica P360 (Product 360) to manage and optimise product data pipelines Develop data workflows and perform advanced analysis using Databricks and Azure Data Services Collaborate with cross-functional teams to gather requirements and translate business needs into technical specs Ensure high data quality, consistency, and alignment across systems Create dashboards … Strong hands-on experience with Informatica/P360 for MDM or product-related data initiatives Proficient in Azure Stack (Azure Data Factory, Azure SQL, Synapse, etc.) Solid experience with Databricks for big data processing and analysis Excellent SQL skills and ability to manipulate, interpret, and present complex data sets Understanding of data governance, data quality, and data modelling principles Strong More ❯

for statistical programming and data analysis Solid understanding of data warehousing, data mapping, and logical data modelling concepts Demonstrated experience with Azure data tools (e.g., Synapse, Data Factory, DataLake, Databricks) in a cloud-based architecture Excellent business analysis capabilities, including stakeholder engagement, process mapping, and requirements gathering Strong interpersonal, communication, and presentation skills, with the ability to translate complex data More ❯

. Oversee the development of dashboards, KPIs, and operational reports that support real-time and batch analytics across front-to-back processes. Ensure integration with enterprise data platforms (e.g. Databricks) and data warehouses. Monitor system performance, identify bottlenecks, and optimise data processing workflows for improved efficiency and scalability. Team Leadership & Governance Manage and mentor a team of BI developers, delivery More ❯

design sessions with stakeholders, ensuring solutions align with NHS requirements and best practices. Develop end-to-end data solutions leveraging Azure services, including Azure Synapse Analytics, Azure Data Factory, Databricks, and Azure SQL. Define data models, integration patterns, and governance frameworks to ensure efficient data management, interoperability, and compliance. Drive cloud migration strategies for NHS data systems, modernizing legacy environments More ❯

Senior Data Architect/Strategic Data Architect/Data Solution Architect Rate: £600-£650 p/d IR35: Outside IR35 Time on site: 1 day a week on site in London ideally but very flexible for the right person, would More ❯

Data Engineer – Python | Databricks | PySpark Company: Fortune 500 Financial Services firm Location: Hybrid - London Type: Permanent Salary: £90k + 20% bonus + Exceptional Benefits Exclusively via DATAHEAD Are you a detail-oriented Python Developer who thrives in complex data environments? Do you have hands-on experience with Databricks , PySpark , and cloud-native data engineering? We’re hiring a Data Engineer … first, engineering-led environment where you’ll play a key role in developing high-quality, scalable data products. What You’ll Do: Build and maintain scalable Python applications using Databricks and PySpark Design and optimise robust data pipelines and processing frameworks Write clean, modular, and testable code aligned with SOLID principles Contribute to CI/CD pipelines , automated testing frameworks … Help shape the data architecture that underpins machine learning and AI models What You’ll Bring: Proven experience developing in Python for data-intensive applications Hands-on expertise with Databricks and PySpark Strong grasp of cloud data platforms and modern engineering practices Familiarity with CI/CD, version control (e.g. Git), and automated testing Ability to work effectively in cross More ❯

a skilled and motivated Lead Data Engineer with Insurance experience to play a key role in the creation of a brand-new data platform within the Azure ecosystem and Databricks . This is an exciting opportunity to be at the forefront of data innovation, working within a newly formed Data & Analytics team in a long standing London Market Insurer . … a Lead Data Engineer, you'll work closely with both technical and business stakeholders, leveraging your expertise to design, develop, and optimize a high-performance data platform built on Databricks . This platform will be built to scale, incorporating the latest advancements in data intelligence while supporting strategic business objectives. Key Responsibilities 🔹 Build & Develop – Design and maintain a robust Databricks … solutions. 🔹 Futureproofing – Drive the evolution of the data platform, ensuring adaptability for new data sources, analytical models, and emerging technologies. What You’ll Bring ✅ Extensive hands-on experience with Databricks , and Microsoft Azure data tools (must-have: Azure Data Factory, Azure Synapse, or Azure SQL). ✅ Dimensional modelling expertise for analytics use cases. ✅ Strong ETL/ELT development skills. ✅ Python More ❯

Data Engineer – Investment Banking – London/Hybrid (Data Engineer, SQL Data Engineer, Java, Python, Spark, Scala, SQL, Snowflake, OO programming, Snowflake, Databricks, Data Fabric, design patterns, SOLID principles, ETL, Unit testing, NUnit, MSTest, Junit, Microservices Architecture, Continuous Integration, Azure DevOps, AWS, Jenkins, Agile, Data Engineer, SQL Data Engineer) We have several fantastic new roles including a Data Engineer position to … teams to join their new and growing IT team! They are looking for an experienced Lead Data Engineer with expert level Java/Python, as well as Snowflake/Databricks to join an exceptional core engineering team and deliver features across their Data Engineering platform. We are seeking a Data Engineer who has advanced working knowledge of Snowflake/Databricks More ❯

Data Engineer - Investment Banking - London/Hybrid (Data Engineer, SQL Data Engineer, Java, Python, Spark, Scala, SQL, Snowflake, OO programming, Databricks, Data Fabric, design patterns, SOLID principles, ETL, Unit testing, NUnit, MSTest, Junit, Microservices Architecture, Continuous Integration, Azure DevOps, AWS, Jenkins, Agile, Data Engineer, SQL Data Engineer) We have several fantastic new roles, including a Data Engineer position to join … to join their new and growing IT division. They are looking for an experienced Lead Data Engineer with expert-level Java/Python skills, as well as Snowflake and Databricks experience, to join their core engineering team and deliver features across their Data Engineering platform. We are seeking a Data Engineer who has advanced working knowledge of Snowflake/Databricks More ❯

data processes within insurance, with a focus on delivering solutions while engaging with senior business stakeholders. Key Responsibilities: Develop and implement data solutions on the Microsoft Azure platform (Azure Databricks, Azure SQL Database). Provide technical guidance to the project development team. Bridge communication gaps between offshore, third-party, internal teams, and business stakeholders. Track and manage development progress, working … needs. Manage tasks and ensure agreed-upon delivery timelines. Key Skills & Experience: Expertise in database design, performance optimization, and data warehousing. Strong experience with Azure data technologies, including Azure Databricks, SQL Database, Data Factory, and T-SQL. In-depth understanding of London Market insurance data and Actuarial/Finance business processes. Experience with ETL processes, data modelling, and dimensional modelling More ❯