1 to 25 of 150 Databricks Jobs in the City of London

Data Engineer - Leading Energy Company - London (Tech Stack: Data Engineer, Databricks, Python, PySpark, Power BI, AWS QuickSight, AWS, TSQL, ETL, Agile Methodologies) Company Overview: Join a dynamic team, a leading player in the energy sector, committed to innovation and sustainable solutions. Our client are seeking a talented Data Engineer to help build and optimise our data infrastructure, enabling them to More ❯

Databricks Architect (Contract) - Greenfield Data Platform Location: Hybrid working (London) Duration: 12-month initial contract Are you a visionary Databricks Architect with a passion for building cutting-edge data platforms from the ground up? Do you thrive on shaping strategy and driving technical excellence in a greenfield environment? Our client is embarking on a pivotal journey to establish a brand … future data-driven initiatives, from advanced analytics to AI/ML. We are looking for a hands-on architect who can translate business needs into robust, scalable, and secure Databricks solutions. The Role: As our Databricks Architect, you will be instrumental in defining and delivering our new data strategy and architecture. This is a greenfield project, meaning you'll have … the exciting challenge of building the entire Databricks Lakehouse Platform from scratch. You will provide critical technical leadership, guidance, and hands-on expertise to ensure the successful establishment of a scalable, high-performance, and future-proof data environment. Phase 1: Strategic Vision & Blueprint Data Strategy & Roadmap: Collaborate with business stakeholders and leadership to define the overarching data vision, strategy, and More ❯

Job Summary : We are seeking a highly skilled and experienced Senior Data Engineer to join our team and contribute to the development and maintenance of our cutting-edge Azure Databricks platform for economic data. This platform is critical for our Monetary Analysis, Forecasting, and Modelling activities. The Senior Data Engineer will be responsible for building and optimising data pipelines, implementing … Development & Optimisation : Design, develop, and maintain robust and scalable data pipelines for ingesting, transforming, and loading data from various sources (e.g., APIs, databases, financial data providers) into the Azure Databricks platform. Optimise data pipelines for performance, efficiency, and cost-effectiveness. Implement data quality checks and validation rules within data pipelines. Data Transformation & Processing: Implement complex data transformations using Spark (PySpark … or Scala) and other relevant technologies. Develop and maintain data processing logic for cleaning, enriching, and aggregating data. Ensure data consistency and accuracy throughout the data lifecycle. Azure Databricks Implementation: Work extensively with Azure Databricks Unity Catalog, including Delta Lake, Spark SQL, and other relevant services. Implement best practices for Databricks development and deployment. Optimise Databricks workloads for performance and More ❯

an experienced Azure Data Engineer to play a key role in delivering the next phase of a business-critical data programme. You'll apply your technical expertise across Azure Databricks, Data Factory, Azure SQL, and SQL Server, contributing directly to the integration of financial data into a new Azure Data Platform. Responsibilities: Design, build and optimise scalable data solutions using … Azure cloud technologies. Lead the development of data pipelines in Azure Databricks, Data Factory, and T-SQL. Provide senior-level technical direction to onshore and offshore development teams. Translate data requirements into effective data engineering solutions. Collaborate closely with business stakeholders to ensure alignment with project goals. Maintain detailed documentation and ensure robust testing practices. Plan and manage progress in … Agile sprints alongside project managers and scrum masters. Requirements: Extensive experience in Azure data engineering, with strong expertise in Databricks, Azure SQL, and Data Factory. Deep technical knowledge of SQL Server including stored procedures and complex data transformation logic. Proven experience in designing and delivering data warehousing and dimensional modelling solutions. Excellent collaboration skills with a track record of working More ❯

from across the EMEA region, as long as you're available to work standard EMEA business hours Responsibilities Build and maintain data pipelines using Azure Data Factory, Synapse, and Databricks Develop Power BI dashboards with strong attention to data modeling, DAX, and Power Query (M) Contribute to the design of semantic models, data marts, and data warehouse structures Integrate data … peer review or architectural debates Qualifications 5+ years of experience in data engineering with a strong focus on Microsoft Azure Practical experience with Azure Data Factory, Synapse Analytics, Azure Databricks, and Azure SQL Advanced Power BI development skills, including DAX, Power Query, and visual/report design Strong SQL proficiency, including performance optimization and query tuning Extensive, hands-on experience More ❯

with BI/reporting tools such as Looker or PowerBI. Excellent communication and stakeholder management skills. Google Cloud Professional certifications. Experience in alternative cloud data platforms such as Snowflake, Databricks, Azure, or AWS. Understanding of DevOps/DataOps practices, CI/CD pipelines, and monitoring tools. Academic background in Computer Science, Mathematics, or a related technical field. What’s on More ❯

transformation, and we’re looking for a talented Data Engineer to join our growing data team. In this role, you’ll be instrumental in building and deploying modern Azure Databricks based data solutions, enabling the business to make faster, data-driven decisions. You’ll work hands-on with Azure Databricks, Azure Data Factory, Delta Lake, and Power BI to design … data at Zodiac Maritime while working with cutting-edge cloud technologies. Key responsibilities and primary deliverables Design, develop, and optimize end-to-end data pipelines (batch & streaming) using Azure Databricks, Spark, and Delta Lake. Implement Medallion Architecture to structure raw, enriched, and curated data layers efficiently. Build scalable ETL/ELT processes with Azure Data Factory and PySpark. Work with … streaming with Kafka/Event Hubs, Knowledge Graphs). Advocate for best practices in data engineering across the organization. Skills profile Relevant experience & education Hands-on experience with Azure Databricks, Delta Lake, Data Factory, and Synapse. Strong understanding of Lakehouse architecture and medallion design patterns. Proficient in Python, PySpark, and SQL (advanced query optimization). Experience building scalable ETL pipelines More ❯

greenfield MLOps pipelines that handle very large datasets. You will be responsible for building out a greenfield standaridised framework for Capital markets. The core platform is built on Azure Databricks Lakehouse, consolidating data from various front and Middle Office systems to support BI, MI, and advanced AI/ML analytics. As a lead, you will shape the MLOps framework and … data sources (orders, quotes, trades, risk, etc.). Essential Requirements: 2+ years of experience in MLOps and at least 3 years in AI/ML engineering. Knowledge in Azure Databricks and associated services. Proficiency with ML frameworks and libraries in Python. Proven experience deploying and maintaining LLM services and solutions. Expertise in Azure DevOps and GitHub Actions. Familiarity with Databricks … CLI and Databricks Job Bundle. Strong programming skills in Python and SQL; familiarity with Scala is a plus. Solid understanding of AI/ML algorithms, model training, evaluation (including hyperparameter tuning), deployment, monitoring, and governance. Experience in handling large datasets and performing data preparation and integration. Experience with Agile methodologies and SDLC practices. Strong problem-solving, analytical, and communication skills. More ❯

greenfield MLOps pipelines that handle very large datasets. You will be responsible for building out a greenfield standaridised framework for Capital markets. The core platform is built on Azure Databricks Lakehouse, consolidating data from various front and Middle Office systems to support BI, MI, and advanced AI/ML analytics. As a lead, you will shape the MLOps framework and … across various data sources (orders, quotes, trades, risk, etc.). 2+ years of experience in MLOps and at least 3 years in AI/ML engineering. Knowledge in Azure Databricks and associated services. Proficiency with ML frameworks and libraries in Python. Proven experience deploying and maintaining LLM services and solutions. Expertise in Azure DevOps and GitHub Actions. Familiarity with Databricks … CLI and Databricks Job Bundle. Strong programming skills in Python and SQL; familiarity with Scala is a plus. Solid understanding of AI/ML algorithms, model training, evaluation (including hyperparameter tuning), deployment, monitoring, and governance. Experience in handling large datasets and performing data preparation and integration. Experience with Agile methodologies and SDLC practices. Strong problem-solving, analytical, and communication skills. More ❯

with the ability to engage technical and non-technical stakeholders alike. The Head of BI will work within a modern, cloud-based BI ecosystem , including : Data Integration: Fivetran , HVR, Databricks , Apache Kafka, Google BigQuery , Google Analytics 4 Data Lake & Storage: Databricks Delta Lake, Amazon S3 Data Transformation: dbt Cloud Data Warehouse: Snowflake Analytics & Reporting: Power BI, Excel, Snowflake SQL REST … API Advanced Analytics: Databricks (AI & Machine Learning) Governance & Infrastructure: Centralised Data Catalogue & Access Control (Okta) Job Scheduling & Monitoring (AWS, Splunk) Agile Data Engineering with centralised code repositories BI Data Portal: Power BI However this coexists with a legacy tech stack , which the head of BI will support the transition to the new architecture already in place . Qualifications/Education More ❯

translating these into technical deliverables. Motivated to expand technical skills. Desirable Experience with Microsoft BI Tools such as Power BI, SSRS & SQL Server. Experience with Azure Data Factory and Databricks (Python). Experience working with DevOps, IaC & CI/CD pipelines (e.g. Terraform and Databricks Asset Bundles). More ❯

performance data pipelines (ETL/ELT) to ingest, transform, and deliver data from diverse sources. Required Skills & Experience: Deep expertise in building and deploying data solutions using platforms like Databricks (Azure/AWS) , Microsoft Fabric , Azure Data Factory , and Azure Synapse . Solid experience with the Microsoft data stack —including SSIS , SSAS , and SSRS . A strong foundation in data More ❯

Senior Databricks Data Engineer | Energy Trading - Contract My client is an Energy Trading Client based in the UK who are looking to appoint a Senior Databricks Data Engineer to help modernise their data infrastructure and deliver end-to-end solutions that drive business value. This is a high-impact role working across data strategy, architecture, and engineering to deliver cloud … AWS, Azure or SAP ETL/ELT Development Data Modeling Data Integration & Ingestion Data Manipulation & Processing Version Control & DevOps: Skilled in GitHub, GitHub Actions, Azure DevOps Azure Data Factory, Databricks, SQL DB, Synapse, Stream Analytics Glue, Airflow, Kinesis, Redshift SonarQube, PyTest If you're ready to take on a new challenge and shape data engineering in a trading-first environment More ❯

My client is based in the London area are currently looking to recruit for an experienced AWS Databricks Data Architect to join their team. They are one of the leaders within the consulting space and are currently going through a period of growth and are looking for an experienced Data Architect to join their team. They are backed by a … utilise the newest technologies on the market. Your role will include: Responsible for designing and implementing effective Architectural solutions around the AWS Severless Technologies (S3, Lambda, Athena, Kafka) and Databricks including Data Lake and Data Warehousing. Assess database implementation procedures to ensure they comply with GDPR and data compliance. Guide, influence and challenge the technology teams and stakeholders to understand … extensive knowledge of writing codes, building data pipelines and doing digital transformation and ingestion a certain tech suite. Extensive experience in implementing solutions around the AWS cloud environment (S3, Databricks, Athena, Glue), In depth understanding of Workflows, Asset Bundles, SQS, EKS, Terraform, Excellent understanding of Data Modelling & Kinesis An understanding of SQL/database management. Strong hands-on experience in More ❯

and business discussions. Desirable Qualifications and Experience Degree (BSc, MSc, or PhD) in Computer Science, Mathematics, or a related field. Familiarity with alternative cloud and data platforms such as Databricks, Snowflake, Azure, or AWS. Knowledge of DevOps/DataOps methodologies and experience with CI/CD pipelines. Understanding of monitoring, logging, and troubleshooting in cloud-based environments. Google Cloud Professional More ❯

drive technical innovation across client projects Document processes and contribute to internal knowledge repositories and best practice libraries Key Skills & Experience Strong hands-on experience with Azure tooling, including: Databricks, Data Factory, Data Lake, and Synapse (or similar data warehouse tools) Azure Analysis Services or comparable BI tooling Solid programming capability in: SQL, Python, Spark, and ideally DAX Familiarity with More ❯

aimed at maximizing value through data utilization. Knowled g e/Experience Expertise in Commercial/Procurement Analytics. Experience in SAP (S/4 Hana). Experience with Spark, Databricks, or similar data processing tools. Stron g technical proficiency in data modelin g , SQL, NoSQL databases, and data warehousing . Hands-on experience with data pipeline development, ETL processes, and More ❯

abreast of opportunities for innovation with new tools and uses of data. Essential Skills and Experience: Educated to degree level or have equivalent professional experience. Experience of MS Azure Databricks Experience working with Database technologies such as SQL Server, and Data Warehouse Architecture with knowledge of big data, data lakes and NoSQL. Experience following product/solution development lifecycles using More ❯

automate ETL pipelines using SQL, SSIS, and Azure Data Factory Support BI delivery, UAT testing, and project work across teams Contribute to our cloud shift using Azure Synapse and Databricks Maintain technical documentation and support data governance Tech Stack Platforms: SQL Server, Azure Synapse, Databricks ETL: SSIS (2014–2022), Azure Data Factory BI Tools: Power BI Other: Power Apps, Power More ❯

design sessions with stakeholders, ensuring solutions align with NHS requirements and best practices. Develop end-to-end data solutions leveraging Azure services, including Azure Synapse Analytics, Azure Data Factory, Databricks, and Azure SQL. Define data models, integration patterns, and governance frameworks to ensure efficient data management, interoperability, and compliance. Drive cloud migration strategies for NHS data systems, modernizing legacy environments … Skills & Experience: Proven experience in architecting Azure data solutions, particularly in NHS or healthcare settings. Expertise in Azure Data Services, such as: Azure Synapse Analytics Azure Data Factory Azure Databricks Azure SQL & Cosmos DB Azure Purview (for Data Governance) Azure Blob Storage & Data Lake Azure Event Hubs & Streaming Analytics Hands-on experience with data integration, ETL/ELT pipelines, and More ❯

Master’s, or PhD in Computer Science, Mathematics, or a related field Familiarity with BI tools such as Looker for reporting and dashboarding Exposure to other environments such as Databricks, Snowflake, AWS, Azure, or DBT Understanding of observability, monitoring, and logging in GCP GCP Professional Data Engineer certification (or similar) What’s on Offer Competitive salary of £80,000 + More ❯

a skilled and motivated Lead Data Engineer with Insurance experience to play a key role in the creation of a brand-new data platform within the Azure ecosystem and Databricks . This is an exciting opportunity to be at the forefront of data innovation, working within a newly formed Data & Analytics team in a long standing London Market Insurer . … a Lead Data Engineer, you'll work closely with both technical and business stakeholders, leveraging your expertise to design, develop, and optimize a high-performance data platform built on Databricks . This platform will be built to scale, incorporating the latest advancements in data intelligence while supporting strategic business objectives. Key Responsibilities 🔹 Build & Develop – Design and maintain a robust Databricks … solutions. 🔹 Futureproofing – Drive the evolution of the data platform, ensuring adaptability for new data sources, analytical models, and emerging technologies. What You’ll Bring ✅ Extensive hands-on experience with Databricks , and Microsoft Azure data tools (must-have: Azure Data Factory, Azure Synapse, or Azure SQL). ✅ Dimensional modelling expertise for analytics use cases. ✅ Strong ETL/ELT development skills. ✅ Python More ❯

Data Engineer – Investment Banking – London/Hybrid (Data Engineer, SQL Data Engineer, Java, Python, Spark, Scala, SQL, Snowflake, OO programming, Snowflake, Databricks, Data Fabric, design patterns, SOLID principles, ETL, Unit testing, NUnit, MSTest, Junit, Microservices Architecture, Continuous Integration, Azure DevOps, AWS, Jenkins, Agile, Data Engineer, SQL Data Engineer) We have several fantastic new roles including a Data Engineer position to … teams to join their new and growing IT team! They are looking for an experienced Lead Data Engineer with expert level Java/Python, as well as Snowflake/Databricks to join an exceptional core engineering team and deliver features across their Data Engineering platform. We are seeking a Data Engineer who has advanced working knowledge of Snowflake/Databricks More ❯

Data Engineer - Investment Banking - London/Hybrid (Data Engineer, SQL Data Engineer, Java, Python, Spark, Scala, SQL, Snowflake, OO programming, Databricks, Data Fabric, design patterns, SOLID principles, ETL, Unit testing, NUnit, MSTest, Junit, Microservices Architecture, Continuous Integration, Azure DevOps, AWS, Jenkins, Agile, Data Engineer, SQL Data Engineer) We have several fantastic new roles, including a Data Engineer position to join … to join their new and growing IT division. They are looking for an experienced Lead Data Engineer with expert-level Java/Python skills, as well as Snowflake and Databricks experience, to join their core engineering team and deliver features across their Data Engineering platform. We are seeking a Data Engineer who has advanced working knowledge of Snowflake/Databricks More ❯

Exposure across all the SDLC process, including testing and deployment. Expertise in Microsoft Azure is mandatory including components like Azure Data Factory, Azure Data Lake Storage, Azure SQL, Azure DataBricks, HD Insights, ML Service etc. Good knowledge of Python and Spark are required. Experience in ETL & ELT Good understanding of one scripting language Good understanding of how to enable analytics … a diverse and challenging set of customers to success. Good understanding of the CPG (Consumer Packaged Goods) domain is preferred. Skills: Data Ops, ML Ops, Deep expertise in Azure Databricks, ETL frameworks. Expertise in Microsoft Azure is mandatory including components like Azure Data Factory, Azure Data Lake Storage, Azure SQL, Azure DataBricks, HD Insights, ML Service etc. Fractal provides equal More ❯