1 to 25 of 125 Permanent Databricks Jobs in Central London

Data Engineer - Leading Energy Company - London (Tech Stack: Data Engineer, Databricks, Python, PySpark, Power BI, AWS QuickSight, AWS, TSQL, ETL, Agile Methodologies) Company Overview: Join a dynamic team, a leading player in the energy sector, committed to innovation and sustainable solutions. Our client are seeking a talented Data Engineer to help build and optimise our data infrastructure, enabling them to More ❯

Job Summary : We are seeking a highly skilled and experienced Senior Data Engineer to join our team and contribute to the development and maintenance of our cutting-edge Azure Databricks platform for economic data. This platform is critical for our Monetary Analysis, Forecasting, and Modelling activities. The Senior Data Engineer will be responsible for building and optimising data pipelines, implementing … Development & Optimisation : Design, develop, and maintain robust and scalable data pipelines for ingesting, transforming, and loading data from various sources (e.g., APIs, databases, financial data providers) into the Azure Databricks platform. Optimise data pipelines for performance, efficiency, and cost-effectiveness. Implement data quality checks and validation rules within data pipelines. Data Transformation & Processing: Implement complex data transformations using Spark (PySpark … or Scala) and other relevant technologies. Develop and maintain data processing logic for cleaning, enriching, and aggregating data. Ensure data consistency and accuracy throughout the data lifecycle. Azure Databricks Implementation: Work extensively with Azure Databricks Unity Catalog, including Delta Lake, Spark SQL, and other relevant services. Implement best practices for Databricks development and deployment. Optimise Databricks workloads for performance and More ❯

greenfield MLOps pipelines that handle very large datasets. You will be responsible for building out a greenfield standaridised framework for Capital markets. The core platform is built on Azure Databricks Lakehouse, consolidating data from various front and Middle Office systems to support BI, MI, and advanced AI/ML analytics. As a lead, you will shape the MLOps framework and … data sources (orders, quotes, trades, risk, etc.). Essential Requirements: 2+ years of experience in MLOps and at least 3 years in AI/ML engineering. Knowledge in Azure Databricks and associated services. Proficiency with ML frameworks and libraries in Python. Proven experience deploying and maintaining LLM services and solutions. Expertise in Azure DevOps and GitHub Actions. Familiarity with Databricks … CLI and Databricks Job Bundle. Strong programming skills in Python and SQL; familiarity with Scala is a plus. Solid understanding of AI/ML algorithms, model training, evaluation (including hyperparameter tuning), deployment, monitoring, and governance. Experience in handling large datasets and performing data preparation and integration. Experience with Agile methodologies and SDLC practices. Strong problem-solving, analytical, and communication skills. More ❯

with the ability to engage technical and non-technical stakeholders alike. The Head of BI will work within a modern, cloud-based BI ecosystem , including : Data Integration: Fivetran , HVR, Databricks , Apache Kafka, Google BigQuery , Google Analytics 4 Data Lake & Storage: Databricks Delta Lake, Amazon S3 Data Transformation: dbt Cloud Data Warehouse: Snowflake Analytics & Reporting: Power BI, Excel, Snowflake SQL REST … API Advanced Analytics: Databricks (AI & Machine Learning) Governance & Infrastructure: Centralised Data Catalogue & Access Control (Okta) Job Scheduling & Monitoring (AWS, Splunk) Agile Data Engineering with centralised code repositories BI Data Portal: Power BI However this coexists with a legacy tech stack , which the head of BI will support the transition to the new architecture already in place . Qualifications/Education More ❯

translating these into technical deliverables. Motivated to expand technical skills. Desirable Experience with Microsoft BI Tools such as Power BI, SSRS & SQL Server. Experience with Azure Data Factory and Databricks (Python). Experience working with DevOps, IaC & CI/CD pipelines (e.g. Terraform and Databricks Asset Bundles). More ❯

performance data pipelines (ETL/ELT) to ingest, transform, and deliver data from diverse sources. Required Skills & Experience: Deep expertise in building and deploying data solutions using platforms like Databricks (Azure/AWS) , Microsoft Fabric , Azure Data Factory , and Azure Synapse . Solid experience with the Microsoft data stack —including SSIS , SSAS , and SSRS . A strong foundation in data More ❯

Senior Databricks Data Engineer | Energy Trading - Contract My client is an Energy Trading Client based in the UK who are looking to appoint a Senior Databricks Data Engineer to help modernise their data infrastructure and deliver end-to-end solutions that drive business value. This is a high-impact role working across data strategy, architecture, and engineering to deliver cloud … AWS, Azure or SAP ETL/ELT Development Data Modeling Data Integration & Ingestion Data Manipulation & Processing Version Control & DevOps: Skilled in GitHub, GitHub Actions, Azure DevOps Azure Data Factory, Databricks, SQL DB, Synapse, Stream Analytics Glue, Airflow, Kinesis, Redshift SonarQube, PyTest If you're ready to take on a new challenge and shape data engineering in a trading-first environment More ❯

My client is based in the London area are currently looking to recruit for an experienced AWS Databricks Data Architect to join their team. They are one of the leaders within the consulting space and are currently going through a period of growth and are looking for an experienced Data Architect to join their team. They are backed by a … utilise the newest technologies on the market. Your role will include: Responsible for designing and implementing effective Architectural solutions around the AWS Severless Technologies (S3, Lambda, Athena, Kafka) and Databricks including Data Lake and Data Warehousing. Assess database implementation procedures to ensure they comply with GDPR and data compliance. Guide, influence and challenge the technology teams and stakeholders to understand … extensive knowledge of writing codes, building data pipelines and doing digital transformation and ingestion a certain tech suite. Extensive experience in implementing solutions around the AWS cloud environment (S3, Databricks, Athena, Glue), In depth understanding of Workflows, Asset Bundles, SQS, EKS, Terraform, Excellent understanding of Data Modelling & Kinesis An understanding of SQL/database management. Strong hands-on experience in More ❯

closely with business and technical stakeholders to deliver robust, scalable, and secure data platforms. Key Responsibilities Design and implement modern data architectures using Azure Synapse , Data Lake , Data Factory , Databricks , and Microsoft Fabric . Lead the development of integrated data solutions supporting BI, advanced analytics, and real-time data processing. Define data governance, security, and compliance standards across the data More ❯

and business discussions. Desirable Qualifications and Experience Degree (BSc, MSc, or PhD) in Computer Science, Mathematics, or a related field. Familiarity with alternative cloud and data platforms such as Databricks, Snowflake, Azure, or AWS. Knowledge of DevOps/DataOps methodologies and experience with CI/CD pipelines. Understanding of monitoring, logging, and troubleshooting in cloud-based environments. Google Cloud Professional More ❯

and brownfield projects. Comfortable working independently in a remote setup. Strong communication and stakeholder engagement skills. Nice to Have: Experience with other Azure services such as Azure Data Lake, Databricks, or Power BI. Familiarity with DevOps/DataOps processes and tools. Microsoft Certified: Fabric Analytics Engineer Associate Start Date: ASAP or July start acceptable Apply now to join a forward More ❯

drive technical innovation across client projects Document processes and contribute to internal knowledge repositories and best practice libraries Key Skills & Experience Strong hands-on experience with Azure tooling, including: Databricks, Data Factory, Data Lake, and Synapse (or similar data warehouse tools) Azure Analysis Services or comparable BI tooling Solid programming capability in: SQL, Python, Spark, and ideally DAX Familiarity with More ❯

aimed at maximizing value through data utilization. Knowled g e/Experience Expertise in Commercial/Procurement Analytics. Experience in SAP (S/4 Hana). Experience with Spark, Databricks, or similar data processing tools. Stron g technical proficiency in data modelin g , SQL, NoSQL databases, and data warehousing . Hands-on experience with data pipeline development, ETL processes, and More ❯

automate ETL pipelines using SQL, SSIS, and Azure Data Factory Support BI delivery, UAT testing, and project work across teams Contribute to our cloud shift using Azure Synapse and Databricks Maintain technical documentation and support data governance Tech Stack Platforms: SQL Server, Azure Synapse, Databricks ETL: SSIS (2014–2022), Azure Data Factory BI Tools: Power BI Other: Power Apps, Power More ❯

design sessions with stakeholders, ensuring solutions align with NHS requirements and best practices. Develop end-to-end data solutions leveraging Azure services, including Azure Synapse Analytics, Azure Data Factory, Databricks, and Azure SQL. Define data models, integration patterns, and governance frameworks to ensure efficient data management, interoperability, and compliance. Drive cloud migration strategies for NHS data systems, modernizing legacy environments … Skills & Experience: Proven experience in architecting Azure data solutions, particularly in NHS or healthcare settings. Expertise in Azure Data Services, such as: Azure Synapse Analytics Azure Data Factory Azure Databricks Azure SQL & Cosmos DB Azure Purview (for Data Governance) Azure Blob Storage & Data Lake Azure Event Hubs & Streaming Analytics Hands-on experience with data integration, ETL/ELT pipelines, and More ❯

a skilled and motivated Lead Data Engineer with Insurance experience to play a key role in the creation of a brand-new data platform within the Azure ecosystem and Databricks . This is an exciting opportunity to be at the forefront of data innovation, working within a newly formed Data & Analytics team in a long standing London Market Insurer . … a Lead Data Engineer, you'll work closely with both technical and business stakeholders, leveraging your expertise to design, develop, and optimize a high-performance data platform built on Databricks . This platform will be built to scale, incorporating the latest advancements in data intelligence while supporting strategic business objectives. Key Responsibilities 🔹 Build & Develop – Design and maintain a robust Databricks … solutions. 🔹 Futureproofing – Drive the evolution of the data platform, ensuring adaptability for new data sources, analytical models, and emerging technologies. What You’ll Bring ✅ Extensive hands-on experience with Databricks , and Microsoft Azure data tools (must-have: Azure Data Factory, Azure Synapse, or Azure SQL). ✅ Dimensional modelling expertise for analytics use cases. ✅ Strong ETL/ELT development skills. ✅ Python More ❯

Data Engineer – Investment Banking – London/Hybrid (Data Engineer, SQL Data Engineer, Java, Python, Spark, Scala, SQL, Snowflake, OO programming, Snowflake, Databricks, Data Fabric, design patterns, SOLID principles, ETL, Unit testing, NUnit, MSTest, Junit, Microservices Architecture, Continuous Integration, Azure DevOps, AWS, Jenkins, Agile, Data Engineer, SQL Data Engineer) We have several fantastic new roles including a Data Engineer position to … teams to join their new and growing IT team! They are looking for an experienced Lead Data Engineer with expert level Java/Python, as well as Snowflake/Databricks to join an exceptional core engineering team and deliver features across their Data Engineering platform. We are seeking a Data Engineer who has advanced working knowledge of Snowflake/Databricks More ❯

Exposure across all the SDLC process, including testing and deployment. Expertise in Microsoft Azure is mandatory including components like Azure Data Factory, Azure Data Lake Storage, Azure SQL, Azure DataBricks, HD Insights, ML Service etc. Good knowledge of Python and Spark are required. Experience in ETL & ELT Good understanding of one scripting language Good understanding of how to enable analytics … a diverse and challenging set of customers to success. Good understanding of the CPG (Consumer Packaged Goods) domain is preferred. Skills: Data Ops, ML Ops, Deep expertise in Azure Databricks, ETL frameworks. Expertise in Microsoft Azure is mandatory including components like Azure Data Factory, Azure Data Lake Storage, Azure SQL, Azure DataBricks, HD Insights, ML Service etc. Fractal provides equal More ❯

Exposure across all the SDLC process, including testing and deployment. Expertise in Microsoft Azure is mandatory including components like Azure Data Factory, Azure Data Lake Storage, Azure SQL, Azure DataBricks, HD Insights, ML Service etc. Good knowledge of Python and Spark are required. Experience in ETL & ELT Good understanding of one scripting language Good understanding of how to enable analytics … a diverse and challenging set of customers to success. Good understanding of the CPG (Consumer Packaged Goods) domain is preferred. Skills: Data Ops, ML Ops, Deep expertise in Azure Databricks , ETL frameworks. Expertise in Microsoft Azure is mandatory including components like Azure Data Factory, Azure Data Lake Storage, Azure SQL, Azure DataBricks, HD Insights, ML Service etc. Fractal provides equal More ❯

to enact step-change operational efficiency and maximize business value by confidently utilizing trustworthy data. What are we looking for? Great experience as a Data Engineer Experience with Spark, Databricks, or similar data processing tools. Proficiency in working with the cloud environment and various software’s including SQL Server, Hadoop, and NoSQL databases. Proficiency in Python (or similar), SQL and … Spark. Proven ability to develop data pipelines (ETL/ELT). Strong inclination to learn and adapt to new technologies and languages. Strong understanding and experience in working with Databricks Delta Lake. Proficiency in Microsoft Azure cloud technologies Strong inclination to learn and adapt to new technologies and languages. What will be your key responsibilities? Collaborate in hands-on development … relevant technologies to create and maintain data assets and reports for business insights. Assist in engineering and managing data models and pipelines within a cloud environment, utilizing technologies like Databricks, Spark, Delta Lake, and SQL. Contribute to the maintenance and enhancement of our progressive tech stack, which includes Python, PySpark, Logic Apps, Azure Functions, ADLS, Django, and ReactJs. Support the More ❯

hybrid role based in the London area. As Data Analyst , you will be responsible for delivering data-driven insights across a wide range of datasets. Using tools such as Databricks, Tableau, Looker Studio, Amplitude, and DBT, you will extract, transform, and analyse data to support key business functions. You will collaborate closely with stakeholders to understand their data needs, generate … with forecasting, target setting, and promotional analysis. Data Governance & Optimisation Conduct QA checks for data accuracy. Improve data architecture and integrate new data sources. Apply advanced analytics using SQL, Databricks, and other tools. What are we looking for? 2+ years’ experience in a similar role. Strong analytical mindset with a passion for uncovering insights. Skilled in SQL and data visualisation More ❯

vision and architectural direction across entire programmes and client estates. Key Responsibilities Senior Data Architects: Lead the design and delivery of cloud-native data solutions using modern platforms (e.g. Databricks, Snowflake, Kafka, Confluent) Architect data lakes, lakehouses, streaming pipelines, and event-driven architectures Oversee engineering teams and collaborate with analysts and QA functions Translate complex requirements into scalable, robust data … Core Experience: Proven track record in data architecture , either from a delivery or enterprise strategy perspective Deep experience with cloud platforms (Azure, AWS, GCP) and modern data ecosystems (Spark, Databricks, Kafka, Snowflake) Strong understanding of Data Mesh , Data Fabric , and data product-led approaches Data modelling expertise (relational, dimensional) and familiarity with tools like Erwin , Sparx , Archi Experience with ETL More ❯

understanding of data modelling, warehousing principles, and analytics best practices Experience working cross-functionally with strong communication skills Nice to have: Python, AWS/GCP/Azure, Medallion Architecture, Databricks, and knowledge of data governance or experience in high-growth environments This is a fantastic opportunity to join a purpose-led business in a high-impact role, helping to shape More ❯

or internal clients within large organisations, through e.g. the RFI/RFP process, as preferred bidder, documented bids and face to face presentations. Experience of data science platforms (e.g. Databricks, Dataiku, AzureML, SageMaker) and machine learning frameworks (e.g. Keras, Tensorflow, PyTorch, scikit-learn) Cloud platforms – demonstrable experience of building and deploying solutions to Cloud (e.g. AWS, Azure, Google Cloud) including More ❯

as PySpark. Extensive experience with PyTorch (preferred) and/or TensorFlow. Hands-on experience with deploying machine learning models in production using cloud platforms, especially Microsoft Azure ML/Databricks ML Flow. Experience in integrating CI/CD pipelines for ML models. A proactive, hands-on approach to building, iterating, and owning solutions end-to-end. Strong interest in machine More ❯