1 to 25 of 75 Databricks Jobs in the City of London

Solution Architect - Databricks Initial 7-month Contract Role Remote Working £600 - £700, Inside IR35 We're partnering with a Global IT Partner who are looking for a Databricks Solution Architect to provide guidance on data architecture and design and optimise Databricks solutions. As a Databricks Solution Architect, you will be responsible for: Lead end-to-end architecture of data lakes … to Azure with a core focus on Databricks Provide technical leadership on Databricks Design and implement data pipelines in Azure Databricks, optimizing for performance, cost, and scalability Advise of data lake migration, data modelling and pipeline optimisation Refactor and modernize existing data models to integrate with Power BI Coordinate phased migrations to ensure zero disruption to live operations Replace legacy … tools by consolidating workflows within Databricks and native Azure services Requirements: 10+ years in cloud architecture/engineering with a strong focus on building scalable data pipelines Expertise in Azure Databricks (7+years) including building and managing ETL pipelines using PySpark or Scala (essential) Solid understanding of Apache Spark, Delta Lake, and distributed data processing concepts Hands-on experience with Azure More ❯

Key Responsibilities * End-to-end development of AI/ML solutions. * MLOps practices: CI/CD, model monitoring, retraining. * Use of open-source and enterprise tools (LangChain, Azure OpenAI, Databricks). * Generative AI features: embeddings, RAG, AI agents. * Clean, testable code with modern engineering practices. * Align with enterprise architecture and governance. * Collaborate with architects and stakeholders. * Lifecycle management of models. More ❯

data modelling and data analytics [programming languages: Python, SQL, Alteryx] - Data Visualisation experience [tools: PowerBI, Tableau or similar] - MetaData management and data cataloguing [tools: Collibra, Alation] - Data Platform tools [DataBricks, Azure, Kafka] Other skills required: - Strong stakeholder management skills - Experience working in a trading environment - An interest in automation and modern technologies [use of automation, ML and AI to drive More ❯

data requirements. Attention to detail and a strong commitment to data integrity. Excellent problem-solving skills and ability to work in a fast-paced environment. Preferred Qualifications Experience with Databricks or Ray. Familiarity with dbt and Airflow. Experience with data quality frameworks. Understanding of ML requirements and experience working with ML teams. Experience in robotics or a related field. Familiarity More ❯

data requirements. Attention to detail and a strong commitment to data integrity. Excellent problem-solving skills and ability to work in a fast-paced environment. Preferred Qualifications Experience with Databricks or Ray. Familiarity with dbt and Airflow. Experience with data quality frameworks. Understanding of ML requirements and experience working with ML teams. Experience in robotics or a related field. Familiarity More ❯

development experience with: C# (.NET/.NET Core) REST APIs, SQL Server (T-SQL), HTML/CSS/JavaScript/React, Python Comfortable with tools like Azure Data Factory , Databricks , Power Automate , Power Platform , Copilot Studio . Strong knowledge of cloud security , automation , and DevOps principles. ?? Bonus Points for: Experience in the insurance market Familiarity with data architecture and data More ❯

and help drive a modern, data-driven culture all within a collaborative environment that values innovation and ownership. Tech you’ll be working with: Azure Data Lake Azure Synapse Databricks Data Factory Python, SQL, PySpark Terraform, GitHub Actions, CI/CD pipelines You’ll thrive here if you: Have strong experience building and leading Azure-based data platforms Enjoy mentoring More ❯

and help drive a modern, data-driven culture all within a collaborative environment that values innovation and ownership. Tech you’ll be working with: Azure Data Lake Azure Synapse Databricks Data Factory Python, SQL, PySpark Terraform, GitHub Actions, CI/CD pipelines You’ll thrive here if you: Have strong experience building and leading Azure-based data platforms Enjoy mentoring More ❯

Agile delivery Knowledge of compliance, regulatory reporting, and enterprise operational processes Proven experience delivering complex software solutions Strong leadership and stakeholder management skills Bonus: Python, Azure Data Factory/Databricks, data analysis, and COTS platforms Package: £85,000 15% non-contributory pension Hybrid working (3 days per week) Performance-based bonus scheme Comprehensive benefits package including pension, healthcare, and wellbeing More ❯

model training and inference. Driving best practices in data governance, security, and performance optimisation. What We're Looking For Strong experience with Azure cloud services, especially Data Factory, Synapse, Databricks, and Azure Functions. Proficiency in Python, SQL, and data transformation techniques. Experience working with both structured and unstructured data sources. Understanding of AI/ML data requirements and how to More ❯

popular Python libraries for data wrangling, such as Pandas, NumPy, and SQLAlchemy. Experience working with traditional SQL databases (e.g. PostgreSQL, MySQL, SQL Server) and cloud data warehouses (e.g. Snowflake, Databricks, BigQuery, Redshift). Experience with time-series data, and/or implementing data pipelines from major financial market data vendors (Bloomberg, Refinitiv, Factset....) SDLC and DevOps: Git, Docker, Jenkins/ More ❯

popular Python libraries for data wrangling, such as Pandas, NumPy, and SQLAlchemy. Experience working with traditional SQL databases (e.g. PostgreSQL, MySQL, SQL Server) and cloud data warehouses (e.g. Snowflake, Databricks, BigQuery, Redshift). Experience with time-series data, and/or implementing data pipelines from major financial market data vendors (Bloomberg, Refinitiv, Factset....) SDLC and DevOps: Git, Docker, Jenkins/ More ❯

Hands-on experience implementing end-to-end ML pipelines (data ingestion, training, validation, serving). Familiarity with ML workflow orchestration tools (Airflow, Prefect, Kubeflow) and feature/data platforms (Databricks, Tecton, etc.). Strong experience with cloud platforms (AWS, GCP, or Azure), Docker, and Kubernetes. Solid coding practices, including Git, automated testing, and CI/CD. Proficiency with Linux environments. More ❯

DATA HEAVY Product Owners who have managed complex, Global products. Read on for more details Experience required: Technical proficiency:Familiarity with Azure services (e.g., Data Lake, Synapse, Fabric) and Databricks for data engineering, analytics, performance optimisation, and governance. Experience with implementing and optimising scalable cloud infrastructure is highly valued. Backlog management:Demonstrated expertise in maintaining and prioritizing product backlogs, writing More ❯

across data modelling, integration, governance, and transformation. Experience with AWS (S3, Glue, Redshift, Lambda, Kinesis) and/or Azure (ADF, Synapse, Fabric). Familiarity with modern data platforms (e.g. Databricks, Snowflake, or Lakehouse environments). Ability to operate confidently in highly secure, mission-focused settings. Why Join? Work on meaningful security projects with real-world impact. Join a culture built More ❯

Job Summary : We are seeking a highly skilled and experienced Senior Data Engineer to join our team and contribute to the development and maintenance of our cutting-edge Azure Databricks platform for economic data. This platform is critical for our Monetary Analysis, Forecasting, and Modelling activities. The Senior Data Engineer will be responsible for building and optimising data pipelines, implementing … Development & Optimisation : Design, develop, and maintain robust and scalable data pipelines for ingesting, transforming, and loading data from various sources (e.g., APIs, databases, financial data providers) into the Azure Databricks platform. Optimise data pipelines for performance, efficiency, and cost-effectiveness. Implement data quality checks and validation rules within data pipelines. Data Transformation & Processing: Implement complex data transformations using Spark (PySpark … or Scala) and other relevant technologies. Develop and maintain data processing logic for cleaning, enriching, and aggregating data. Ensure data consistency and accuracy throughout the data lifecycle. Azure Databricks Implementation: Work extensively with Azure Databricks Unity Catalog, including Delta Lake, Spark SQL, and other relevant services. Implement best practices for Databricks development and deployment. Optimise Databricks workloads for performance and More ❯

Job Summary : We are seeking a highly skilled and experienced Senior Data Engineer to join our team and contribute to the development and maintenance of our cutting-edge Azure Databricks platform for economic data. This platform is critical for our Monetary Analysis, Forecasting, and Modelling activities. The Senior Data Engineer will be responsible for building and optimising data pipelines, implementing … Development & Optimisation: Design, develop, and maintain robust and scalable data pipelines for ingesting, transforming, and loading data from various sources (e.g., APIs, databases, financial data providers) into the Azure Databricks platform. Optimise data pipelines for performance, efficiency, and cost-effectiveness. Implement data quality checks and validation rules within data pipelines. Data Transformation & Processing: Implement complex data transformations using Spark (PySpark … or Scala) and other relevant technologies. Develop and maintain data processing logic for cleaning, enriching, and aggregating data. Ensure data consistency and accuracy throughout the data lifecycle. Azure Databricks Implementation: Work extensively with Azure Databricks Unity Catalog, including Delta Lake, Spark SQL, and other relevant services. Implement best practices for Databricks development and deployment. Optimise Databricks workloads for performance and More ❯

leadership Hands-on background in model deployment Strong B2C experience (e.g. ecommerce, gaming, telco) MSc required; PhD optional Executive-level stakeholder management Exposure to GenAI and cloud ML stacks (Databricks, Azure, AWS) Pathway to Chief Data Officer Interested? Please apply below. More ❯

relationships. Key Skills & Experience: Proven experience as a Platform Technical Lead, Data Architect, or similar role, leading enterprise data platforms at scale. Strong technical expertise in Azure, AWS, Snowflake, Databricks, Data Lakes, and Kafka. Deep understanding of modern data platform architectures, including Lakehouse, Kappa, and Lambda. Experience designing and managing secure, scalable, and governed data platforms. Strong knowledge of Data More ❯

leadership Hands-on background in model deployment Strong B2C experience (e.g. ecommerce, gaming, telco) MSc required; PhD optional Executive-level stakeholder management Exposure to GenAI and cloud ML stacks (Databricks, Azure, AWS) Pathway to Chief Data Officer Interested? Please apply below. More ❯

or similar). Experience leading multidisciplinary data teams in consulting or enterprise environments. Excellent communication skills with technical and non-technical stakeholders. Desirable: Experience with other modern data platforms (Databricks, Azure, AWS, GCP). Experience with platform observability and CI/CD for data platforms. Hands-on experience with tools like dbt, Fivetran, Matillion, or Airflow. Experience supporting pre-sales More ❯

of analytics, data modelling, architecture, and schema design. Experience building sophisticated dashboards using leading BI tools. What will make you stand out Deep expertise with advanced data platforms (e.g. Databricks, Snowflake, Palantir). Familiarity with Python-based or low-code frameworks for building data-driven applications. Background in financial services or the ability to quickly understand industry-specific challenges. This More ❯

long-term client relationships is a must have. Relevant data or AI qualification (e.g. CDMC, DAMA, DCAM, ISACA ) Hands-on experience with data and AI platforms such as Snowflake, Databricks, Microsoft Purview, Collibra, Informatica, or AWS Experience with data analytics tools such as Alteryx, Tableau, Power BI, SQL , etc. Two stage interview process to start ASAP. More ❯

or similar). Experience leading multidisciplinary data teams in consulting or enterprise environments. Excellent communication skills with technical and non-technical stakeholders. Desirable: Experience with other modern data platforms (Databricks, Azure, AWS, GCP). Experience with platform observability and CI/CD for data platforms. Hands-on experience with tools like dbt, Fivetran, Matillion, or Airflow. Experience supporting pre-sales More ❯

new technologies Experience in startups or top-tier consultancies is a plus Nice to Have (not essential): Familiarity with dashboarding tools, Typescript, and API development Exposure to Airflow, DBT, Databricks Experience with ERP (e.g. SAP, Oracle) and CRM systems What's On Offer: Salary: £50,000-£75,000 + share options Hybrid working: 2-3 days per week in a More ❯