1 to 25 of 179 Databricks Jobs in London

problem-solving, and communication skills Nice to Have Experience with DevOps and CI/CD pipelines. Familiarity with cloud-native data platforms such as Databricks or Microsoft Fabric. Understanding of data privacy regulations (e.g. GDPR) and secure data handling practices. If you’re looking for a Data Engineering position that ...

solving skills, with an ability to work both independently and collaboratively within multi-disciplinary teams Desirable Skills for the AWS Data Engineer: Experience with Databricks , Kafka , or Kinesis for real-time data streaming Knowledge of containerisation (Docker, ECS) and modern orchestration tools such as Airflow Familiarity with machine learning model ...

. Understanding of data modeling concepts and ETL/ELT processes. Ability to troubleshoot and optimize data workflows. Required Knowledge and Skills: Exposure to Databricks or Spark-based environments. Experience with data governance tools and practices. Experience with JavaScript Full-Stack development environments including MongoDB or similar NoSQL databases Knowledge ...

lead technical discussions and provide guidance on data engineering best practices. Ability to troubleshoot and optimize data workflows. Required Knowledge and Skills: Experience with Databricks or Spark-based environments, including optimization strategies. Familiarity with data governance tools and frameworks (e.g., Unity Catalog). Experience with JavaScript Full-Stack development environments ...

automation tasks. Proven ability to design, build, and maintain scalable data pipelines for batch and streaming data using modern frameworks. Experience working with Databricks for big data processing, Spark-based transformations, and collaborative analytics workflows. Familiarity with Azure Data Factory (ADF) for data orchestration and integration across cloud environments. Skilled ...

best practices - Lead cloud migration and modernization initiatives Cloud & Technology Leadership: - AWS: EC2, S3, RDS, Lambda, EKS, Elastic Beanstalk, IAM, Neptune - Azure: Data Factory, Databricks, Synapse, Data Lake, Event Hub, AKS, Power BI - Microservices and APIs using Java/Spring Boot - CI/CD using GitLab, Azure DevOps, Kubernetes, Helm ...

Data technologies (Spark, Hadoop, Kafka). - Strong understanding of API-based integrations. Preferred Skills: - NoSQL experience (MongoDB, DynamoDB). - Snowflake/Redshift/Databricks exposure. - CI/CD for data platforms (GitLab, Terraform, CloudFormation). - Data governance and security frameworks. ...

have most, if not all the following: Experience as a Senior Data Engineer or similar Strong knowledge of Azure Cloud architecture and Azure Databricks, DevOps and CI/CD. Experience with PySpark, Python, SQL and other data engineering development tools. Experience with metadata driven pipelines and SQL serverless data warehouses. ...

related frameworks (e.g., LangChain, TensorFlow, etc.) Possess knowledge of cloud-based AI/ML services like AWS SageMaker, Azure Machine Learning, Google AI Platform, Databricks GenAI/BI, Snowflake GenAI, and their relevant services for building and deploying GenAI workloads. Knowledge of market insight, technology trends, and competitor intelligence Experience ...

experience in data engineering roles (midsenior level) Hands-on expertise with Azure data services such as Data Factory, Synapse and Azure SQL Experience using Databricks and Python in data processing workflows Strong SQL skills and practical knowledge of dimensional modelling and warehouse design Familiarity with CI/CD approaches ...

Collaborate with stakeholders and technical teams to translate business needs into data solutions Azure Data Engineer requirements: Hands-on experience with Azure Data Factory, Databricks, and SQL-based databases Strong understanding of data engineering principles, including ELT and data modelling Experience working with CI/CD pipelines, automation, and testing ...

Code and Kubernetes for container orchestration, ensuring efficient deployments and robust system performance. Working within Azure, you'll integrate advanced analytics platforms like Databricks and build data pipelines using Python and PySpark to support large-scale processing. The position also involves optimising data models, delivering insights through Power ...

continuously improve your skills and knowledge. Profile The Data Engineer will have mastery of data management and processing tools, including Power BI, Data Factory, Databricks, SQL Server, and Oracle. Proficient in SQL and experienced in database administration. Familiarity with cloud platforms such as Azure and AWS. Excellent problem-solving ...

this page. For further details or to enquire about other roles, please contact Nick Mandella at Harnham. KEYWORDS Python, SQL, AWS, GCP, Azure, Cloud, Databricks, Docker, Kubernetes, CI/CD, Terraform, Pyspark, Spark, Kafka, machine learning, statistics, Data Science, Data Scientist, Big Data, Artificial Intelligence, private equity, finance. ...

Experience Required Strong experience in a senior or lead data engineering role Deep hands-on experience with Azure data services (e.g. Data Factory, Synapse, Databricks, Azure SQL, ADLS) Proven use of Terraform to provision and manage cloud infrastructure Solid understanding of data modelling, ETL/ELT patterns, and large-scale ...

visualisation Leading client workshops and translating business needs into technical solutions Designing and implementing scalable ETL/ELT pipelines using Azure tools (Fabric, Databricks, Synapse, Data Factory) Building data lakes with medallion architecture Migrating legacy on-prem data systems to the cloud Creating impactful dashboards and reports using Power ...

visualisation Leading client workshops and translating business needs into technical solutions Designing and implementing scalable ETL/ELT pipelines using Azure tools (Fabric, Databricks, Synapse, Data Factory) Building data lakes with medallion architecture Migrating legacy on-prem data systems to the cloud Creating impactful dashboards and reports using Power ...

across both technical and non‐technical stakeholders. Work collaboratively with engineers, architects, and transformation teams. Operate confidently within enterprise‐scale data platforms. Exposure to Databricks or Snowflake is advantageous. Experience in Life & Pensions insurance (or broader insurance domains) is highly preferred. Ideal Candidate Profile Strong experience in analytics with advanced ...

JavaScript Machine Learning: TensorFlow, PyTorch, Scikit-learn GenAI/LLMs: OpenAI API, Hugging Face Transformers, LangChain, Langraph, RAG pipelines Data & Analytics: Pandas, NumPy, Spark, Databricks, Hadoop, Vector Databases, Snowflake Model Deployment: MLflow, FastAPI, Flask, Docker, Kubernetes Cloud & Infrastructure: AWS, Azure, GCP CI/CD pipelines (GitLab, Jenkins) Desirable Skills Experience ...

would be beneficial but is by no means essential. Interested candidates must have experience in a similar role with MS Azure Data Platforms, Synapse, Databricks or other Cloud platforms such as AWS, GCP, Snowfake etc. Key Responsibilities will include; Design, implement, and optimize end-to-end solutions using Fabric components ...

would be beneficial but is by no means essential. Interested candidates must have experience in a similar role with MS Azure Data Platforms, Synapse, Databricks or other Cloud platforms such as AWS, GCP, Snowfake etc. Key Responsibilities will include; Design, implement, and optimize end-to-end solutions using Fabric components ...

data structures, data quality challenges, and how to design reliable transformation logic ·Experience working with modern data platforms such as Azure, GCP, AWS, Databricks, Snowflake, or similar ·Confidence working with imperfect data and making it fit for consumption downstream ·Experience supporting or mentoring other engineers through code reviews, pairing ...

data engineering skills with technical leadership, mentoring a small team while remaining deeply involved in delivery. Key Skills & Experience: Microsoft Azure data platforms Azure Databricks , SQL & data modelling Power BI for analytics and reporting Terraform/Infrastructure as Code CI/CD with GitHub Actions Experience leading or mentoring data ...

Databricks Engineer - £400PD - Remote About the Role We are seeking a highly skilled Azure Data Engineer with deep, hands-on Databricks experience to join our growing data engineering team. In this role, you will design, build, and optimise scalable data pipelines and lakehouse architectures on Azure, enabling advanced analytics … data-driven decision making across the business. Key Responsibilities Design, develop, and maintain ETL/ELT pipelines using Azure Databricks, PySpark, and Delta Lake. Build and optimise data lakehouse architectures on Azure Data Lake Storage ( ADLS ) . Develop high-performance data solutions using Azure Synapse, Azure Data Factory, and Databricks ...

Experience with developing data features and associated transformation procedures on a modern data platform. Examples include (but not limited to) Azure Fabric, AWS Lakeformation, Databricks or Snowflake. Experience automating operations tasks with one or more scripting languages. Due to the nature of the project and the short turnaround required ...