101 to 125 of 145 Data Lake Jobs

Azure Data Engineer (Databricks) - Must be SC Cleared (Outside IR35) Contract: 6 months Location: Newcastle Upon Tyne (2-3 days onsite per week) Expenses: Not reimbursable Domain: Public Sector Clearance: SC Clearance is mandatory Role Overview: We are seeking an experienced Azure Data Engineer with strong … experience as a Data Engineer in Azure environments. Strong hands-on expertise with Databricks and Spark. Knowledge of Azure Data Lake, Azure Synapse, and related services. Experience in Public Sector projects. SC Clearance (must be active). Must be SC Cleared Candidates ideally based ...

Data Engineer (Python) - Leading Asset Manager | London (Hybrid) Brand-new team. High autonomy. Big impact and high bonus! A top Asset Manager is building a new data function from scratch , and they need a sharp Data Engineer to help define and deliver their future … office, 2 days WFH (flexible when needed). The Role Build and optimise scalable data pipelines for a new Data Lake Work with Python, SQL, Spark, AWS, Docker, CI/CD Orchestrate workflows using Airflow, Prefect, or Dagster Support ETL, API integrations, and high-quality ...

Senior Data Engineering Consultant - £60,000 - Hybrid Key Responsibilities Lead, mentor, and develop a team of Technical Consultants. Manage resource planning, scheduling, and overall delivery workflows. Collaborate with Pre-sales, Commercial, and Project Management teams to scope … deliver projects. Contribute to technical delivery, designing scalable data solutions in Azure/Microsoft environments. Support cloud migrations, data lake builds, and ETL/ELT pipeline development. Ensure delivery follows best practices and internal standards. Skills & Experience Strong leadership and relationship-building skills. Experience guiding ...

Senior Data Engineering Consultant - £60,000 - Hybrid Key Responsibilities Lead, mentor, and develop a team of Technical Consultants. Manage resource planning, scheduling, and overall delivery workflows. Collaborate with Pre-sales, Commercial, and Project Management teams to scope … deliver projects. Contribute to technical delivery, designing scalable data solutions in Azure/Microsoft environments. Support cloud migrations, data lake builds, and ETL/ELT pipeline development. Ensure delivery follows best practices and internal standards. Skills & Experience Strong leadership and relationship-building skills. Experience guiding ...

reinsurer operating globally, with a focus on complex commercial and industrial risks. The business combines deep sector expertise with advanced use of data, analytics, and modern technology to deliver informed, efficient, and fair underwriting decisions. The company is in a period of significant growth and is committed … C# ecosystem and modern front-end frameworks (ideally Angular). Experience with data architecture concepts including relational, NoSQL, and data lake technologies. Knowledge of cloud-based AI architectures, including LLMs, Generative AI, and Agentic AI. Experience integrating proprietary systems with COTS finance or ERP platforms ...

reinsurer operating globally, with a focus on complex commercial and industrial risks. The business combines deep sector expertise with advanced use of data, analytics, and modern technology to deliver informed, efficient, and fair underwriting decisions.The company is in a period of significant growth and is committed to maintaining … C# ecosystem and modern front-end frameworks (ideally Angular). Experience with data architecture concepts including relational, NoSQL, and data lake technologies. Knowledge of cloud-based AI architectures, including LLMs, Generative AI, and Agentic AI. Experience integrating proprietary systems with COTS finance or ERP platforms ...

mining, and visualization to extract insights. Design and implement Big Data solutions using Hadoop, Spark, PySpark, and Data Lake architectures. Develop and deploy ML models (supervised, unsupervised, tree-based, ensemble methods). Implement deep learning architectures (CNN, RNN, LSTM) using TensorFlow, PyTorch … Analytical Tools: EDA, Data Mining, Visualization (Plotly, Matplotlib, Seaborn) Statistical & Multivariate Analysis Big Data: Hadoop, MapReduce, HDFS, DataBricks, Spark, PySpark DataLake Architecture, AWS Redshift, Kinesis, EMR Machine Learning Supervised Models: Nave Bayes, Logistic Regression, SVM, Linear Regression, KNN, Tree-Based Models: Decision Trees, Random Forest, Gradient ...

team champions continuous learning, clean engineering practices, and building scalable, impactful applications that shape the firm's future. If you're passionate about data, eager to solve complex problems, and want to see your insights drive real organisational change, this is the role for you. About the Role … Strong understanding of data modelling, star schema design, and ETL processes Experience with Microsoft Fabric, Synapse, Azure SQL or Data Lake Familiarity with Azure DevOps/GIT for source control Strong analytical mindset with excellent communication and stakeholder engagement skills Experience with systems such ...

Tittle: Data Scientist/Data Engineer Rate: £450-£550 Outside IR35 Contract duration: 3 months (Initially) Start date: January start date! I'm working with a cutting-edge tech business building a next-generation data platform, and they're looking for a Data Scientist/Data Engineer to join at a critical early stage. This is a hands-on role working closely with the CTO, owning the design and delivery of end-to-end data pipelines and advanced AI-driven capabilities. You'll be responsible for ingesting ...

have an exciting opportunity for a Senior DataOps Engineer to join the Group and take responsibility for inspiring and coaching our Data Engineering teams. The person in post will be working alongside domain-oriented, multi-disciplinary data product teams to design, develop, and test high-quality … data solutions that serve as the backbone of our decision-making and digital services. Driving automation and CI/CD practices across data pipelines and infrastructure, the Senior DataOps Engineer will build robust, scalable, and secure infrastructure that supports our data platform strategy. ...

role We have an exciting opportunity for a Senior DataOps Engineer to join the Group and take responsibility for inspiring and coaching our Data Engineering teams. The person in post will be working alongside domain-oriented, multi-disciplinary data product teams to design, develop, and test … high-quality data solutions that serve as the backbone of our decision-making and digital services. Driving automation and CI/CD practices across data pipelines and infrastructure, the Senior DataOps Engineer will build robust, scalable, and secure infrastructure that supports our data platform ...

with Microsoft technologies like Azure, .NET, Power Platform, and Microsoft 365. They are currently looking for a Power BI Developer that will turn data into actionable insights. You'll design and maintain BI solutions, collaborate across teams, and deliver high-quality dashboards and reports. Key Responsibilities Develop data … Proven Power BI expertise (Desktop & Service), DAX, Power Query. Strong data modelling and ETL knowledge. Experience with Azure SQL, Data Lake, Synapse, or Microsoft Fabric. Understanding of GDPR and data security standards. Excellent communication and stakeholder engagement. Guidant, Carbon60, Lorien & SRG - The Impellam ...

help to build a better working world. General information Location: Dublin Available for Work Visa Sponsorship: No Business area: Technology Consulting (AI and Data) Contract type: Permanent The team and the opportunity Are you ready to shape how organisations use data to make smarter decisions … expectations, navigate complex requirements, and deliver tailored solutions across diverse industries. 7 years' experience working with Databricks. Good hands on experience with Spark, Delta Lake, and Unity Catalog Strong understanding of cloud platforms like Azure, AWS and/or Google Cloud Experience designing data lakes, lakehouses ...

Data Platform Engineer - ADF, Terraform, Synapse, Spark £Market Rate (Inside IR35) London/Hybrid 6 months We are currently working with a client who urgently require a Data Platform Engineer with hands on expertise in Synapse Spark Architecture, ADF and Terraform (IaC), to assist in designing … Factory (ADF), including network configurations and integration runtime options. Strong expertise in Synapse Analytics Spark, Azure Key Vault, and Data Lake Storage Accounts. Experience in efficient handling of large-scale data Deep understanding of Azure DevOps pipelines and deploying Terraform (Infrastructure as Code ...

Data Platform Engineer - ADF, Terraform, Synapse, Spark £Market Rate (Inside IR35) London/Hybrid 6 months We are currently working with a client who urgently require a Data Platform Engineer with hands on expertise in Synapse Spark Architecture, ADF and Terraform (IaC), to assist in designing … Factory (ADF), including network configurations and integration runtime options. Strong expertise in Synapse Analytics Spark, Azure Key Vault, and Data Lake Storage Accounts. Experience in efficient handling of large-scale data Deep understanding of Azure DevOps pipelines and deploying Terraform (Infrastructure as Code ...

cross-functional stakeholders. Technical Skills: Cloud Architecture: Deep expertise in Microsoft Azure, including: Azure Functions, Azure AI Search, API Management, Azure Data Lake, Azure Service bus, (link removed) NET Frameworks Azure AD/Entra ID, Azure App-Insights, Azure DevOps and CI/CD pipelines Databricks: Hands … experience with: Data engineering workflows Delta Lake and Spark optimization Integration with Azure services Unity Catalog Data Serving strategies Frontend Development: Proficiency in React, Angular, JavaScript and related frameworks Experience with enterprise-grade Front end applications. Integration of frontend apps with Azure-hosted APIs Responsibilities ...

Developer to their in-house development team. In this role, you will be responsible for designing, developing, and maintaining BI solutions that transform data into actionable insights. As a Power BI Developer, you will be expected to have strong experience in data modelling, DAX, and visualization … Experience: Experience delivering enterprise-grade Power BI SQL Databases Data Visualisation, Power Query, DAX Azure SQL, Microsoft Fabric, Synapse, Data Lake would be desirable Understanding of ETL processes and Data Modelling Excellent analytical skills GIT/Azure DevOps Relevant professional certifications would ...

Data Engineer (Hybrid) Bristol – Spark, S3, Redshift, Lambda, EMR, Glue Avanti Recruitment is working with a rapidly growing technology business in Bristol to recruit a talented AWS Data Engineer to join their team. You will help build and evolve their AWS cloud-based data infrastructure. You’ll work with large datasets and a modern AWS stack, designing scalable pipelines and delivering reliable data to teams across the organisation. In this role, you will develop and maintain ETL and ELT workflows using Apache Spark, Python and AWS Glue, while architecting solutions ...

RoleWe are looking for a Python Data Engineer with strong hands-on experience in Behave-based unit testing, PySpark development, Delta Lake optimisation, and Azure cloud services. This role focusses on designing and deploying scalable data processing solutions in a containerised environment, emphasising maintainable, configurable … pipelines using Python and PySpark. Implement unit and behavior-driven testing with Behave, ensuring robust mocking and patching of dependencies. Design and maintain Delta Lake tables for optimised query performance, ACID compliance, and incremental data loads. Build and manage containerised environments using Docker for consistent development, testing ...

RoleWe are looking for a Python Data Engineer with strong hands-on experience in Behave-based unit testing, PySpark development, Delta Lake optimisation, and Azure cloud services. This role focusses on designing and deploying scalable data processing solutions in a containerised environment, emphasising maintainable, configurable … pipelines using Python and PySpark. Implement unit and behavior-driven testing with Behave, ensuring robust mocking and patching of dependencies. Design and maintain Delta Lake tables for optimised query performance, ACID compliance, and incremental data loads. Build and manage containerised environments using Docker for consistent development, testing ...

technologies (HTML, CSS, React JS), REST-APIs, and Backend using Java/J2EE technologies. Big Data/Data Lake system architecture is a plus Experience with Agile software development processes with a strong emphasis on test driven development Strong knowledge of CI/CD pipelines ...

bonus, private healthcare and more! A large and growing retail business are looking to expand their IT function with the addition of a Data Integration Manager. This role will focus on end to end data engineering and integration solutions for a company wide modernisation programme. Duties … will include: - Managing the processes for data engineering and data integration in order to align to business needs - Leading a team across engineering and integration - Work to ensure the data infrastructure of the business effectively supports analytics, AL, machine learning ML and reporting - Designing ...

bonus, private healthcare and more! A large and growing retail business are looking to expand their IT function with the addition of a Data Integration Manager. This role will focus on end to end data engineering and integration solutions for a company wide modernisation programme. Duties … will include: - Managing the processes for data engineering and data integration in order to align to business needs - Leading a team across engineering and integration - Work to ensure the data infrastructure of the business effectively supports analytics, AL, machine learning ML and reporting - Designing ...

bonus, private healthcare and more! A large and growing retail business are looking to expand their IT function with the addition of a Data Integration Manager. This role will focus on end to end data engineering and integration solutions for a company wide modernisation programme. Duties … will include: - Managing the processes for data engineering and data integration in order to align to business needs - Leading a team across engineering and integration - Work to ensure the data infrastructure of the business effectively supports analytics, AL, machine learning ML and reporting - Designing ...

bonus, private healthcare and more! A large and growing retail business are looking to expand their IT function with the addition of a Data Integration Manager. This role will focus on end to end data engineering and integration solutions for a company wide modernisation programme. Duties … will include: - Managing the processes for data engineering and data integration in order to align to business needs - Leading a team across engineering and integration - Work to ensure the data infrastructure of the business effectively supports analytics, AL, machine learning ML and reporting - Designing ...