126 to 150 of 229 Data Lake Jobs in the UK excluding London

JOB SUMMARY In this role, you'll play a pivotal part in building and optimizing data pipelines that transform large, multi-modal datasets into high-quality training inputs for cutting-edge AI models for drug discovery. You'll help evolve our data pipeline and storage infrastructure to support faster, more reliable turnarounds for research and development … engineering best practices-including version control, testing, and documentation. This role is remote, with preference for candidates on the East Coast or UK. KEY RESPONSIBILITIES Design and improve data pipelines that process large, multi-modal datasets from a variety of internal and external sources into training datasets for AI models. Evolve our data storage layer to … ML workflows is a plus. Hands-on experience with multi-terabyte scale data processing. Familiarity with AWS; Kubernetes experience is a bonus. Knowledge of data lake technologies such as Parquet, Iceberg, AWS Glue etc. Strong Python software engineering skills. Pragmatic mindset - able to evaluate tradeoffs find solutions that empower ML researchers to move quickly. Background More ❯

in the office) We have an exciting opportunity, in the engineering team to Provide Technical project management services for the delivery of the Dealer Management System (DMS) and data aggregation and Value Chain Reporting (VCR) as a Contractor outside of IR35 What you’ll be doing: Lead the design, development, and deployment of scalable solutions on Microsoft Azure. … Data pipelines and ETL processes using Azure Data Factory, Data Flow, Databricks/Spark (using Python & associated frameworks), Azure storage (data lake, blob) etc. Microsoft Dynamics development, primarily in and integrating with ERP modules, utilising relevant tech stack (Power Platform, Dataverse, C# & .Net, JavaScript & TypeScript, HTML & CSS). Monitoring and alerting … feature delivery. Production monitoring and support to rectify any technical issues raised by automated monitoring or system users. Collaboration with other technical implementation teams to align on Enterprise Data Platform (EDP) technical design, development and data standards. Collaborate with cross-functional teams to ensure seamless integration of Azure solutions with existing systems. Ensure best practices in More ❯

opportunity for a proactive and detailed-oriented Power BI Developer. You will need to havestrong analytical capabilities, deep understanding of finance processes, and a passion for turning complex data into clear insights using Power BI. This role is responsible for working within Enterprise Integration and Reporting team and alongside the Finance team, and other business areas. The purpose … Operations. Responsibilities Develop, optimise, and maintain Power BI reports and dashboards that meet finance business needs. Work with finance stakeholders to understand reporting requirements and translate them into data models and visualisations. Connect to, transform, and model data from D365 Finance and Operations and other relevant systems. Ensure data accuracy, consistency, and integrity across … Strong grasp of financial principles, terminology, and common reporting requirements. Excellent communication skills with the ability to work across business and technical teams. Familiarity with Azure Data Lake, Synapse, or similar tools (desirable but not essential). Experience Needed to Fulfil the Role: Multiple years' experience delivering Power BI solutions with hands on experience working with D365 More ❯

a portfolio of accounts and being the dedicated point of contact . You will: Lead and mentor a team of 6+ Tech Leads overseeing product engineering squads delivering data-centric products Provide strategic guidance , coaching, and professional development opportunities for engineering managers to empower their teams and deliver results Establish and maintain a high-performance culture focused on … and continuous improvement Foster a culture of mentorship and leadership development, identifying and nurturing talents Drive the recruitment, retention, and development of top-tier engineering talent within the data engineering team Define and drive the overall data platform strategy with a focus on Azure Cloud to ensure that the organisation's data infrastructure is … the design, implementation, and ongoing optimization of the data platform, leveraging Azure technologies such as Azure Data Factory, Azure Synapse Analytics, Azure Data Lake, and Azure SQL Database Ensure the data platform supports data engineering, analytics, and data science initiatives across squads Ensure the data More ❯

Want to drive a top brand's Data team with 1m+ users? If you love building software in Python, implementing robust data pipelines & driving best practices, you may be interested in a Senior Engineer role I have with a highly successful Tech driven business, where you'd drive the Data function playing a key … holiday (raising to 32), L&D budget, Health insurance, excellent maternity/paternity/adoption leave. What you'll like: You'd scale & improve a top brand's Data function, optimising retention of millions of users & having a real life impact You'd have autonomy over Tech to pick the latest, best languages/frameworks for the job … several high profile household name What you'll bring: Strong Python & TDD skills Expertise in modern data architecture (dimensional modelling, data mesh, data lake) & best practices (agile, CI/CD, IaC, observability). Experience with Cloud and big data technologies (e.g. Spark/Databricks/Delta Lake/BigQuery). More ❯

Data Engineer I am looking for a Data Engineer to join an established retailer as they look to migrate to Microsoft Fabric. They are looking to integrate a number of their data systems into one fully-scalable solution and create a brand-new data warehouse that will enable data to … you would work collaboratively with the Engineering manager and other Engineers. You will also work collaboratively with the wider IT function to enable to organisation to leverage its data and orchestrate data-driven decisions. As part of this role, you will be responsible for. Help create a scalable data platform and integrate data … PySpark Data pipeline development experience utilising Azure Data Factory or Fabric Pipelines Experience working within an Azure environment such as Lakehouse Architecture, Data Lake, Delta Lake, Azure Synapse Strong SQL knowledge Strong communication skills This is a hybrid role based out of the organisations office in Bristol 3 times per week with More ❯

Data Engineer I am looking for a Data Engineer to join an established retailer as they look to migrate to Microsoft Fabric. They are looking to integrate a number of their data systems into one fully-scalable solution and create a brand-new data warehouse that will enable data to … you would work collaboratively with the Engineering manager and other Engineers. You will also work collaboratively with the wider IT function to enable to organisation to leverage its data and orchestrate data-driven decisions. As part of this role, you will be responsible for. Help create a scalable data platform and integrate data … PySpark Data pipeline development experience utilising Azure Data Factory or Fabric Pipelines Experience working within an Azure environment such as Lakehouse Architecture, Data Lake, Delta Lake, Azure Synapse Strong SQL knowledge Strong communication skills This is a hybrid role based out of the organisations office in Bristol 3 times per week with More ❯

Data Engineer I am looking for a Data Engineer to join an established retailer as they look to migrate to Microsoft Fabric. They are looking to integrate a number of their data systems into one fully-scalable solution and create a brand-new data warehouse that will enable data to … you would work collaboratively with the Engineering manager and other Engineers. You will also work collaboratively with the wider IT function to enable to organisation to leverage its data and orchestrate data-driven decisions. As part of this role, you will be responsible for. Help create a scalable data platform and integrate data … PySpark Data pipeline development experience utilising Azure Data Factory or Fabric Pipelines Experience working within an Azure environment such as Lakehouse Architecture, Data Lake, Delta Lake, Azure Synapse Strong SQL knowledge Strong communication skills This is a hybrid role based out of the organisations office in Bristol 3 times per week with More ❯

Data Engineer I am looking for a Data Engineer to join an established retailer as they look to migrate to Microsoft Fabric. They are looking to integrate a number of their data systems into one fully-scalable solution and create a brand-new data warehouse that will enable data to … you would work collaboratively with the Engineering manager and other Engineers. You will also work collaboratively with the wider IT function to enable to organisation to leverage its data and orchestrate data-driven decisions. As part of this role, you will be responsible for. Help create a scalable data platform and integrate data … PySpark Data pipeline development experience utilising Azure Data Factory or Fabric Pipelines Experience working within an Azure environment such as Lakehouse Architecture, Data Lake, Delta Lake, Azure Synapse Strong SQL knowledge Strong communication skills This is a hybrid role based out of the organisations office in Bristol 3 times per week with More ❯

Data Engineer I am working with a data driven Microsoft partnered consultancy who are looking for a Data Engineer to join their growing team. You will have the opportunity to work with some of the latest Microsoft technologies with a focus on projects on Databricks implementations. You will join a team at the centre … of a number of data-driven projects where you will be responsible for the design, development and creation of data solutions. You will work on the full end-to-end product lifecycle from platform design to insights creation. As part of this role, you will be responsible for some of the following areas Design, develop and … and delivering data solutions focused on Databricks Strong ETL experience with tools such as ADF or SSIS Experience working with Azure technologies - Synapse, Fabric, Data Lake Knowledge of data architecture principles and data modelling In this role you will be required to attend the office on an ad-hoc basis in More ❯

Join us as a Data and Software Engineer at Barclays where you'll spearhead the evolution of our digital landscape, driving innovation and excellence. You'll harness cutting-edge technology to revolutionise our digital offerings, ensuring unapparelled customer experiences. This role will leverage technologies such as Kafka and Java and others listed below, to improve our legacy technology … stack. You will play a key role in the input of the design and creation of these initial systems. To be successful as a Data and Software Engineer , you should have experience with: Kafka developing Java developing Databases (e.g. Snowflake, MongoDB) Other highly valued skills include: Cloud Platform (e.g. AWS) Flink/Kstreams Linux/Shell You may … technical skills This role will be based out of our Glasgow campus. Purpose of the role To build and maintain the systems that collect, store, process, and analyse data, such as data pipelines, data warehouses and data lakes to ensure that all data is accurate, accessible, and secure. Accountabilities Build More ❯

Nottingham Trent House (95002), United Kingdom, Nottingham, Nottinghamshire Software Engineering Manager - Data Operations About the Role It's an exciting time at Capital One UK. We're on a mission to create our next generation, cloud-based data ecosystem, to unlock the power of data, increase financial inclusion and deliver transformative experiences to our … customers. The Win with Data team owns the data ecosystem. It captures and distributes data … in real time, makes sure data is accurate & easy to use, enabling analysis for deep insight backed by the power of a Cloud based data Lake House, scalable machine learning, and powerful data processing, to provide the right data to enable effective Business decisions. We are seeking a Data More ❯

Integration: Collaborating with stakeholders to understand business requirements and designing integration solutions to connect Dynamics 365 Finance and Operations with other systems such as ProjOps, Dataverse, Data Lake, Synapse, Enterprise Data Warehouse (EDW), or third-party applications. Development: Writing code and developing custom solutions using relevant programming languages and technologies (such as X , C#, SQL … XML, JSON) to create seamless data flow and functionality between different systems. API Development: Utilizing APIs (Application Programming Interfaces) provided by Dynamics 365 Finance and Operations and other systems to facilitate data exchange and communication between them. Data Mapping and Transformation: Defining data mapping rules and implementing data transformation processes … Operations Skills of X development, including solutions. T-SQL development skills Cloud Integration specialisms familiar with the range of Azure integration technologies Functions Logic App Azure Data Lake Azure Synapse Azure DevOps Experience in developing solutions within Power platform (Power Apps, Power automate), Dataverse Experience of integration work with payroll/HR/Finance systems. Good understanding More ❯

onsite and can be in either the Edinburgh or Glasgow office. As part of the team, you’ll engage with a modern tech ecosystem, solving complex software and data problems that contribute directly to business goals. You'll play a key role in crafting reliable, scalable solutions within … an agile, DevOps-oriented setup, working with cutting-edge Microsoft and Azure technologies. Initially, you’ll work closely with a Technical Architect to build an advanced data lake, setting the stage for innovative AI-driven tools aimed at boosting team productivity. Main responsibilities will include: Partnering with AI specialists to help design and implement bespoke copilot tools. … high-quality engineering practices. Playing a role in pushing forward innovation and enhancing the sophistication of our frameworks, system integrations, and automated processes. Working closely with the internal data engineering community to stay current with new Azure data technologies, and helping to share best practices and strategic guidance across the company. Things that will align you More ❯

onsite and can be in either the Edinburgh or Glasgow office. As part of the team, you’ll engage with a modern tech ecosystem, solving complex software and data problems that contribute directly to business goals. You'll play a key role in crafting reliable, scalable solutions within … an agile, DevOps-oriented setup, working with cutting-edge Microsoft and Azure technologies. Initially, you’ll work closely with a Technical Architect to build an advanced data lake, setting the stage for innovative AI-driven tools aimed at boosting team productivity. Main responsibilities will include: Partnering with AI specialists to help design and implement bespoke copilot tools. … high-quality engineering practices. Playing a role in pushing forward innovation and enhancing the sophistication of our frameworks, system integrations, and automated processes. Working closely with the internal data engineering community to stay current with new Azure data technologies, and helping to share best practices and strategic guidance across the company. Things that will align you More ❯

proven track record of delivering technical solutions in Agile environments with expertise in JIRA and Confluence. Proven Solution Architect Experience: Deep technical expertise in various solution architectures, especially data platforms and complex data pipelines on public cloud platforms, with a focus on GCP. Robust Application Design Skills: Experience in designing scalable, low-latency solutions that meet … diverse functional and non-functional requirements. Data Architecture Expertise: Proven experience in designing and delivering complex batch and streaming data pipelines, understanding orchestration, and database capabilities across multiple solutions. Problem-Solving Ability: A strategic and creative thinker who can tackle complex challenges and develop innovative, robust solutions. Collaboration & Communication Skills: Strong interpersonal skills that can influence … Architect certifications. Cyber & Data Security Understanding: An appreciation for the architectural implications of application design and implementation. Modern Data Capabilities: Knowledge of Data Lake, Data Warehouse, and Machine Learning architectures. QA Best Practices: A solid understanding of data quality concepts and practices. Why Join Us? This is your chance More ❯

for musicians and artists to use to promote their music. At un:hurd music, we've created a music-tech platform that uses proprietary technology to turn music data into stronger promotional campaigns for artists. We've created both mobile and web applications which act as the 'digital marketing companion' for independent artists, facilitating data-led … responsible for the software development and technical infrastructure needed to design, train, deploy and scale machine learning models in production environments. You will harness a diverse array of data sources, from streaming services to social media, revolutionizing the way artists promote their music. Our Data Team members are full stack data professionals who independently … Management and Storage : Utilize PySpark for scalable processing of large datasets, implementing best practices for distributed computing. Optimize data storage and querying within a data lake environment to enhance accessibility and performance. ML R&D : Collaborate on model prototyping and development, identifying the most relevant and efficient solutions to the core problems that artists face. More ❯

for our service and growing by systematically creating conditions that encourage and amplify organic customer-driven growth. We have an exceptional exec team in place and a strong data & analytics capability. However, to realise our full potential, we need somebody to lead us through driving greater strategic clarity, deeper customer and market insight, and more precise measurement of … amplifies the key drivers of viral organic growth. Build a scalable customer and market intelligence engine anticipating behavioural shifts, emerging customer trends and market opportunities. Oversee an integrated data, analytics and decision-making infrastructure to enhance performance delivery across all teams. Responsibilities Strategy & Leadership With the CEO, lead and support development and execution of a comprehensive long- and … short-term data-driven, predictive growth strategy focused on advocacy, virality, and rapid expansion. Partner with the executive & senior leadership team to ensure insights and analytics shape company-wide strategic planning, from growth initiatives to product development. Lead the evolution of our customer, market, and competitive intelligence capabilities to drive long-term business success. Prepare and deliver insights More ❯

Data Architect Full Time/Permanent £80,000 - £100,000 + car allowance, generous bonus, private medical and other extensive benefits Hybrid - weekly travel required to Oxfordshire head office required initially, flexible longer term The Company: My client is an industry leading and award-winning financial services organisation who operate on a global scale. They are headquartered in … Oxfordshire, UK and also have offices throughout Europe and the US. The Role: We are looking for a driven and experienced Data Architect to join a rapidly growing team during an exciting period of change. This is a role where you can really make your mark and impact change very quickly . As the Data Architect … fast paced and complex business environment. Must have solid experience and knowledge of Azure data platform technologies, particularly Azure Data Factory, Azure Data Lake Storage and Azure Databricks. Must have hands on experience of working in Databricks, in particular design and usage of the Delta Lake storage format is required You will More ❯

has opened for an experienced Azure Project Consultant to join a leading Azure Expert MSP, delivering transformative cloud solutions for enterprise clients. This role involves guiding businesses through data centre exits, on-prem to cloud migrations, and landing zone deployments, collaborating with architects and engineers to implement secure, scalable, and high-performing Azure solutions. Key Responsibilities: Lead and … Strong experience with Azure security tools (Firewall, Bastion, App Gateway, Load Balancer, etc.) Hands-on knowledge of Azure Policy, Azure Advisor, and Entra ID Cloud migration experience, including data centre exits and on-prem to Azure transformations Certifications: AZ-104 Any exposure to the following will be beneficial … Experience with App Services, SQL MI/Azure SQL SB, Landing Zone deployments Understanding of Well-Architected and Cloud Adoption Frameworks Familiarity with Azure Containers, Synapse, Data Lake, Azure ML, vWAN, Azure Arc Knowledge of ADO, Bicep, Terraform, and PowerShell Certifications: AZ-305, AZ-400/AZ-500/AZ-700, AWS Cloud Practitioner, AWS Solutions Architect More ❯

Job Title: Data Architect/Data Modeler Location: Belfast Position Type: Contract/FTE Job Description: Key Responsibilities: - Work with data lakes, implementing strategies for faster data retrieval and optimized processing. - Design and manage dynamic data models, ensuring adaptability based on evolving datasets. - Implement caching and filtering techniques to improve … query performance. - Utilize big data technologies such as Apache Spark for large-scale data processing. - Apply AWS expertise, including Infrastructure-as-Code (IaC) for scalable deployment. - Work with SQL transpilers and predicate pushing to optimize query execution. - GraphQL - good to have Regards Anita More ❯

Data Analyst - Southampton - £30,000 - £45,000 per annum. VIQU have partnered with a fast-growing SaaS and data analytics company who are looking to expand their current team and hire a geospatial data analyst. The ideal candidate will hold experience with modelling physical objects, have extensive experience with Python (or a similar language … Southampton office. You will be joining a team of high achievers, working within a rapidly emerging industry with exposure to a range of technologies. Job Duties of the Data Analyst: Run and develop simulations and experiments, utilising scripting languages (Python) to do so. Utilising a range of data analysis techniques. Extracting and transforming data … applied to real-life objects. Must hold a year's professional experience using Python/Pyspark/Pandas. Experience with processing data and working with databases/datalakes (SQL). Strong understanding of data manipulation, analysis and processing. Ability to communicate to technical and non-technical stakeholders. To discuss this exciting opportunity in more detail, please More ❯

Data Architect with 15+ years of … experience, specializing in Microsoft Azure Cloud Platform Architecture and technologies. The ideal candidate should be having extensive expertise in Azure ADF, Data Bricks, Azure Data Lake with Synapse database experience. Responsibilities: • Lead the design and architecture of innovative and scalable solutions using Microsoft Azure Architecture and Technologies. • Collaborate closely with stakeholders, including development teams, project … technical evaluations of existing systems, identify areas for improvement, and propose effective solutions. • Architect and implement third-party integrations to enhance system functionality and interoperability. • Design and implement data strategies to optimize application performance and responsiveness. • Strong analytical and problem-solving abilities, with a strategic mindset and attention to detail. • Develop database optimization techniques, including indexing, query optimization More ❯

Role: Data & Migration Workstream Lead Location: London (Oxford Circus) Hybrid: 1-2 days per week in the office About Monument Technology Limited In building its core UK banking business, Monument has developed an open, scalable, cloud native and world-leading banking technology platform, the third pillar of the Banks’ strategy, which has allowed it to make significant inroads … deploy, scale and innovate at a fraction of the cost. From front end mobile apps, through core banking and CRM to the general ledger and a data lake, the platform can be consumed in whole (“bank in a box”) or in part as required. Our technology stack has gained positive attention from multiple players globally and over … Our dynamic integration frame and architecture allows the platform to scale globally with many components adaptable to other geographies. THE ROLE We’re looking for a highly experienced Data & Migration Workstream Lead to drive the planning, execution, and delivery of all data-related activities across multiple complex projects and workstreams. This is a hands-on leadership More ❯

reviews, system architecture, and best practice implementation. Ensure software development aligns with industry standards, security protocols, and business goals. Coordinate with cross-functional teams including Product, QA, and Data teams. Work closely with stakeholders to define technical requirements and translate them into effective solutions. Drive continuous improvement in system reliability, security, and efficiency. Collaborate with data teams to develop data lakes and integrate AI-driven solutions. Why This Role? This is an opportunity to step into a leadership position where you will have full ownership of the development function and play a key role in shaping the company’s technological future. Apply now to learn more about the opportunity. More ❯