1 to 25 of 270 Permanent Data Lake Jobs

detail-oriented Business Intelligence Consultant to join this growing team. In this critical role, you will play a key role in transforming raw data into actionable insights that empower our business users to make data-driven decisions. Responsibilities: Utilise Power Query to extract, transform, and load … Design and develop robust and efficient data models within the Microsoft Data Fabric architecture (experience with Azure Data Lake Storage a plus). Perform thorough data validation to guarantee the integrity and reliability of information. Test and compare new reports against … audiences. Ability to work independently and manage multiple projects simultaneously. Preferred Skills: Experience with Microsoft Data Fabric architecture (Azure Data Lake Storage, Azure Data Factory, etc.). Experience with data migration methodologies. Experience with SQL queries for data manipulation. more »

What can you expect as a Senior Azure Data Engineer Working for a respected Microsoft Solutions Partner that will help you gain Microsoft certifications Joining a rapidly expanding and innovative organisation with exciting plans for 2024, that will give you the freedom to grow as an Azure Data … Working with clients across various industries on exciting projects that will see you utilising the latest products within the Microsoft Azure and SQL Data analytics stack A flat hierarchy and no-door policy. You won t have a manager but a mentor who will help you grow from … experience working with one or more of the following Azure data services (ADS): Azure Data Factory, Azure Data Lake Store and Azure Data Lake Analytics with uSQL, Azure Databricks, Azure SQL DB and Azure SQL DW/Azure Synapse Analytics more »

We are currently seeking a Data Engineer to join our client, a unique scale-up business who have received significant investment, that is entering a phase of large scale growth. This role will be a pivotal role in shaping the organisation s data operations, as the … company grows and matures. As a Data Engineer you will design and build data pipelines that will fuel our client s groundbreaking product lines. You'll collaborate across multiple disciplines to ensure seamless data flow, and go beyond pipelines to champion data … familiarity with RDBMS (Relational Database Management Systems). Experience working with cloud-based data warehousing solutions like Snowflake and data lake architectures. Experience in designing, developing, and maintaining data pipelines for efficient data movement. Experience with message brokers like Kafka or more »

and growth of our bespoke development capabilities. Leading design phases on Business IT programmes, validating requirements and ensuring that all aspects of UX, data and solution design are within scope, complete, and deliverable. Having responsibility for the architecture of data and solutions delivered to the business … premise application to the cloud Azure Cloud development, Blazor, NServiceBus, CQRS and Domain-driven design, Scrum/Agile, API Design and management skills Data Engineering projects using Data Lakes, Data factories, ETL techniques, Synapse and … PowerBI MS Logic and Power Apps Azure DevOps Experience with the following would be beneficial: AI Machine Learning Analytics: Analysis Services, Data Lake Analytics, Data Catalog DevOps: Application Insights Key Competencies Will use a coaching and influencing style to achieve outcomes rather than taking a more »

I'm looking for a Data Engineer with skills in Databricks, Azure Data Factory and SQL Server, for a Fixed Term Contract role until August 2025. It's with an international organisation headquartered in London, and this role will require you to be in their offices … near Victoria station 3 days per week. You will be joining a small cross-functional data team, where you will focus on the development of their newly-created Azure Data Platform - this will involve designing and developing ETL pipelines using Azure Data Factory, and … Factory Strong experience with Azure Databricks Some knowledge of the Azure tech stack would be beneficial but not essential: Databricks, Azure Data Lake, Azure Data Factory, Azure Synapse Analytics Benefits: Salary up to £70,000 depending on experience Please Note: This is a permanent role more »

I'm looking for a Data Engineer with skills in Databricks, Azure Data Factory and SQL Server, for a Fixed Term Contract role until August 2025. It's with an international organisation headquartered in London, and this role will require you to be in their offices … near Victoria station 3 days per week. You will be joining a small cross-functional data team, where you will focus on the development of their newly-created Azure Data Platform - this will involve designing and developing ETL pipelines using Azure Data Factory, and … Factory Strong experience with Azure Databricks Some knowledge of the Azure tech stack would be beneficial but not essential: Databricks, Azure Data Lake, Azure Data Factory, Azure Synapse Analytics Benefits: Salary up to £70,000 depending on experience Please Note: This is a permanent role more »

Senior Data Engineer opportunity to join a well-known financial services company working on a Remote first basis (once a month in the office on average). Senior Data Engineer - Remote First (office Nr Penrith) - Up to £65,000 + Great Benefits Purpose of Job Advise … Engineers. Providing Technical guidance to Data Engineers Knowledge, Experience and qualifications Excellent level of knowledge & experience in Azure Data, including DataLake, Data Factory, DataBricks, Azure SQL (Indicative experience = 5yrs+) Build and test processes supporting data extraction, data transformation, data structures, metadata, dependency and workload management. Knowledge on Spark architecture and modern Datawarehouse/Data-Lake/Lakehouse techniques Build transformation tables using SQL. Moderate level knowledge of Python/PySpark or equivalent programming language. PowerBI Data Gateways and DataFlows, permissions. Creation, utilisation more »

with Data Architecture, Data Management, The EDP product owner and the Technology team delivering the EDP. Responsibilities: Data Lake Structure and Design:  Validate the medallion model data lake structure (bronze, silver, gold) as the optimal approach and recommend alternatives if … needed.  Create a data lake design including namespace hierarchy, and naming standards. Design to optimise storage and retrieval based on data types, usage patterns, and the Bank’s emerging Data Domain segregation.  Establish policies for moving data between different storage tiers … within the data lake (e.g. hot, warm, cold storage) based on access frequency and cost considerations.  Establish policies for data integrity, consistency, conformance, and accuracy when transforming data from Bronze to Silver, and Silver to Bronze.  Develop a Data Modelling Design more »

LEAD DATA ARCHITECT All candidates must be located in the UK Are you looking for a company where YOUR VOICE is heard? Where you can MAKE A DIFFERENCE ? Do you THRIVE in a FAST-PACED work environment? Do you wake every morning EXCITED to work with GREAT PEOPLE … teamwork and transparency. We hold each other accountable and always have each other’s back! About the role: We are seeking a Lead Data Architect to oversee the design and implementation of our organization's data architecture. The ideal candidate will have a strong understanding of … governance Experience leading teams of data architects and engineers Strong understanding of data warehousing and data lake principles and technologies Experience with data governance frameworks and tools Excellent communication and collaboration skills Strong problem-solving and analytical skills Diversity more »

Job Description Summary The Senior Data Architect is accountable for enabling the technical vision and strategy for the organisations use of data. You will be accountable for defining and maintaining the technical data architecture roadmap offering advise and guidance to business stakeholders. You will ensure that … data architect solutions align with business requirements and data policies and regulations. You will be accountable for the choice of the toolsets required to design, build, manage and maintain the data storage facility within an agreed budget. You will lead a team of people … to deliver a simplified data solution ensuring that data is secure, available, and accurate to business stakeholders. Job Description You will review the ‘as is’ solution, work closely across TalkTalk and wider teams to identify an approach that allows us to move data across more »

As the Data Architect, you'll lead data architecture initiatives, establishing standards and practices essential for effective data management. Your focus will be on designing and implementing robust data and information systems, ensuring they meet stringent criteria for modelling, flow, lineage, quality … and security. Working closely with stakeholders and data governance teams, you'll drive adherence to established standards, facilitating data-driven decision-making processes across the organisation. This role encompasses: Developing and maintaining a comprehensive Data Architecture Framework and maturity roadmap. Enforcing best data … Azure stacks, including data pipelines, security and networking Expertise with data warehousing, data lakes and data lake houses Experience with master data management software and technologies Experience with ETL technologies including SQL Server, SSIS, Azure Data Factory more »

Job Title: AWS Data Architect Location: United Kingdom (U.K.) - Hybrid The Company: At Amber Labs, we are a cutting-edge UK and European technology consultancy dedicated to delivering exceptional value to our clients in the public sector. Our company culture promotes collaboration, autonomy, and rapid learning, empowering our … work environment focused on deliverable outcomes, along with a range of benefits to support our employees’ well-being and professional development. Key Responsibilities: Data Architecture Design: Collaborate with stakeholders to understand data requirements and design scalable and secure data architectures on AWS. Develop data models, data flow diagrams, and architectural documentation to support financial applications. Cloud Data Management: Design, implement, and manage cloud-based data solutions on AWS, including data lakes, data warehouses, and analytics platforms. Ensure the scalability, performance, and cost more »

unparalleled educational experience. By attracting and retaining the finest educators worldwide, our client ensures an environment where excellence thrives. Job Purpose As the Data Platform Engineer overseeing the Data Warehouse and analytics infrastructure, you will shape the backbone of our client's business analysis and reporting … capabilities, both now and into the future. They are seeking a visionary with a proven track record of architecting enterprise-level data ecosystems, someone who thrives on pushing the boundaries of what's possible in data manipulation. Your expertise will ensure data is not … Remarkable experience with Development and maintenance of data pipeline and ETL processes using Azure data Factory, Azure Data Lake and Azure Databricks. Exceptional experience developing codes, testing for quality assurance, administering Azure Databricks platform and exposure to Unity Catalog. Exposure of Performance Tuning more »

unparalleled educational experience. By attracting and retaining the finest educators worldwide, our client ensures an environment where excellence thrives. Job Purpose As the Data Platform Engineer overseeing the Data Warehouse and analytics infrastructure, you will shape the backbone of our client's business analysis and reporting … capabilities, both now and into the future. They are seeking a visionary with a proven track record of architecting enterprise-level data ecosystems, someone who thrives on pushing the boundaries of what's possible in data manipulation. Your expertise will ensure data is not … Remarkable experience with Development and maintenance of data pipeline and ETL processes using Azure data Factory, Azure Data Lake and Azure Databricks. Exceptional experience developing codes, testing for quality assurance, administering Azure Databricks platform and exposure to Unity Catalog. Exposure of Performance Tuning more »

The role As a Data Engineer at Storm you will work alongside other software and cloud infrastructure engineers to design, build and maintain data pipelines, data management solutions and analytics components for our client’s projects across the healthcare, justice and local government sectors. … About you We know that many Data Engineer's will come from a variety of backgrounds, and we encourage and value people who bring unique perspectives and knowledge to our team. Your previous experience will include working daily with disparate data sources, ingesting, transforming and processing … service components. implement data solutions using a range of Azure cloud data services such as Azure storage (data lake), Cosmos DB and Azure SQL (inc. Synapse) work with members of the Data Science team, supporting and building data processing more »

The role As a Data Engineer at Storm you will work alongside other software and cloud infrastructure engineers to design, build and maintain data pipelines, data management solutions and analytics components for our client’s projects across the healthcare, justice and local government sectors. … About you We know that many Data Engineer's will come from a variety of backgrounds, and we encourage and value people who bring unique perspectives and knowledge to our team. Your previous experience will include working daily with disparate data sources, ingesting, transforming and processing … service components. implement data solutions using a range of Azure cloud data services such as Azure storage (data lake), Cosmos DB and Azure SQL (inc. Synapse) work with members of the Data Science team, supporting and building data processing more »

Senior Data Engineer About us HENI is a technology company pioneering art markets and information. We work with world-leading artists and estates across various sectors, including printmaking, book publishing, NFTs, digital content, and art research – all underpinned by cutting-edge research in analytics, AI, and machine learning. … Our team comprises of talented web developers, data engineers, data scientists, and infrastructure specialists who thrive on solving complex problems. We enjoy developing machine learning tools and embracing the latest in infrastructure and coding methodologies. We maintain high standards of technical quality via thorough testing and … ingestion, Prefect for pipeline orchestration, AWS Glue for managed ETL, along with Pandas and PySpark for pipeline logic implementation. We utilize Delta Lake and PostgreSQL for data storage, emphasizing the importance of data integrity and version control in our workflows. Day-to-day more »

are a tech for good business dedicated to promoting transparency in the corporate world they are seeking a highly skilled and experienced Lead Data Engineer. You will be the first hire in the Data Engineering team allowing you to take the drivers seat regarding the direction … taken by the data team to follow. Key Responsibilities: Platform Development: Design, develop, maintain, test, and optimise a scalable and reliable data platform for collecting, processing, and transforming large volumes of company data from various sources. Collaboration: Work with cross-functional teams to understand … data platform requirements and deliver solutions that meet business needs. Optimisation: Enhance data storage and retrieval systems for high performance and data integrity. Quality Assurance: Implement and maintain a data quality and observability framework to ensure data accuracy, completeness, and more »

what role you come in and do, we will have a plan targeted to help you advance. What you’ll do As a Data Architect will be required to analyse an array of product offerings and business requirements within Mercury, to create and own the overall data architecture spanning the Mercury product suite. You will work with the product development teams (e.g. Core, Web, Integration, Data Practice and product owners) to evaluate and identify solutions that meet the needs of the business and customers, utilising industry standard best practices, patterns and creative design … support of Microsoft cloud-based services and products including (but not limited to): Azure Fabric Azure Data Factory Azure Data Lake Store Azure Data Lake Analytics Azure Storage Azure AI Search Azure Cosmos DB Azure SQL, Azure DW, Synapse and Azure Analytics more »

Lead Data Engineer A brand-new and exciting opportunity has arisen for a Lead Data Engineer with extensive experience in Azure, SQL, ETL and mentoring/managing engineers to join the UK's Largest Charity based remote. The Lead Data Engineer will be intrumental … engineering function. If you are currently a Lead Data Engineer who has a strong experience in Azure (ADF, Databricks, Data Lake), SQL, ETL and mentoring/managing a team who is looking for a step up then this would be perfect for you! Skills that … are required for the Lead Data Engineer are: Experience in Azure (ADF, Databricks and Data Lake) Experience leading/mentoring/managing a team and keen to manage a bigger team Strong data engineering principles working with SQL, ETL Pipelines and Cloud infrastructure more »

Role: Mid-Level Data Engineer Business Area: Engineering/Technology Reporting to: Head of Data Location: Hybrid/Remote About Validis Validis is a dynamic fintech company with a leading software platform, built for the extraction and standardisation of accounting data. Validis is both a B2B … and SasS with operations across the UK, US & Australia. We specialize in providing quality data for both audits and financial services, with the guarantee that our product will streamline processes for the financial industry. Our current client base holds a number of blue- chip companies, mainly comprising audit … excel in. We believe in the power of being... YOU! About the role We are looking for a skilled and motivated Mid-Level Data Engineer to join our growing team. This role will be fully remote, offering flexibility and the chance to contribute to our data more »

Role: Mid-Level Data Engineer Business Area: Engineering/Technology Reporting to: Head of Data Location: Hybrid/Remote About Validis Validis is a dynamic fintech company with a leading software platform, built for the extraction and standardisation of accounting data. Validis is both a B2B … and SasS with operations across the UK, US & Australia. We specialize in providing quality data for both audits and financial services, with the guarantee that our product will streamline processes for the financial industry. Our current client base holds a number of blue- chip companies, mainly comprising audit … excel in. We believe in the power of being... YOU! About the role We are looking for a skilled and motivated Mid-Level Data Engineer to join our growing team. This role will be fully remote, offering flexibility and the chance to contribute to our data more »

My leading Energy and Commodities conglomerate are currently seeking a skilled individual with experience in Python-based data pipeline, data lake house development and an appreciation for Object orientated languages such as C# or Java, using leading orchestration tools. In this role, you'll lead … the development of robust, fully tested data pipelines in Python using cutting-edge platforms like Dagster/Airflow and apply your expertise … in real-time data streaming solutions using Kafka. You'll play a key role in expanding their on prem data lake infrastructure to help enhance support for new lines of businesses they have including Systematic trading, Trading and Shipping commodities, ensuring strong data more »

Data Engineering Lead Salary - Circa £65,000 Are you someone with a proven track record in data engineering, ready to lead and innovate? We have an exciting opportunity for an experienced Data Engineering Manager to manage a data development team who are … responsible for delivering IT data solutions across Aviva. As demand for data continues to grow across the organisation, we are in need of an experienced team lead from a technology … background to manage development projects through the full project life cycle with specific focus on ETL/data store (data lake/warehouse) in an Amazon Web Services environment. A bit about the job: This is a great opportunity for the right technology-aware individual more »

We are currently seeking a Data Engineer to join our client, a unique scale-up business who have received significant investment, that is entering a phase of large scale growth. This role will be a pivotal role in shaping the organisation’s data operations, as the … company grows and matures. As a Data Engineer you will design and build data pipelines that will fuel our client’s groundbreaking product lines. You'll collaborate across multiple disciplines to ensure seamless data flow, and go beyond pipelines to champion data … familiarity with RDBMS (Relational Database Management Systems). Experience working with cloud-based data warehousing solutions like Snowflake and data lake architectures. Experience in designing, developing, and maintaining data pipelines for efficient data movement. Experience with message brokers like Kafka or more »