101 to 125 of 611 Data Lake Jobs in the UK

from legacy/on-prem systems to AWS Develop and optimize data models for data warehouse/data lake (Redshift, S3) Build and orchestrate data workflows (Glue, Step Functions, Lambda, EMR) Implement data quality checks, validation, and reconciliation processes … data warehousing (Redshift, dimensional modeling) Proficiency in Python or Scala for data engineering Experience with S3-based data lake architecture (partitioning, cataloging, formats like Parquet) Experience migrating data pipelines from on-prem or other cloud platforms Understanding of data ...

from legacy/on-prem systems to AWS Develop and optimize data models for data warehouse/data lake (Redshift, S3) Build and orchestrate data workflows (Glue, Step Functions, Lambda, EMR) Implement data quality checks, validation, and reconciliation processes … data warehousing (Redshift, dimensional modeling) Proficiency in Python or Scala for data engineering Experience with S3-based data lake architecture (partitioning, cataloging, formats like Parquet) Experience migrating data pipelines from on-prem or other cloud platforms Understanding of data ...

Shaping a smarter, more agile world! AtkinsRéalis is growing its Solution Design team and is looking for Data Architects who are curious, who want to become part of our customers' digital journey and who want to have a voice in shaping our future digital world. Is this … Overview Shaping a smarter, more agile world! AtkinsRéalis is growing its Solution Design team and is looking for Data Architects who are curious, who want to become part of our customers' digital journey and who want to have a voice in shaping our future digital world. Is this ...

driving transformational change for our clients. We leverage proven technology, specialist teams, and extensive domain expertise to create robust technical foundations across platforms, data, and business applications. Our mission is fueled by a passion for technology as a means to solve complex business problems and achieve our clients … pipelines using Azure Data Factory, Synapse, Fabric Pipelines, or Databricks Integrate data from multiple systems into Azure Data Lake Storage or OneLake, ensuring consistency, reliability, and performance Develop and maintain ELT/ETL processes using Azure-native tools and Fabric capabilities, optimising ...

maintenance of data warehouses, ensuring efficient storage and retrieval. Work with data lakes (such as Azure Data Lake Storage) to ingest and process large volumes of data. Understand ETL (Extract, Transform, Load) processes and their impact on BI solutions. Implement data governance practices within the data warehouse and data lake environments. General Participate in cross-functional teams to provide insights and recommendations for data-related initiatives. Participate in training and development opportunities to enhance technical skills and domain knowledge. Demonstration of safe ...

example: Storage & Lakehouse: RDS, DynamoDB, S3, SageMaker Lakehouse Processing & ETL: Glue, EMR, Athena, CDC/DMS, DataBrew, Kinesis, dbt Data management & metadata: Lake Formation, Glue Data Catalog, SageMaker Catalog/DataZone AI: Bedrock (including AgentCore) and SageMaker AI On Azure, you’re fluent across … core services, for example: Storage & lakehouse: SQL, PostgreSQL, Cosmos, Blob, Data Lake, OneLake, Fabric Warehouse Processing & ETL: Fabric Data Factory, Data Factory CDC, Fabric/Synapse Spark, ADF, Synapse Pipelines, Fabric, dbt Data management & metadata: Purview AI: Foundry, (including Agent Service ...

Data Architect needed in Lincoln Paying £30.29 per hr ref 376142 Full time hours on a temporarybasis Key Responsibilities Data Architecture & Strategy Develop, implement, and maintain the organisation's data architecture strategy. Design scalable data models, data warehouses, data lakes, and integration solutions. Define and maintain data standards, frameworks, and best practices. Ensure the organisation's data architecture aligns with strategic business objectives. Lead data migration and transformation initiatives. Leadership & Management Provide technical leadership and mentoring to Data Engineers ...

DescriptionRole –Senior Databricks ArchitectTechnology – Databricks (Spark, PySpark/SQL) and data lake architecturesLocation – London, UKCompensation – Competitive (including bonus)Job DescriptionWe are looking for a Data Architect to design, build, and operate scalable data pipelines and data products, enabling a modern data … support analytics, reporting, and domain-owned data products. Primary Skills Strong experience with Databricks (Spark, PySpark/SQL) and data lake architectures. Hands-on experience with streaming technologies, especially Kafka/Confluent Kafka. Experience integrating legacy data sources such as DB2 (mainframe ...

Senior Databricks Architect Technology: Databricks (Spark, PySpark/SQL) and data lake architectures. Location: London, UK. Compensation: Competitive (including bonus). Job Description We are looking for a Data Architect to design, build, and operate scalable data pipelines and data products … support analytics, reporting, and domain-owned data products. Primary Skills Strong experience with Databricks (Spark, PySpark/SQL) and data lake architectures. Hands‐on experience with streaming technologies, especially Kafka/Confluent Kafka. Experience integrating legacy data sources such as DB2 (mainframe ...

Data, AI, Solutions & Engineering (DAISE) practice is looking for an experienced Data Engineer to join the team. In DAISE, we are focused on delivering value‐adding, sustainable data capabilities, aligned to our client’s specific needs. This expertise is applied across clients … . What you will be doing Defining and implementing on premise or cloud architectures, e.g. a cloud data warehouse, data lake or data platform to enable digital transformation Working with clients on data warehousing, building operational ETL/ELT data ...

tomorrow’s public sector skills demands. This high-impact initiative offers a unique opportunity to join a team dedicated to building a scalable, data-driven recruitment ecosystem. Through redesigning, building, and rolling out a sophisticated Big Data system, our diverse roles span architecture, project management, data … scalable ETL/ELT pipelines using modern tools and frameworks. Cloud Platforms: Familiarity with AWS (Redshift, Athena, S3) or Azure (Synapse, Data Lake, ADF). Data Governance: Understanding of GDPR, PII, and secure data management principles. Communication: Strong data storytelling skills ...

all. We make it, store it, move it, sell it, and mend it. Join a team at the heart of Centrica's data transformation journey, building the data products and platforms that power insight, customer experiences and business decision-making across the organisation. As a Data Engineer, you will design and develop scalable data solutions that underpin strategic assets such as Centrica's Single Customer View and Property View, enabling trusted, governed and accessible data for analytics, operational processes, customer engagement, AI and digital use cases across the business. ...

hands-on delivery, you will: Help define and deliver the data platform roadmap Design and deliver solutions using Azure Data Lake, Databricks, SQL and Python Lead the transformation and migration of existing systems and data services Translate the wider programme into practical workstreams … mentored engineers and is ready for greater responsibility. You must be able to demonstrate: Hands-on experience with Azure Data Lake Strong practical experience with Databricks and SQL Experience using Python within a data engineering environment Data platform design and architecture experience ...

Technology Stack You will work with a modern Microsoft data platform including: Azure Data Factory Azure Data Lake Gen2 Azure SQL Database/SQL Server Microsoft Power BI Tabular Models T-SQL PowerShell Azure Cloud Services Experience with the following would be advantageous … modelling techniques. Experience working with large and complex datasets from multiple sources. Expertise in Azure Data Factory, Azure Data Lake and Azure SQL technologies. Strong Power BI and data visualisation experience. Advanced SQL and data transformation skills. Excellent analytical ...

Technology Stack You will work with a modern Microsoft data platform including: Azure Data Factory Azure Data Lake Gen2 Azure SQL Database/SQL Server Microsoft Power BI Tabular Models T-SQL PowerShell Azure Cloud Services Experience with the following would be advantageous … modelling techniques. Experience working with large and complex datasets from multiple sources. Expertise in Azure Data Factory, Azure Data Lake and Azure SQL technologies. Strong Power BI and data visualisation experience. Advanced SQL and data transformation skills. Excellent analytical ...

all. We make it, store it, move it, sell it, and mend it. Join a team at the heart of Centrica's data transformation journey, building the data products and platforms that power insight, customer experiences and business decision-making across the organisation. As a Data Engineer, you will design and develop scalable data solutions that underpin strategic assets such as Centrica's Single Customer View and Property View, enabling trusted, governed and accessible data for analytics, operational processes, customer engagement, AI and digital use cases across the business. ...

products including data transformation, data modelling, and reporting on both on-premises and cloud environments Build modern data lake/lakehouse architectures that support scalable, flexible data processing Enhance platform integration by leveraging domain events to enable more efficient and real … data engineering, with demonstrated expertise in designing and implementing scalable, flexible modern data architectures Experience building modern data lake/lakehouse architectures (e.g., Apache Iceberg) in cloud environments Strong knowledge of AWS (S3, Redshift) or Azure data services Experience with ...

team you'll be working with: We are seeking an accomplished and detail-oriented Lead Data Migration Engineer to join our Data & AI practice. The successful candidate will bring deep expertise in data migration, ETL/ELT pipeline engineering, and cloud-native data … validating: End-to-end ETL/ELT pipelinesData migration workflowsFamiliarity with: Apache Iceberg and Lakehouse architecturesDistributed processing frameworks (e.g., Apache Spark)Data lake environmentsUnderstanding of: ETL vs ELT design patternsMigration performance tuningExperience with version control and CI/CD tools desirableWho we are: We're a business ...

seeking an experienced Enterprise Data Architect to define and lead the organisation's target-state data architecture, ensuring our data landscape supports business growth, operational excellence, regulatory compliance, and future AI-driven innovation. This is a strategic role at the heart of our data … management, finance, CRM, analytics, and AI platforms. Working closely with senior technology and business leaders, you will help shape a scalable and sustainable data ecosystem that reduces integration complexity, improves data accessibility, and enables advanced analytics and predictive capabilities. What you will do? As our Enterprise ...

seeking an experienced Enterprise Data Architect to define and lead the organisation's target-state data architecture, ensuring our data landscape supports business growth, operational excellence, regulatory compliance, and future AI-driven innovation. This is a strategic role at the heart of our data … management, finance, CRM, analytics, and AI platforms. Working closely with senior technology and business leaders, you will help shape a scalable and sustainable data ecosystem that reduces integration complexity, improves data accessibility, and enables advanced analytics and predictive capabilities. What you will do? As our Enterprise ...

Data Factory (ADF) . Develop and optimise data solutions using Azure Synapse Analytics . Work with Azure Data Lake for data storage and processing. Implement and maintain cloud infrastructure using Terraform . Use GitLab for source control, version management … experience with Azure Data Factory (ADF) . Strong experience with Azure Synapse Analytics . Experience working with Azure Data Lake . Hands-on experience with Terraform and Infrastructure as Code (IaC). Experience using GitLab , including GitLab CI/CD. Strong data engineering ...

Senior Data Engineer Make a difference and advance your career by helping deliver some of the UK’s most important #tech4good projects, making the world a smarter, safer, greener, and healthier place. You’ll join a talented team of dynamic and driven professional problem solvers; creative thinkers … digital transformation consultancy. We deliver multi-Queen’s Award for innovation winning platforms and services that support large-scale digital transformation. Our digital, data and technology solutions are used by globally recognised public and private sector brands operating in a variety of sectors including Civil Defence, Healthcare, Sustainable ...

global enterprise. Role Overview: This position is for a technical lead for data acquisition within the AI Labs/Data Lake team. We are seeking a Lead Java Software Engineer to join our Wholesale Data and Operations Technology team. This position will serve … acquisition processes and will be instrumental in shaping the architecture, implementation, and ongoing evolution of our comprehensive AI Labs/Data Lake platform. This role demands a seasoned professional who can seamlessly blend hands-on technical expertise with strategic leadership capabilities to deliver robust, scalable data ...

global enterprise. Role Overview: This position is for a technical lead for data acquisition within the AI Labs/Data Lake team. We are seeking a Lead Java Software Engineer to join our Wholesale Data and Operations Technology team. This position will serve … acquisition processes and will be instrumental in shaping the architecture, implementation, and ongoing evolution of our comprehensive AI Labs/Data Lake platform. This role demands a seasoned professional who can seamlessly blend hands‐on technical expertise with strategic leadership capabilities to deliver robust, scalable data ...

shape the future of motor insurance? Prima could be the place for you. Since 2015, we’ve been using our love of data and tech to rethink motor insurance and bring drivers a great experience at a great price. Our story began in Italy, where we’ve quickly … expanding to help millions more drivers in the UK and Spain. To help fuel that growth, we need a Senior Data Engineer to join our Engineering team. This team is the beating heart of Prima. You’ll be joining over 350 engineers across software development, infrastructure, operations ...