101 to 125 of 724 Data Lake Jobs in the UK

trusted data models, reusable datasets, reporting foundations and self-service analytical capabilities. Bring the team onto a centralized snowflake data lake, starting from scratch and ensuring best practices for data stewardship are implemented (data modelling, quality and governance, for example). … analytics solutions. Act as the senior link between Ad Sales and central data platform teams, ensuring that the Snowflake data lake, ingestion patterns and governance controls support the division's current and future BI needs. Collaborating with engineers, analysts and technical teams across the BBC. ...

maintain consistent quality and performance across all platform components Manage dataset operations including creation, updates, and deletion across Snowflake and AWS data lake environments Provide technical support and troubleshooting for incidents, service requests, and enhancement requests with emphasis on timely resolution and minimal platform downtime Monitor … warehouse design and management Demonstrated expertise with Snowflake platform architecture, configuration, and optimization Strong experience with AWS services including data lake storage, compute, and related data platform services Proficiency in data modeling, SQL, and ETL/ELT processes Proven experience with ...

stored procedures, supporting the migration to AWSContribute to data pipelines, integrations and modern data warehouse/data lake developmentWork closely with business stakeholders to translate requirements into scalable technical solutionsCollaborate with legacy data warehouse teams and support data migration … experience, including a good understanding packages and stored proceduresYou have experience with AWS and modern data warehouse/data lake architecturesYou're a confident communicator with strong stakeholder management skills - able to bridge the gap between business and technologyYou're comfortable taking ownership, managing competing ...

journey, we're consolidating our data capabilities onto a single Azure Databricks platform, leveraging technologies including Azure Data Factory, Delta Lake, Unity Catalog and Power BI to unlock new insights and drive innovation. Joining our Enterprise Data Team , you'll play … organisation. You'll be responsible for building and maintaining data pipelines using Azure Databricks, Azure Data Factory and Delta Lake, supporting the modernisation of legacy platforms while ensuring data quality, reliability and performance. The role will also involve implementing data governance ...

journey, we're consolidating our data capabilities onto a single Azure Databricks platform, leveraging technologies including Azure Data Factory, Delta Lake, Unity Catalog and Power BI to unlock new insights and drive innovation. Joining our Enterprise Data Team , you'll play … organisation. You'll be responsible for building and maintaining data pipelines using Azure Databricks, Azure Data Factory and Delta Lake, supporting the modernisation of legacy platforms while ensuring data quality, reliability and performance. The role will also involve implementing data governance ...

from legacy/on-prem systems to AWS Develop and optimize data models for data warehouse/data lake (Redshift, S3) Build and orchestrate data workflows (Glue, Step Functions, Lambda, EMR) Implement data quality checks, validation, and reconciliation processes … data warehousing (Redshift, dimensional modeling) Proficiency in Python or Scala for data engineering Experience with S3-based data lake architecture (partitioning, cataloging, formats like Parquet) Experience migrating data pipelines from on-prem or other cloud platforms Understanding of data ...

Azure Data Factory (ADF). Develop and optimise data solutions using Azure Synapse Analytics. Work with Azure Data Lake for data storage and processing. Implement and maintain cloud infrastructure using Terraform. Use GitLab for source control, version management, and CI/… architectures and Real Time data processing. Key Technologies Azure | Azure Data Factory | Azure Synapse Analytics | Azure Data Lake | Terraform | GitLab | Databricks | Kafka | Kafka Connect Diversity & Inclusion At Scrumconnect Consulting, we believe that diversity drives innovation and better outcomes. We are committed ...

description Connect to your IndustryDo you want to be at the heart of some of the biggest and most ambitious AI & Data projects? Technology & Transformation: How do you make intelligent, future-proof decisions in a world where change is the one constant? That's exactly what … data lakes, data warehouses) and compute engine options (e.g., Spark, Databricks, Synapse Analytics, BigQuery).Implementing and optimising data lake and data warehouse solutions on cloud platforms using services such as object storage (e.g., S3, ADLS, GCS), cloud data warehouses ...

driving transformational change for our clients. We leverage proven technology, specialist teams, and extensive domain expertise to create robust technical foundations across platforms, data, and business applications. Our mission is fueled by a passion for technology as a means to solve complex business problems and achieve our clients … pipelines using Azure Data Factory, Synapse, Fabric Pipelines, or Databricks Integrate data from multiple systems into Azure Data Lake Storage or OneLake, ensuring consistency, reliability, and performance Develop and maintain ELT/ETL processes using Azure-native tools and Fabric capabilities, optimising ...

team you'll be working with:We are seeking a detail-oriented and capable Data Migration Engineer to join our Data & AI practice. The successful candidate will bring solid experience in data migration, ETL/ELT pipeline development, and cloud-based data … especially AWS GluePython/PySparkSQL querying and validationYAML configuration (desirable)Experience testing or supporting:ETL/ELT pipelinesData migration processesFamiliarity with:Data lake/Lakehouse concepts (e.g., Apache Iceberg)Distributed processing frameworks (e.g., Spark)Basic understanding of:ETL vs ELT approachesCloud-based data architecturesExposure ...

maintenance of data warehouses, ensuring efficient storage and retrieval. Work with data lakes (such as Azure Data Lake Storage) to ingest and process large volumes of data. Understand ETL (Extract, Transform, Load) processes and their impact on BI solutions. Implement data governance practices within the data warehouse and data lake environments. General Participate in cross-functional teams to provide insights and recommendations for data-related initiatives. Participate in training and development opportunities to enhance technical skills and domain knowledge. Demonstration of safe ...

Lead Data Engineer (Azure Data Platform) Inside IR35 Rate: £650.00 - £675.00 per day Location: UK Remote Clearance: Active SC Clearance Required The Opportunity We're looking for an experienced Lead Azure Data Engineer to play a key role in delivering a large-scale Azure … Data Platform programme. You'll be responsible for designing, building, and optimising modern data solutions that transform data from multiple source systems into trusted, governed, and reusable data products. This is a hands-on leadership role focused on data platform ...

example: Storage & Lakehouse: RDS, DynamoDB, S3, SageMaker Lakehouse Processing & ETL: Glue, EMR, Athena, CDC/DMS, DataBrew, Kinesis, dbt Data management & metadata: Lake Formation, Glue Data Catalog, SageMaker Catalog/DataZone AI: Bedrock (including AgentCore) and SageMaker AI On Azure, you’re fluent across … core services, for example: Storage & lakehouse: SQL, PostgreSQL, Cosmos, Blob, Data Lake, OneLake, Fabric Warehouse Processing & ETL: Fabric Data Factory, Data Factory CDC, Fabric/Synapse Spark, ADF, Synapse Pipelines, Fabric, dbt Data management & metadata: Purview AI: Foundry, (including Agent Service ...

Data Architect needed in Lincoln Paying £30.29 per hr ref 376142 Full time hours on a temporarybasis Key Responsibilities Data Architecture & Strategy Develop, implement, and maintain the organisation's data architecture strategy. Design scalable data models, data warehouses, data lakes, and integration solutions. Define and maintain data standards, frameworks, and best practices. Ensure the organisation's data architecture aligns with strategic business objectives. Lead data migration and transformation initiatives. Leadership & Management Provide technical leadership and mentoring to Data Engineers ...

DescriptionRole –Senior Databricks ArchitectTechnology – Databricks (Spark, PySpark/SQL) and data lake architecturesLocation – London, UKCompensation – Competitive (including bonus)Job DescriptionWe are looking for a Data Architect to design, build, and operate scalable data pipelines and data products, enabling a modern data … support analytics, reporting, and domain-owned data products.Primary Skills• Strong experience with Databricks (Spark, PySpark/SQL) and data lake architectures.• Hands-on experience with streaming technologies, especially Kafka/Confluent Kafka.• Experience integrating legacy data sources such as DB2 (mainframe ...

Senior Databricks Architect Technology: Databricks (Spark, PySpark/SQL) and data lake architectures. Location: London, UK. Compensation: Competitive (including bonus). Job Description We are looking for a Data Architect to design, build, and operate scalable data pipelines and data products … support analytics, reporting, and domain-owned data products. Primary Skills Strong experience with Databricks (Spark, PySpark/SQL) and data lake architectures. Hands‐on experience with streaming technologies, especially Kafka/Confluent Kafka. Experience integrating legacy data sources such as DB2 (mainframe ...

tomorrow’s public sector skills demands. This high-impact initiative offers a unique opportunity to join a team dedicated to building a scalable, data-driven recruitment ecosystem. Through redesigning, building, and rolling out a sophisticated Big Data system, our diverse roles span architecture, project management, data … scalable ETL/ELT pipelines using modern tools and frameworks. Cloud Platforms: Familiarity with AWS (Redshift, Athena, S3) or Azure (Synapse, Data Lake, ADF). Data Governance: Understanding of GDPR, PII, and secure data management principles. Communication: Strong data storytelling skills ...

Data, AI, Solutions & Engineering (DAISE) practice is looking for an experienced Data Engineer to join the team. In DAISE, we are focused on delivering value‐adding, sustainable data capabilities, aligned to our client’s specific needs. This expertise is applied across clients … . What you will be doing Defining and implementing on premise or cloud architectures, e.g. a cloud data warehouse, data lake or data platform to enable digital transformation Working with clients on data warehousing, building operational ETL/ELT data ...

Python, orchestrating complex data workflows through Databricks Workflows, Delta Live Tables, Azure Data Factory• Implement and maintain Delta Lake storage architectures and Unity Catalog governance frameworks, ensuring data quality, security, and compliance across the data estate• Design data solutions … boardsThe candidateExperience and knowledge requirements:RequiredExperience in data engineering disciplines, with advanced, hands-on expertise in Databricks platform including PySpark framework, Delta Lake storage format, and Unity Catalog governanceExtensive experience in designing advanced data models, including medallion architectures, data lakes, and enterprise data ...

all. We make it, store it, move it, sell it, and mend it. Join a team at the heart of Centrica's data transformation journey, building the data products and platforms that power insight, customer experiences and business decision-making across the organisation. As a Data Engineer, you will design and develop scalable data solutions that underpin strategic assets such as Centrica's Single Customer View and Property View, enabling trusted, governed and accessible data for analytics, operational processes, customer engagement, AI and digital use cases across the business. ...

Role We are seeking a capable Consulting Data Architect to join our Insurance Technology Consulting practice and play a pivotal role in shaping, selling, and delivering data-led consulting engagements for insurance clients.Location: London, United Kingdom (Hybrid)Scope: UK-Based Role with Global Client & Delivery RemitReporting … secure enterprise data platforms using technologies such as Microsoft Fabric, Azure Synapse Analytics, Azure Data Factory, Azure Data Lake, Databricks, and Snowflake.Data Management, Governance & Controls – Experience establishing data governance frameworks, operating models, metadata management, lineage, cataloguing, data quality ...

Role We are seeking a capable Consulting Data Architect to join our Insurance Technology Consulting practice and play a pivotal role in shaping, selling, and delivering data-led consulting engagements for insurance clients.Location: London, United Kingdom (Hybrid)Scope: UK-Based Role with Global Client & Delivery RemitReporting … secure enterprise data platforms using technologies such as Microsoft Fabric, Azure Synapse Analytics, Azure Data Factory, Azure Data Lake, Databricks, and Snowflake.Data Management, Governance & Controls – Experience establishing data governance frameworks, operating models, metadata management, lineage, cataloguing, data quality ...

Technology Stack You will work with a modern Microsoft data platform including: Azure Data Factory Azure Data Lake Gen2 Azure SQL Database/SQL Server Microsoft Power BI Tabular Models T-SQL PowerShell Azure Cloud Services Experience with the following would be advantageous … modelling techniques. Experience working with large and complex datasets from multiple sources. Expertise in Azure Data Factory, Azure Data Lake and Azure SQL technologies. Strong Power BI and data visualisation experience. Advanced SQL and data transformation skills. Excellent analytical ...

Technology Stack You will work with a modern Microsoft data platform including: Azure Data Factory Azure Data Lake Gen2 Azure SQL Database/SQL Server Microsoft Power BI Tabular Models T-SQL PowerShell Azure Cloud Services Experience with the following would be advantageous … modelling techniques. Experience working with large and complex datasets from multiple sources. Expertise in Azure Data Factory, Azure Data Lake and Azure SQL technologies. Strong Power BI and data visualisation experience. Advanced SQL and data transformation skills. Excellent analytical ...

design and maintenance of data warehouses, ensuring efficient storage and retrieval.Work with data lakes (such as Azure Data Lake Storage) to ingest and process large volumes of data.Understand ETL (Extract, Transform, Load) processes and their impact on BI solutions.Implement data governance … practices within the data warehouse and data lake environments.GeneralParticipate in cross-functional teams to provide insights and recommendations for data-related initiatives.Participate in training and development opportunities to enhance technical skills and domain knowledge.Demonstration of safe approach in working practises.Encouragement of attention ...