1 to 25 of 243 Data Lake Jobs in England

Data/Data Specialisation). Extensive experience in Data Engineering, in both Cloud & On-Prem, Big Data and Data Lake environments. Expert knowledge in data technologies, data transformation tools, data governance techniques. Strong analytical and problem-solving abilities. Good understanding of Quality and … or Azure hands-on experience. ETL Tools such as Azure Data Fabric (ADF) and Databricks or similar ones. Data Lakes: Azure Data, Delta Lake, Data Lake or Databricks Lakehouse. Certifications: AWS, Azure, or Cloudera certifications are a plus. To be considered for this role you MUST have in-depth experience … Engineer, Spark, Java, Python, PySpark, Scala, Big Data, AWS, Azure, Cloud, On-Prem, ETL, Azure Data Fabric, ADF, Hadoop , HDFS , Azure Data, Delta Lake, Data Lake Please note that due to a high level of applications, we can only respond to applicants whose skills and qualifications are suitable for this More ❯

About Us: ATREIDES is a leader in advanced geospatial technologies, big data solutions, and data analytics. We specialize in providing cutting-edge software platforms that enable our clients to process, analyze, and visualize large-scale geospatial and sensor data for critical decision-making. Our mission is to deliver innovative, scalable, and high-performance platforms … that help organizations unlock the full potential of their data for various industries, including defense, national security, environmental monitoring, and urban planning. We are seeking a highly skilled Senior Data Engineer with a strong focus on big data, data analytics, and geospatial data intelligence to join our engineering team. In … this role, you will help design and develop robust, scalable platforms that handle vast amounts of geospatial data and provide actionable insights through data analytics. Job Overview: As a Senior Data Engineer at ATREIDES, you will be responsible for developing and optimizing our core software platform, with an emphasis on integrating and processing big More ❯

Join to apply for the Data Architect - Defence role at Kainos 5 days ago Be among the first 25 applicants Join to apply for the Data Architect - Defence role at Kainos Join Kainos and Shape the Future At Kainos, were problem solvers, innovators, and collaborators - driven by a shared mission to create real impact. Whether were … youll be part of a diverse, ambitious team that celebrates creativity and collaboration. Ready to make your mark? Join us and be part of something bigger. As a Data Architect (Manager) in Kainos, youll be responsible for providing SME guidance in traditional data architecture disciplines around data structures, data flows, data … defining information handling models and capacity planning across heterogeneous data store technologies. Experience of establishing data governance processes Experience of architecting a data lake and solutions that reside within a data lake ecosystem Enterprise Data Architecture experience. Embracing our differences At Kainos, we believe in the power of More ❯

Data Engineer Data Engineer - SQL Server 2022, ADF, ETL Pipelines, Azure Synapse Analytics, Azure SQL, Azure Data Lake, SSIS, SSAS, SSRS, Power BI, Fabric, Data Warehousing, Agile - Hybrid (Warrington) - £50k + excellent bonus + excellent bens An experienced Data Engineer is urgently required by my client with strong experience … of the Microsoft BI stack and Azure Services (SQL Server 2022, ADF, ETL Pipelines, Azure Synapse Analytics, Azure SQL, Azure Data Lake, SSIS, SSAS, SSRS, Power BI, Fabric). You will be responsible for helping to build a new, best practice-based data warehouse solution which will function Enterprise-wide. You will be involved in … role on a product development programme using the Microsoft BI stack and Azure Services (SQL Server 2022, ADF, ETL Pipelines, Azure Synapse Analytics, Azure SQL, Azure Data Lake, SSIS, SSAS, SSRS, Power BI, Fabric) - Work collaboratively within an Agile team of Developers, Testers and BA/Product Owners on a set of new greenfield projects - Observe object More ❯

that solves real-world problems for large blue-chip companies and governments worldwide. We're pioneers of meaningful AI : our solutions go far beyond chatbots. We are using data and AI to solve the world's biggest issues in telecommunications, sustainable water management, energy, healthcare, climate change, smart cities, and other areas that have a real impact on … re underpinned by over 300 engineers and scientists working across Africa, Europe, the UK and the US. ABOUT THE ROLE Sand Technologies focuses on cutting-edge cloud-based data projects, leveraging tools such as Databricks, DBT, Docker, Python, SQL, and PySpark to name a few. We work across a variety of data architectures such as Data Mesh, lakehouse, data vault and data warehouses. Our data engineers create pipelines that support our data scientists and power our front-end applications. This means we do data-intensive work for both OLTP and OLAP use cases. Our environments are primarily cloud-native spanning AWS, Azure and GCP More ❯

condition, or are neurodiverse. If you'd like to apply and need adjustments made, you can let us know in your application. About the role As an Azure Data Engineer, your role involves designing, implementing, and managing data solutions on the Microsoft Azure cloud platform. Your responsibilities will include, but not limited to: Data … grow! Data Management and Storage: You'll work with various data storage options provided by Microsoft Azure, including Azure SQL Database, Azure Data Lake Storage, and Azure Blob Storage. Your task is to design data storage systems that scale, perform efficiently, and meet the organisation's requirements. Essential Skills Proficiency in … SQL, SQL Server Understand parallel processing and data architecture patterns Azure Data Factory (ADF) Azure Synapse Analytics Azure Data Lake Storage Ability to design and implement pipelines in ADF. Build complex efficient processes for transformation using SQL & ADF Pipelines Ensure secure, organised, and reliable data pipelines Troubleshoot all data More ❯

Key Responsibilities: Data Pipelines and ETL Processes ETL Design: Design and implement ETL processes within MaPS architectural patterns to extract, transform, and load data from various source systems into our reporting solutions. Pipeline Development: Develop and configure meta-data driven data pipelines using data orchestration tools such as Azure Data … unusual data profiles and establish recovery processes to maintain data integrity. Azure Services Utilise Azure Cloud based data technology platforms, Data Lake based storage & Cloud data pipeline/orchestration tools for efficient data storage, processing, and transfer. Ensure the scalability and reliability of data solutions … implementing ETL processes. Experience in using SQL to query and manipulate data. Proficiency in Azure data tooling such as Synapse Analytics, Microsoft Fabric, Azure Data Lake Storage/One Lake, and Azure Data Factory. Understanding of data extraction from vendor REST APIs. Spark/Pyspark or Python skills a bonus More ❯

Join our rapidly expanding team as a hands-on Cloud Data Analytics Platform Engineer and play a pivotal role in shaping the future of data at Citi. We're building a cutting-edge, multi-cloud data analytics platform that empowers our users with secure, scalable, and efficient data insights. This role sits … like S3/GCS, Glue, BigQuery, Pub/Sub, SQS/SNS, MWAA/Composer, and more to create a seamless data experience. (Required) Data Lake , Data Zone, Data Governance: Design, build, and manage data lakes and data zones within our cloud environment, ensuring data … Infrastructure Automation Cloud Expertise: Proven hands-on experience with AWS and/or GCP, including a deep understanding of their data analytics service offerings. Data Lake/Zone/Governance Experience: Demonstrable experience designing, building, and managing data lakes and data zones. Familiarity with data governance tools and frameworks. More ❯

Join our rapidly expanding team as a hands-on Cloud Data Analytics Platform Engineer and play a pivotal role in shaping the future of data at Citi. We're building a cutting-edge, multi-cloud data analytics platform that empowers our users with secure, scalable, and efficient data insights. This role sits … like S3/GCS, Glue, BigQuery, Pub/Sub, SQS/SNS, MWAA/Composer, and more to create a seamless data experience. (Required) Data Lake , Data Zone, Data Governance: Design, build, and manage data lakes and data zones within our cloud environment, ensuring data … Infrastructure Automation Cloud Expertise: Proven hands-on experience with AWS and/or GCP, including a deep understanding of their data analytics service offerings. Data Lake/Zone/Governance Experience: Demonstrable experience designing, building, and managing data lakes and data zones. Familiarity with data governance tools and frameworks. More ❯

Senior Data Engineer - (Azure/Databricks) page is loaded Senior Data Engineer - (Azure/Databricks) Apply locations London - Scalpel time type Full time posted on Posted 7 Days Ago job requisition id REQ05851 This is your opportunity to join AXIS Capital - a trusted global provider of specialty lines insurance and reinsurance. We stand apart for our … data solutions are secure, efficient, and optimized. Key Responsibilities: Design and implement data solutions using Azure services, including Azure Databricks, ADF, and Data Lake Storage. Develop and maintain ETL/ELT pipelines to process structured and unstructured data from multiple sources. - Automate loads using Databricks workflows and Jobs Develop, test and … warehousing, and real-time streaming. Knowledge of developing and processing full and incremental loads. Experience of automated loads using Databricks workflows and Jobs Expertise in Azure Databricks, including Delta Lake, Spark optimizations, and MLflow. Strong experience with Azure Data Factory (ADF) for data integration and orchestration. Hands-on experience with Azure DevOps, including pipelines, repos More ❯

Join to apply for the Data Architect (Trading) role at Keyrock Join to apply for the Data Architect (Trading) role at Keyrock About Keyrock Since our beginnings in 2017, we've grown to be a leading change-maker in the digital asset space, renowned for our partnerships and innovation. Data Architect About Keyrock Since … s progress with our research and governance initiatives. At Keyrock, we're not just envisioning the future of digital assets. We're actively building it. Position Overview The Data Architect is responsible for designing, implementing, and maintaining an organization's data architecture and strategy, ensuring that data is collected, stored, and processed efficiently and … securely to support business intelligence, data analytics, and machine learning operations (MLOps) practices. Key Responsibilities Designing Data Architecture: Plan and implement a robust, scalable data architecture that integrates data from various sources and supports diverse analytical needs, while optimizing costs and meeting business requirements. Implementing Data Engineering Pipelines: Design and More ❯

Senior Data Ops Data EngineerLeeds 2/3 days per week£55,000-£67,000+ Excellent Benefits18 Months FTC with long term potential Your New Company Our large public sector organisation are seeking a senior Data Platform DataOps Engineer to serve as our clients first DataOps specialist in a growing team of Data … and DevOps teams to align operational strategies with technical and business requirements. Optimize operational performance and cost management for services including Azure Data Factory, Azure Databricks, Delta Lake, and Azure Data Lake Storage. Serve as the domain expert in DataOps by providing strategic guidance, mentoring colleagues, and driving continuous process improvements. What you will … within the Microsoft Azure ecosystem. Hands-on expertise with Azure Data Platform with components such as Azure Data Factory, Azure Databricks, Azure Data Lake Storage, Delta Lake, Azure SQL, Purview and APIM. Proficiency in developing CI/CD data pipelines and strong programming skills in Python, SQL, Bash, and PySpark More ❯

expanding the existing mission critical ecosystem of Data and BI systems for analytics and decision making, using Primarily Data Warehouse/Mart, Data Lake, Self Service BI Tools and Data Science/ML platform using Oracle, Snowflake and AWS tech stack. Qualifications We are looking for a skilled Lead Data … pipeline/ELT development, including performance tuning Very good SQL skills Past experience and solid understanding of end to end Data Warehouse, Data Lake and BI design and delivery at scale (high level and detailed) and various architectural strategies Solid information architecture skills/experience: Data ingestions, and Data Warehouse and Date Lake modelling experience (Star Schema, Snowflake Schema, Data Vault etc.) Past hands-on development experience in at least one enterprise analytics database like Oracle/Terradata/SQL Server/Snowflake Understanding of AWS cloud and fundamental services like IAM, Lambda, S3 etc. Experience of liaising with both in-house and third More ❯

ll be part of a diverse, ambitious team that celebrates creativity and collaboration. Ready to make your mark? Join us and be part of something bigger. As a Data Architect (Manager) in Kainos, you'll be responsible for providing SME guidance in traditional data architecture disciplines around data structures, data flows, data sourcing and data governance. Data Architects work closely with clients to understand their data requirements and take responsibility for ensuring solutions are fit for purpose. They also provide technical leadership for the rest of the team in the area of data. Data Architects may also work at the solution or … defining information handling models and capacity planning across heterogeneous data store technologies. Experience of establishing data governance processes Experience of architecting a data lake and solutions that reside within a data lake ecosystem At Kainos, we believe in the power of diversity, equity and inclusion. We are committed to building More ❯

ll be part of a diverse, ambitious team that celebrates creativity and collaboration. Ready to make your mark? Join us and be part of something bigger. As a Data Architect (Manager) in Kainos, you'll be responsible for providing SME guidance in traditional data architecture disciplines around data structures, data flows, data sourcing and data governance. Data Architects work closely with clients to understand their data requirements and take responsibility for ensuring solutions are fit for purpose. They also provide technical leadership for the rest of the team in the area of data. Data Architects may also work at the solution or … defining information handling models and capacity planning across heterogeneous data store technologies. Experience of establishing data governance processes Experience of architecting a data lake and solutions that reside within a data lake ecosystem At Kainos, we believe in the power of diversity, equity and inclusion. We are committed to building More ❯

is a strong culture where people are encouraged to be present, bold and curious, allowing them to achieve their individual goals. We are currently looking for a Lead Data Architect, to be based in London on a full-time basis, reporting into the Data & Transformation Strategy Lead. Responsibilities: Collaborate with business stakeholders, data scientists … analysts, and IT teams end to end to understand business needs and translate them into effective data architecture solutions. Design and implement scalable and efficient data architectures that support business goals, including data warehousing, data lakes, and real-time data processing. Lead the evaluation, selection, and integration of new data technologies and tools to optimise data management processes. Support the development of data products, through robust architecture, data governance policies, ensuring data quality, integrity, security, and compliance with relevant regulations (GDPR). Lead a team to develop and maintain data models, schemas, and data dictionaries to More ❯

BI Developer BI Developer - SQL Server 2022, ADF, ETL Pipelines, Azure Synapse Analytics, Azure SQL, Azure Data Lake, SSIS, SSAS, SSRS, Power BI, Fabric, Data Warehousing, Agile - Hybrid (Warrington) - £45-50k + excellent bonus + excellent bens An experienced BI Developer is urgently required by my client with strong experience of the Microsoft BI … stack and Azure Services (SQL Server 2022, ADF, ETL Pipelines, Azure Synapse Analytics, Azure SQL, Azure Data Lake, SSIS, SSAS, SSRS, Power BI, Fabric). You will be responsible for helping to build a new, best practice-based reporting solution which will function Enterprise-wide. You will be involved in the full software development lifecycle on this … role on a product development programme using the Microsoft BI stack and Azure Services (SQL Server 2022, ADF, ETL Pipelines, Azure Synapse Analytics, Azure SQL, Azure Data Lake, SSIS, SSAS, SSRS, Power BI, Fabric) - Work collaboratively within an Agile team of Developers, Testers and BA/Product Owners on a set of new greenfield projects - Observe object More ❯

Social network you want to login/join with: We are looking for an experienced Data Governance Lead to drive enterprise-wide data governance initiatives with a focus on modern cloud data platforms, specifically Databricks and Microsoft Azure. This role will define and enforce data policies, standards, and best practices that ensure … business objectives and compliance requirements. Collaborate with data engineering and platform teams to integrate data governance within Databricks and Azure environments, including Data Lake, Synapse, and Purview. Define and enforce data classification, access management, and lineage tracking using Azure-native and third-party tools. Lead the Data Governance Council … governance, data management, or data architecture. Hands-on experience with Databricks and/or Microsoft Azure data services (e.g., Data Lake Storage, Synapse, Purview, Azure Data Factory). Strong understanding of data governance frameworks such as DAMA-DMBOK. Familiarity with data privacy, security, and More ❯

management technical knowledge so as to be able to work across full data cycle. • Proven Experience working with AWS data technologies (S3, Redshift, Glue, Lambda, Lake formation, Cloud Formation), GitHub, CI/CD • Coding experience in Apache Spark, Iceberg or Python (Pandas) • Experience in change and release management. • Experience in Database Warehouse design and data … ETL/ELT development in AWS preferred with experience in the following: • Strong experience in some of the AWS services like Redshift, Lambda,S3,Step Functions, Batch, Cloud formation, Lake Formation, Code Build, CI/CD, GitHub, IAM, SQS, SNS, Aurora DB • Good experience with DBT, Apache Iceberg, Docker, Microsoft BI stack (nice to have) • Experience in data warehouse design (Kimball and lake house, medallion and data vault) is a definite preference as is knowledge of other data tools and programming languages such as Python & Spark and Strong SQL experience. • Experience is building Data lake and building CI/CD data pipelines • A candidate is expected More ❯

EIT Oxford fosters a culture of collaboration, innovation, and resilience, valuing diverse expertise to drive sustainable solutions to humanity's enduring challenges. We are currently recruiting for a Data Architect to support the EIT Pathogen Programme. In this role, you will play a pivotal part in designing and implementing cutting-edge data architectures to support the … pathogen mission. You'll collaborate closely with cross-functional teams to understand business requirements and translate them into robust data models and architectures. As a Data Architect , you'll have the opportunity to shape the future of our data platform and collaborate with platform and product teams to deliver analytical and AI products to … architecture governance processes Essential Knowledge, Skills and Experience: Knowledge and experience of architecting and delivering modern data platform standards, tools and patterns including data lakes, lake houses, iceberg, data mesh Experience of architecting, building, and delivering modern data platforms at scale Familiar with TOGAF and other enterprise architecture frameworks Experience and More ❯

Company Details Required Skills & Knowledge: Solid hands-on experience with Microsoft Technology Stack (2022) Extensive experience of SQL/data warehousing Proven experience of data engineering activities such as release management, environment controls, CICD pipeline orchestration. Experience working with Python and Power BI Proven ability of Microsoft BI software development including database/reporting development, full … development cycle Strong data modelling, problem solving, information analysis, attention to detail, flexibility of approach. The ability to deal directly with business functions and interpret their thoughts into successful data & analytics solutions. The ability to design and develop complex BI/MI solutions in line with agreed architectural principles and toolsets. Capable of working closely with … movements, including assisting with solution design for new and existing requirements. Collaborate with the business to understand needs. Design and develop ETL/ELT processes, data lake storage architecture, data warehouse, data marts, cubes, reports and dashboards; in-line with the company data management framework and enterprise data More ❯

Company Details Required Skills & Knowledge: Solid hands-on experience with Microsoft Technology Stack (2022) Extensive experience of SQL/data warehousing Proven experience of data engineering activities such as release management, environment controls, CICD pipeline orchestration. Experience working with Python and Power BI Proven ability of Microsoft BI software development including database/reporting development, full … development cycle Strong data modelling, problem solving, information analysis, attention to detail, flexibility of approach. The ability to deal directly with business functions and interpret their thoughts into successful data & analytics solutions. The ability to design and develop complex BI/MI solutions in line with agreed architectural principles and toolsets. Capable of working closely with … movements, including assisting with solution design for new and existing requirements. Collaborate with the business to understand needs. Design and develop ETL/ELT processes, data lake storage architecture, data warehouse, data marts, cubes, reports and dashboards; in-line with the company data management framework and enterprise data More ❯

Data Engineer Lead - NatGas and Power page is loaded Data Engineer Lead - NatGas and Power Apply locations London posted on Posted 12 Days Ago job requisition id JR102268 Job Title: Data Engineer Lead - NatGas and Power Contract Type: Time Type: Job Description: Reporting to the Head of Data, the Data … ETL/ELT frameworks and orchestration tools (e.g., Airflow, Azure Data Factory, Dagster). Proficient in cloud platforms (preferably Azure) and services such as Data Lake, Synapse, Event Hubs, and Functions. Authoring reports and dashboards with either open source or commercial products (e.g. PowerBI , Plot.ly, matplotlib) Programming OOP DevOps Application development lifecycle Web technologies HTTP … with time-series databases (e.g., InfluxDB, kdb+, TimescaleDB) and real-time data processing. Familiarity with distributed computing and data warehousing technologies (e.g., Spark, Snowflake, Delta Lake). Strong understanding of data governance, master data management, and data quality frameworks. Excellent communication and stakeholder management skills. Ability to mentor junior More ❯

that adventure. Our employees are at the heart of delivering impactful and meaningful work for our clients and helping them to reach for and realise their vision. Snowflake Data Architect Slalom is seeking an experienced Snowflake Data Architect to contribute to its growing Data Capability. The ideal candidate is a professional … with experience in designing, developing, validating and communicating enterprise data solutions using Snowflake. Deep experience in developing enterprise data management strategies including data lake/warehouse implementations, data movement, data services, data acquisition, data conversion, and archive/recovery. What will you do? Lead as …/advise clients on reference architectures Requirements: Expert in capturing end user requirements and aligning technical solutions to the business objectives Experience in leading, designing and implementing Cloud Data strategies, including designing multi-phased implementation roadmaps Significant experience (and ideally certified) in Snowflake and acted as lead data architect on at least two Snowflake implementations. Expertise More ❯

are a brand embarking on a very exciting plan and we are seeking new members of the team who want to contribute to this. THE ROLE As a Data Architect (contract) for Project Nova, you will be instrumental in shaping AllSaints' journey to become a truly data-driven, AI-powered organisation. You will lead the design … and implementation of our enterprise data architecture. This role is central to the Data & Integration workstream of Project Nova, building on our existing data warehouse and BI capabilities to accelerate our data transformation and enable the use of greater AI-driven insight and actions. You will ensure our data assets … ensure a cohesive and integrated technology ecosystem. You will be part of a dynamic team driving the core technology transformation for AllSaints. WHAT WILL I BE DOING? 1. Data Strategy & Architecture Design: Define and evolve AllSaints' enterprise data architecture to support Project Nova's strategic objectives Design and develop conceptual, logical, and physical data More ❯