1 to 25 of 105 Permanent Data Lake Jobs in London

About Us: ATREIDES is a leader in advanced geospatial technologies, big data solutions, and data analytics. We specialize in providing cutting-edge software platforms that enable our clients to process, analyze, and visualize large-scale geospatial and sensor data for critical decision-making. Our mission is to deliver innovative, scalable, and high-performance platforms … that help organizations unlock the full potential of their data for various industries, including defense, national security, environmental monitoring, and urban planning. We are seeking a highly skilled Senior Data Engineer with a strong focus on big data, data analytics, and geospatial data intelligence to join our engineering team. In … this role, you will help design and develop robust, scalable platforms that handle vast amounts of geospatial data and provide actionable insights through data analytics. Job Overview: As a Senior Data Engineer at ATREIDES, you will be responsible for developing and optimizing our core software platform, with an emphasis on integrating and processing big More ❯

that solves real-world problems for large blue-chip companies and governments worldwide. We're pioneers of meaningful AI : our solutions go far beyond chatbots. We are using data and AI to solve the world's biggest issues in telecommunications, sustainable water management, energy, healthcare, climate change, smart cities, and other areas that have a real impact on … re underpinned by over 300 engineers and scientists working across Africa, Europe, the UK and the US. ABOUT THE ROLE Sand Technologies focuses on cutting-edge cloud-based data projects, leveraging tools such as Databricks, DBT, Docker, Python, SQL, and PySpark to name a few. We work across a variety of data architectures such as Data Mesh, lakehouse, data vault and data warehouses. Our data engineers create pipelines that support our data scientists and power our front-end applications. This means we do data-intensive work for both OLTP and OLAP use cases. Our environments are primarily cloud-native spanning AWS, Azure and GCP More ❯

that solves real-world problems for large blue-chip companies and governments worldwide. We're pioneers of meaningful AI : our solutions go far beyond chatbots. We are using data and AI to solve the world's biggest issues in telecommunications, sustainable water management, energy, healthcare, climate change, smart cities, and other areas that have a real impact on … re underpinned by over 300 engineers and scientists working across Africa, Europe, the UK and the US. ABOUT THE ROLE Sand Technologies focuses on cutting-edge cloud-based data projects, leveraging tools such as Databricks, DBT, Docker, Python, SQL, and PySpark to name a few. We work across a variety of data architectures such as Data Mesh, lakehouse, data vault and data warehouses. Our data engineers create pipelines that support our data scientists and power our front-end applications. This means we do data-intensive work for both OLTP and OLAP use cases. Our environments are primarily cloud-native spanning AWS, Azure and GCP More ❯

support EQ's transition into a stronger, more profitable, product-led business, driven by real insights and the needs of its customers. We are looking for a skilled Data Engineer to join our Data team. As a Data Engineer, you will be responsible for designing, building, and maintaining large-scale data pipelines … cloud-based data pipelines and architectures Strong expertise in Microsoft Fabric and Databricks, including data pipeline development, data warehousing, and data lake management Proficiency in Python, SQL, Scala, or Java Experience with data processing frameworks such as Apache Spark, Apache Beam, or Azure Data Factory Strong understanding … Azure and or AWS Strong collaboration and communication skills, with the ability to work effectively with cross-functional teams Desirable: Experience with Azure Synapse Analytics, Azure Data Lake Storage, or other Azure data services Experience with agile development methodologies and version control systems such as Git Certification in Microsoft Azure, Databricks, or other relevant technologies More ❯

mission dedicated to driving positive change. Read on to learn more about the opportunity and how you can have a positive impact! About the Role The Insight and Data team is a recently formed team focusing on analytics, data extraction, transformation, and load, as well as governance and stewardship. The team is primarily responsible for enabling … is quickly growing. Key Responsibilities: Oversee a team of 2 ETL Developers to ensure all source systems in EcoOnline's landscape are properly ingested into the data lake hosted within AWS S3. Own and develop all data pipelines and infrastructure using a combination of Talend Data Studio, Stitch Data Replication, and … SQL. Own development of automation to enable near real time data availability within datasets and data lake (AWS Redshift and AWS S3). Own initial development of datasets from the data lake to enable surfacing topical data with light normalization to end users. Ensure methodology used is compliant with best More ❯

individual without security clearance is welcome to apply and will always be considered for this or wider opportunities in CGI, where appropriate. We're looking to grow our Data Office team, working directly with the client to ensure that all aspects of their Data Management strategy are delivered. Your future duties and responsibilities As a Technical … BA you will be responsible for: Data Modelling Design and develop Conceptual, Logical, and Physical Data Models to support robust data architectures. Produce high-quality documentation including data dictionaries, database designs, and data flow diagrams. Work with Ontologies and Knowledge Graphs to enhance semantic understanding of data. Utilise modelling tools … such as Visio, Sparx Enterprise Architect, or Draw.IO. Apply modelling languages such as UML or Express-G to effectively communicate data structures. Data Standards & Governance Author and maintain detailed technical specifications and documentation. Lead efforts to assure systems and data models against internal and industry standards. Develop, promote, and govern data standards More ❯

Data Engineer/Scientist Department: Consultancy Employment Type: Full Time Location: United Kingdom/Hybrid Description As a Data Engineer/Scientist at Actica, you will have the opportunity to design, build, and maintain data pipelines while developing advanced analytics solutions to unlock business problems for high-profile UK public sector organisations. Your expertise … will enable organisations to maximise the value of their data assets through robust data infrastructure and sophisticated analysis, playing a key role in nationally critical projects that make a real difference to people's everyday lives. Locations: London, Guildford, Bristol, M4 corridor Hybrid working Roles and Responsibilities Actica recognises that data engineering, analytics, and … data science are distinct but interconnected disciplines. Our data engineers focus on building and maintaining the data infrastructure that enables analytics and data science work, while our data scientists and analysts focus on deriving insights and developing models. At Actica, we're at the forefront of the UK government's More ❯

Company Details Required Skills & Knowledge: Solid hands-on experience with Microsoft Technology Stack (2022) Extensive experience of SQL/data warehousing Proven experience of data engineering activities such as release management, environment controls, CICD pipeline orchestration. Experience working with Python and Power BI Proven ability of Microsoft BI software development including database/reporting development, full … development cycle Strong data modelling, problem solving, information analysis, attention to detail, flexibility of approach. The ability to deal directly with business functions and interpret their thoughts into successful data & analytics solutions. The ability to design and develop complex BI/MI solutions in line with agreed architectural principles and toolsets. Capable of working closely with … movements, including assisting with solution design for new and existing requirements. Collaborate with the business to understand needs. Design and develop ETL/ELT processes, data lake storage architecture, data warehouse, data marts, cubes, reports and dashboards; in-line with the company data management framework and enterprise data More ❯

people too - creating new challenges and opportunities all the time. We're a business that never stands still. Join us on our journey to as a Head of Data Engineering & Insight, and you won't either. As part of our accelerated digital transformation and the implementation of our new Next Generation BI platform, we are looking for a … years' experience in Business Intelligence, with 5+ years in a BI leadership role in a global or matrixed organisation . Proven expertise in modern BI architecture (Data Lake, EDW, Streaming, APIs, Real-Time & Batch Processing). Demonstrated experience delivering cloud-based analytics platforms (Azure, AWS, GCP). Strong knowledge of data governance, cataloguing, security, automation … Engineering & Insight will work within a modern, cloud-based BI ecosystem , including: Data Integration: Fivetran , HVR, Databricks , Apache Kafka, Google BigQuery , Google Analytics 4 Data Lake & Storage: Databricks Delta Lake, Amazon S3 Data Transformation: dbt Cloud Data Warehouse: Snowflake Analytics & Reporting: Power BI, Excel, Snowflake SQL REST API Advanced Analytics More ❯

people too - creating new challenges and opportunities all the time. We're a business that never stands still. Join us on our journey to as a Head of Data Engineering & Insight, and you won't either. As part of our accelerated digital transformation and the implementation of our new Next Generation BI platform, we are looking for a … years' experience in Business Intelligence, with 5+ years in a BI leadership role in a global or matrixed organisation . Proven expertise in modern BI architecture (Data Lake, EDW, Streaming, APIs, Real-Time & Batch Processing). Demonstrated experience delivering cloud-based analytics platforms (Azure, AWS, GCP). Strong knowledge of data governance, cataloguing, security, automation … Engineering & Insight will work within a modern, cloud-based BI ecosystem , including: Data Integration: Fivetran , HVR, Databricks , Apache Kafka, Google BigQuery , Google Analytics 4 Data Lake & Storage: Databricks Delta Lake, Amazon S3 Data Transformation: dbt Cloud Data Warehouse: Snowflake Analytics & Reporting: Power BI, Excel, Snowflake SQL REST API Advanced Analytics More ❯

thefoundation for delivering our full suite of capabilities. Job Description Employment Type: Full-Time Location: UK Job Overview We are looking for a skilled Database Designer/Senior Data Engineer to join our data engineering team. This role involves the development and maintenance of data solutions in the cloud environment. The ideal candidate will … have a strong background in data processing and a solid understanding of cloud data services. Requirements Key Responsibilities Design and implement modern data platforms in cloud such as AWS or Azure. Develop data pipelines and data streaming processes. Ensure data solutions are scalable, reliable, and efficient. Collaborate with … services. Proficiency in SQL, Python, or Scala. Experience with AWS RDS, AWS Glue, AWS Kinesis, AWS S3, Redis and/or Azure SQL Database, Azure Data Lake Storage , Azure Data Factory. Knowledge of data modelling , ETL processes, and data warehousing principles. Preferred Skills Strong problem-solving and analytical skills . More ❯

Data Engineer I, Business Data Technologies Business Data Technologies (BDT) makes it easier for teams across Amazon to produce, store, catalog, secure, move, and analyze data at massive scale. Our managed solutions combine standard AWS tooling, open-source products, and custom services to free teams from worrying about the complexities of operating … lets BDT customers move beyond the engineering and operational burden associated with managing and scaling platforms, and instead focus on scaling the value they can glean from their data, both for their customers and their teams. We own the one of the biggest (largest) data lakes for Amazon where 1000's of Amazon teams can search … share, and store EB (Exabytes) of data in a secure and seamless way; using our solutions, teams around the world can schedule/process millions of workloads on a daily basis. We provide enterprise solutions that focus on compliance, security, integrity, and cost efficiency of operating and managing EBs of Amazon data. Key job responsibilities CORE RESPONSIBILITIES: Be More ❯

Senior Delivery Consultant -Data Analytics & GenAI, AWS Professional Services Public Sector Job ID: Amazon Web Services Australia Pty Ltd Are you a Senior Data Analytics … and GenAI consulting specialist? Do you have real-time Data Analytics, Data Warehousing, Big Data, Modern Data Strategy, Data Lake, Data Engineering and GenAI experience? Do you have senior stakeholder engagement experience to support pre-sales and deliver consulting engagements? Do you like to solve the most … learn and use services such as AWS Glue, Amazon S3, Amazon DynamoDB, Amazon Relational Database Service (RDS), Amazon Elastic Map Reduce (EMR), Amazon Kinesis, Amazon Redshift, Amazon Athena, AWS Lake Formation, Amazon DataZone, Amazon SageMaker, Amazon Quicksight and Amazon Bedrock. Solutions: Support pre-sales and deliver technical engagements with partners and customers. This includes participating in pre-sales visits More ❯

look to our Business Consultants, to drive change, inspire trust with clients, and take responsibility for some of the world's most ambitious transformation projects. As a Senior Data Technology Consultant within our successful Business Consulting practice, you, alongside a collaborative team of consulting colleagues, will take a central role in providing support to our clients with a … focus on driving their business outcomes using data and technology. You will also support the growth of the wider evolving business consulting function. A key focus of the role will be to deliver practical advice to clients relating to their enterprise data technologies and strategies whilst looking out for opportunities to support them further leveraging our … quality consulting services to clients, presenting technical concepts in simple, accessible language. • Leverage your deep understanding of legacy and cloud-based data technologies, including Data Lake, Data Warehouse, and Lakehouse concepts. • Bring broad technical knowledge of Azure data products, with subject matter expertise in areas like Azure Data Factory More ❯

and much more. Innovation: We develop and execute groundbreaking products and techniques that shape industry norms. These innovations enhance how audiences experience sports, entertainment, and news. Disney's Data Product Management team guides the strategy and vision for Disney's incredible global digital products portfolio - from Disney+, Hulu, ESPN+ and the ESPN App to Marvel Unlimited, National Geographic … technologies ranging from Machine Learning to Extended Reality. We are seeking a Product Manager to join our team, and this role will be pivotal in building and maintaining data products that are responsible for landing, processing, and aggregating data related to user sessions and media playback. The ideal candidate will have a strong background in data management, product development, and a keen understanding of data governance and privacy. Responsibilities Oversee the vision, strategy, and technical roadmap for Disney's Media and Session Data and the associated tooling supporting Disney Entertainment and ESPN Product & Technology's digital properties. Build strategic relationships with key business, analytics, and technology leaders to ensure program More ❯

Overview We're seeking a Senior Data Scientist to lead the development of advanced analytics and AI/ML solutions that unlock real value across our business. This is a contract role for 6 months . In this contract role, you'll work with proprietary and B2B research datasets to design, deliver, and scale data-driven … Research, and Technology, you'll help turn strategic ideas into working MVPs-ensuring high standards of methodology, quality, and business relevance throughout. You'll also help shape the data science environment by working alongside our tech teams to support a robust and flexible infrastructure, including sandbox environments for onboarding and evaluating new data sources. This is … analytics "products" could be refined and productionized in the future. Work with Tech colleagues to improve the Data Science workspace, including providing requirements for Data Lake, Data Pipeline, and Data Engineering teams. Expand on the tools and techniques already developed. Help us understand our customers (both internal and external) better so More ❯

Data Architect (18-Month Fixed Term Contract) Salary: £65,000 - £72,000 per annum + benefits Location: London/Hybrid Our client is underway with an exciting Transformation Programme and is looking to recruit an experienced Data Architect. The role is critical in helping to shape their future as a data-led organisation.The Data … will be instrumental in building the foundational architecture, governance and strategy needed to bring that vision to life. Key Responsibilities Include Defining and owning the enterprise and commercial data strategy aligned with the digital transformation goals Establishing data governance policies, metadata standards, data stewardship roles, and classification models Designing scalable data architecture … strategy experience Expertise in data governance, data modelling, metadata management Strong understanding of Azure or Data Factory, Synapse & Data Lake Ability to translate complex data concepts into business-aligned strategies and actionable outcomes Hays Specialist Recruitment Limited acts as an employment agency for permanent recruitment and employment More ❯

Senior Data Architect Joining Capco means joining an organisation that is committed to an inclusive working environment where you're encouraged to Be Yourself At Work. We celebrate individuality and recognize that diversity and inclusion, in all forms, is critical to success. It's important to us that we recruit and develop as diverse a range of talent … Focused on maintaining our nimble, agile and entrepreneurial culture. Your Capco Day/Key Responsibilities You will accompany and lead complex agile consulting projects supporting our clients with data architecture design and delivery. You will work with technology leaders across Financial Services to provide best practice guidance for data management and data architecture development … . Knowledge of the relevant procedures, architectures and technologies in one or more of the following topics: Contextual Datamodelling, Entity Relationship Modelling, Logical & Physical Datamodelling and Data Lake design. Experience designing and implementing cloud data migration and storage patterns on one or more of AWS, GCP and Microsoft Azure Experience implementing and integrating data More ❯

Data Architect (18-Month Fixed Term Contract) Salary: £65,000 - £72,000 per annum + benefits Location: London/Hybrid Our client is underway with an exciting Transformation Programme and is looking to recruit an experienced Data Architect. The role is critical in helping to shape their future as a data-led organisation. The … Data Architect will be instrumental in building the foundational architecture, governance and strategy needed to bring that vision to life. Key Responsibilities Include Defining and owning the enterprise and commercial data strategy aligned with the digital transformation goals Establishing data governance policies, metadata standards, data stewardship roles, and classification models Designing scalable … strategy experience Expertise in data governance, data modelling, metadata management Strong understanding of Azure or Data Factory, Synapse & Data Lake Ability to translate complex data concepts into business-aligned strategies and actionable outcomes Hays Specialist Recruitment Limited acts as an employment agency for permanent recruitment and employment More ❯

Head of Data & Analytics Architecture and AI page is loaded Head of Data & Analytics Architecture and AI Apply locations Chiswick Park time type Full time posted on Posted 30+ Days Ago job requisition id JR19765 Want to help us bring happiness to the world? Want to join an ambitious and fast growing global biscuits and confectionery … Amazon Web Services): Knowledge of services like S3, EC2, Lambda, RDS, Redshift, EMR, SageMaker, Glue, and Kinesis. Azure: Proficiency in services like Azure Blob Storage, Azure Data Lake, VMs, Azure Functions, Azure SQL Database, HDInsight, and Azure Machine Learning Studio. Data Storage & Databases: SQL & NoSQL Databases: Experience with databases like PostgreSQL, MySQL, MongoDB, and Cassandra. … API Development: RESTful API design and GraphQL. Microservices Tools: Istio, Envoy, and Linkerd. Security & Governance: Identity & Access Management: AWS IAM, Azure Active Directory. Data Governance Tools: AWS Lake Formation, Azure Purview. Data Security Tools: AWS Key Management Service (KMS), Azure Key Vault. Data Analytics & BI: Visualization Tools: Tableau, Power BI, Looker, and Grafana. More ❯

The role We are looking for a Lead Data Engineer to modernise our Corporate - Player Sustainability, Finance, Legal, and Regulatory data infrastructure, focusing on cloud technologies, next-generation data warehousing and Self-Serve Analytics This role required expertise in working with data warehouse platforms, managing data pipelines and deliver end … to end data engineering project. You will work closely with cross-function teams, designing and implementing scalable cloud solutions that seamlessly integrate and optimise business operations. What you will do Design, develop, test, and support data-driven solutions while providing technical and design expertise. Lead and contribute to technical planning, requirements gathering, and the delivery of … any of your compliance obligations or the requirements are unclear. Your experience Strong data engineering background, with experience in designing and building AWS cloud data lake, data pipelines, data warehouse and self-service solutions Experience delivering large-scale transformation projects with a focus on end-to-end data architecture More ❯

Senior Azure Data Engineer Remote Working - UK Home-based with occasional travel into the office £39,784 - £49,477 (National) £45,456- £55,149 (London within the M25) Additional allowance for exceptional candidates - Max Salary - £62,000 (National) and £67,000 (London within the M25) Homeworking allowance of £581 per annum Job Ref: J12964 A Senior Azure Data Engineer is required to design, build, test and maintain data on the enterprise data platform, allowing accessibility of the data that meets business and end user needs. The successful individual will be responsible for maximising the automations, scalability, reliability and security of data services, focusing on opportunities for re-use … adaptation and efficient engineering. Additionally you will support the team but knowledge sharing and mentoring more junior members of the team. Accountabilities: ·Design, build and test data pipelines and services, based on feeds from multiple systems using a range of different storage technologies and/or access methods provided by the Enterprise Data Platform, with a More ❯

Location: London (hybrid working 3 office days per week) Employment Type: Permanent, full time Are you passionate about data and eager to make a significant impact? The AA is a well-loved brand with a range of driver services much wider than most people realise. We have an enviable set of data assets from breakdown, service … the full potential of our data. As a Data Engineer, you'll be crucial in developing and maintaining our data pipelines and data lake house architecture. You'll be central to the success of our data team's efforts to maximise our data capabilities. This is an exciting time … quality, readable and maintainable code. Designing, building and maintaining resilient data pipelines Identifying and automating manual or repetitive data processes. Establishing the data lake house as a single source of insight for the business. Developing analytical datasets to support insights into customer behaviour, operational efficiency and key performance metrics. Triaging and resolving data More ❯

a positive contribution to the world of banking and society. THE ROLE We are looking for a visionary and results-driven Head ofData & Analytics to lead our global data strategy, governance, andanalytics initiatives within the Corporate Global IT organization. This role iscritical in transforming data into actionable insights that drive businessperformance, innovation, and competitive advantage. You … into the core of our operations. Additionally, youwill oversee the ownership and evolution of enterprise budgeting andforecasting software, ensuring it supports strategic financial planning andintegrates seamlessly with broader data and analytics frameworks. OPPORTUNITIES You will Data Strategy & Leadership: Define and execute the enterprise-wide data and analytics strategy aligned with global business goals, offering … architecture, data warehousing, and modern analytics platforms (e.g., Snowflake, Power BI, Tableau, Databricks). Hands-on experience with Microsoft Data Factory, Azure Data Lake, and Microsoft Fabric. Strong knowledge of data governance, privacy regulations (e.g., GDPR), and data lifecycle management. Excellent communication and stakeholder management skills. Experience in change More ❯

As a Senior Data Engineer specializing in Natural Gas and Power, you will support the design, development, and optimization of advanced data infrastructure and products that support critical trading, risk, and market analytics functions. You will collaborate closely with traders, quantitative analysts, risk managers, and IT stakeholders to ensure data systems are scalable, secure … ETL/ELT frameworks and orchestration tools (e.g., Airflow, Azure Data Factory, Dagster). Proficient in cloud platforms (preferably Azure) and services such as Data Lake, Synapse, Event Hubs, and Functions. Authoring reports and dashboards with either open source or commercial products (e.g. PowerBI, Plot.ly, matplotlib) Programming OOP DevOps Web technologies HTTP/S REST … with time-series databases (e.g., InfluxDB, kdb+, TimescaleDB) and real-time data processing. Familiarity with distributed computing and data warehousing technologies (e.g., Spark, Snowflake, Delta Lake). Strong understanding of data governance, master data management, and data quality frameworks. Solid grasp of web technologies and APIs (REST, JSON, XML More ❯