1 to 25 of 130 Data Lake Jobs in London

build tool) – modelling layers, testing, documentation, incremental strategies Application Databases Data Integration AWS – S3, RDS, Aurora, Redshift (migration context), Glue, Lake Formation, IAM, VPC Data Governance Data lineage, cataloguing (Apache Atlas/Collibra/Snowflake Horizon), GDPR, BCBS … contracts, schema versioning Redis Cache invalidation, persistence strategy, data consistency MongoDB TimescaleDB NATS JetStream AWS S3/Glue Data lake staging, archival, batch ingestion into Snowflake Partitioning, file format (Parquet/ORC), Lake Formation governance Finance Domain Knowledge Candidates should have hands ...

Data & Automation Lead, Data Laking, Data Lake, Data Lakes, AWS, AI, LLM, Automation, Part Remote, Part Central London AI Data & Automation Lead required to work for a fast growing and already established company based in Central London. This … more details Key Responsibilities Lead the design and evolution of a modern data architecture, leveraging cloud-based data lake technologies and scalable data engineering practices. Define and deliver a roadmap for transforming investment operations through automation, workflow modernisation and AI enabled solutions. Build ...

Data & Automation Lead, Data Laking, Data Lake, Data Lakes, AWS, AI, LLM, Automation, Part Remote, Part Central London AI Data & Automation Lead required to work for a fast growing and already established company based in Central London. This … more details... Key Responsibilities Lead the design and evolution of a modern data architecture, leveraging cloud-based data lake technologies and scalable data engineering practices. Define and deliver a roadmap for transforming investment operations through automation, workflow modernisation and AI enabled solutions. Build ...

configure and support Microsoft Fabric solutions for our clients across public and private sector environments. This role will support the delivery of modern data platforms, analytics services and integrated data solutions, helping customers make better use of their data through Microsoft Fabric, Power … best practice. Experience with Azure data services such as Azure Data Factory, Azure Synapse Analytics, Azure SQL, Data Lake Storage or Databricks would be advantageous. Understanding of data governance, security, access control, data quality and data lifecycle ...

warehouse architectures: Implement logical and physical data models, data quality controls, and governance practices using Amazon S3 (data lake), AWS Lake Formation, Amazon Redshift, and Amazon Athena to ensure trusted data.* Ability to design AI/ML-enabled solutions: Enable feature-ready … governed delivery.* Stay ahead of the curve: Continuously learn and adapt to the evolving AWS data and analytics landscape, including AWS Glue, Lake Formation, Redshift, Athena, EMR, and related best practices.* Define the platform operating model: Establish AWS multi-account strategies, environment separation (dev/test/ ...

IaaS, specifically: App Services, Azure SQL Database, SQL Server, Storage, Synapse Analytics, Purview, Databricks, Azure data Factory, Azure data lake & Fabric. Understand which Azure data focused components to use for which purpose.Microsoft certified architectDatabricksDesign and build high-performance data platforms: Utilize … Databricks and Apache Spark to extract, transform, and load data into Azure Data Lake Storage and other Azure services.Ability to Design, Build and deploy AI/ML models: Integrate Machine Learning into data pipelines, leverage Databricks ML and Azure ML to develop predictive ...

business-critical decision-making. Essential to this role are strong python skills, expertise in MS Fabric, SQL, data warehousing, Data Lake and Azure technologies. The Role As the Lead Data Engineer, you will be responsible for designing, developing, and maintaining modern data … development skills. Experience building and optimising ETL/ELT pipelines. Strong understanding of Lakehouse, Data Warehouse, and Data Lake architectures. Experience with Azure Data Factory, Azure Synapse, and Apache Spark. Knowledge of data modelling, semantic models, and enterprise reporting solutions. Experience ...

every report and every operational data feed at ECP, continue the migration of our data warehouse and data lake to a modern dimensional model, finish the rollout of a principle-of-least-privilege security model across a multi-server SQL estate, and continue … personally delivering target architecture for the ECP data estate the data warehouse, data hub, data lake, raw/ODS layers, dimensional model, audit and schema-change tracking, and the load schedules that tie them together. Hands-on T-SQL, stored procedure ...

batch and real-time data pipelines from internal and external data sources. Develop and maintain cloud-native data lake and data warehouse solutions. Ensure data platforms remain scalable, reliable and cost-effective. Data Quality & Governance Implement automated … learning or AI platforms. Knowledge of data governance, metadata management and data lineage. Experience building modern cloud data lake or warehouse architectures. Education Bachelor's degree in Computer Science, Engineering, Information Systems or a related discipline. Master's degree in Big Data ...

batch and real-time data pipelines from internal and external data sources. Develop and maintain cloud-native data lake and data warehouse solutions. Ensure data platforms remain scalable, reliable and cost-effective. Data Quality & Governance Implement automated … learning or AI platforms. Knowledge of data governance, metadata management and data lineage. Experience building modern cloud data lake or warehouse architectures. Education Bachelor's degree in Computer Science, Engineering, Information Systems or a related discipline. Master's degree in Big Data ...

Olympic Committee, and Mercedes-Benz. Our services span the full digital delivery lifecycle, including architecture, engineering, delivery management, user-centred design, business analysis, data, DevOps, and AI. We operate as a collaborative and inclusive organisation that empowers our people to take ownership, innovate, and develop their expertise. … approach, you’re comfortable engaging with clients, understanding their needs, and translating them into effective outcomes. The Role: We are looking for experienced Data Architects to work on our projects with our public sector clients, helping to deliver Solirius’ services to the highest standard. The role will involve ...

deliver meaningful impact beyond commercial outcomes, this is a chance to combine professional excellence with genuine social purpose. More about the team The Data and Platform Engineering team are the foundation for the Data Office function. Responsible for designing, building, and maintaining PfP's data … associated tech stack Strongunderstanding of Data Mesh principles (direct experience beneficial) Technical mentoring/coaching skills Extensiveexperience with Data Lake/Warehouse solutions Strong proficiency in multiple languages with SQL and Python as must haves In-depth knowledge of queryoptimization techniques and experience in fine ...

Cloud Data Engineer About the Job you are considering: We are seeking an experienced Cloud Data Engineer with expertise in AWS, Snowflake, dbt, PySpark, and SQL to design, build, and maintain scalable cloud-based data platforms and analytics solutions. The ideal candidate should have … Manager Data Modelling Skills Star Schema Snowflake Schema Dimensional Modeling Data Vault Data Warehouse Design Data Lake Architecture Data Formats Experience working with: Parquet Avro JSON CSV ORC CI/CD & DevOps Git GitHub GitLab Bitbucket Jenkins GitHub Actions ...

time data pipelines that ingest data from legacy and modern platforms Lead the development of cloud-based data lake architectures within AWS Build streaming data solutions using Kafka Design and implement Medallion Architecture (Bronze, Silver, Gold layers) to support analytics … Essential Experience Strong hands-on expertise with Databricks Extensive experience using Spark, PySpark and SQL Proven track record designing and implementing data lake architectures Experience building both batch and streaming data pipelines Strong knowledge of Kafka Good understanding of AWS data services, particularly ...

transformation, and integration processes. Infrastructure Management: Create and maintain data models, tables, and supporting infrastructure within Databricks and associated data lake/lakehouse environments. Business Logic Development: Implement business logic, calculations, and data processing solutions using Python. Collaboration: Work closely with business stakeholders … analytical and problem-solving skills. Excellent stakeholder management and communication skills. Highly Desirable: Experience with Spark/PySpark. Familiarity with Azure Data Lake, Snowflake, or similar platforms. Knowledge of orchestration tools like Airflow or Azure Data Factory. Exposure to financial services environments. Experience supporting large ...

Senior Databricks Architect Technology: Databricks (Spark, PySpark/SQL) and data lake architectures. Location: London, UK. Compensation: Competitive (including bonus). Job Description We are looking for a Data Architect to design, build, and operate scalable data pipelines and data products … support analytics, reporting, and domain-owned data products. Primary Skills Strong experience with Databricks (Spark, PySpark/SQL) and data lake architectures. Hands‐on experience with streaming technologies, especially Kafka/Confluent Kafka. Experience integrating legacy data sources such as DB2 (mainframe ...

seeking an experienced Enterprise Data Architect to join a major transformation programme within a complex enterprise environment. You will be responsible for defining and delivering the organisation’s data architecture strategy, enabling improved decision-making, operational efficiency, and governance across key business functions including Finance, Procurement … Property, and Asset Management. Working closely with Enterprise Architecture and Data & AI teams, you will design scalable data models, integration patterns, and governance frameworks that support the full asset lifecycle and future business growth. Key Responsibilities Design end-to-end data architecture across ...

pharmaceuticals and life sciences, manufacturing, and technology, media and telecoms, with capabilities spanning strategy, transformation and operational excellence – all powered by advanced technology, data, AI and digital innovation. Clients value Baringa’s collaborative approach and the way its teams integrate seamlessly – all working with a shared understanding … . What you will be doing Defining and implementing on premise or cloud architectures, (e.g. a cloud data warehouse, data lake or data platform) to enable digital transformation Working with clients on data warehousing, building operational ETL/ELT data ...

Join us and build for travelers everywhere. Expedia Group seeks a motivated and results-oriented Senior Software Engineer to join our Data Lake and Catalog team. As a Senior Software Engineer on an industry-leading data platform, you will build the core of Expedia Group … with tools like Circus Train, a Hive dataset replication tool, and Waggle Dance, a federation service that allows access to data lake tables across multiple catalogs. Recently, we added Hive support to Apache Iceberg, a high-performance open table format for large analytic datasets that underpins many ...

Join us and build for travelers everywhere. the company seeks a motivated and results-oriented Senior Software Engineer to join our Data Lake and Catalog team. As a Senior Software Engineer on an industry-leading data platform, you will build the core of the company … with tools like Circus Train , a Hive dataset replication tool, and Waggle Dance , a federation service that allows access to data lake tables across multiple catalogs. Recently, we added Hive support to Apache Iceberg , a high-performance open table format for large analytic datasets that underpins many ...

Architect - London (Hybrid) - Databricks - Spark - PySpark - AWS - Up to £115,000 + Bonus Our client, a large-scale organisation undergoing significant investment in data and cloud technologies, is seeking an experienced Senior Databricks Architect to join its expanding Data & AI practice. This is an exciting opportunity … Data Engineer or Solutions Architect Strong expertise in Databricks, Apache Spark, PySpark and SQL Experience designing and implementing modern data lake or lakehouse architectures Hands-on experience with Kafka Strong understanding of AWS data services, particularly Amazon S3 Experience integrating data ...

Title: HR Data Delivery Lead Role Overview A new data lake solution is being implemented and the HR Data Delivery Lead will be responsible for end-to-end delivery of data products, pipelines, and reporting solutions across multiple teams. This role … will be accountable for connecting data from the new Oracle HCM Cloud to the new AWS Snowflake solution and unifying data layer between legacy and new HRIS system feed to data warehouse. The role ensures high-quality, timely, and governed delivery of data ...

Senior Data Engineer Market Data 5 day onsite We are looking for a Senior Data Engineer to build and operate a high-performance market data platform supporting quantitative research and production trading systems. You'll design scalable data pipelines, process … actions. Implement automated data quality, reconciliation and monitoring frameworks. Optimise data storage using Parquet, Arrow and modern data lake technologies. Partner with quantitative researchers, traders and engineering teams to deliver trusted datasets. Essential Skills Strong Python and SQL. Experience building scalable, production data ...

Required Skills Data Architect Azure London Job Summary Data Architect – Insurance (Azure Data Platform) We're working with a leading organisation investing heavily in its data and analytics capabilities, and they're looking for an experienced Data Architect with … models, integration frameworks and enterprise data standards Lead the architecture of modern Azure-based data platforms, including Data Lake, Lakehouse and Data Warehouse environments Drive the design of scalable data pipelines, ETL/ELT frameworks and integration patterns Establish ...

data warehouse solutions with Azure Data Factory (ADF) and Azure Databricks, services related to Azure Analytics, Azure data lake storage Gen2, Azure SQL/Postgres, Azure analytics services and good understanding of Azure identity (SPN, SAMI, UAMI etc.) experience in designing large data distribution, integration with service-oriented architecture, data mesh, data Lake and data analytics solution using Azure data services with large and multi-format data proficient coding experience using Spark (Python/Scala) and SQL hands on experience ...