1 to 25 of 113 Data Lake Jobs in London

business in the coming years. We’re looking for an engineer to help us build BPP’s platforms, products and user experiences. The Data Engineering team will enable BPP to leverage data, to inform and optimise how we deliver scalable, secure and performant experiences that delight … pipelines to extract data from various source systems (log files, APIs, databases) and load them into our data lake/data warehouse confidentially using modern data tools and technologies Transform data into highly accessible and usable structures … as AWS, GCP, Azure) Good understanding of data modelling methods and data partitioning and compaction methods in Data Lake Experience of working with event driven architectures or real time streaming analytics Confident in writing and reading code in multiple languages Possess a deep more »

for designing, developing, and maintaining data pipelines and systems in Azure Cloud environments, with a strong emphasis on Azure Data Lake, Azure Data Factory, and Microservices architecture. Key Responsibilities: Develop and maintain data pipelines in Azure Cloud environments, leveraging Azure Data Lake, Azure Data Factory, and Microservices architecture. Design and implement scalable and efficient data processing solutions using technologies such as ServiceBus, REST API, App Service, SQL Data Warehouse, and Data Mart. Utilize Vector Databases, Cosmos DB, Redis, and Elasticsearch … engineering. Bachelor’s Degree or higher in Computer Science or a related field. Proficiency in Azure Cloud environments, Azure Data Lake, Azure Data Factory, Microservices architecture. Experience with Vector Databases, Cosmos DB, Redis, Elasticsearch. Strong programming skills in Python, Spark, Databricks, Pyspark, SQL more »

Are you ready to take your data engineering expertise to new heights? This is your chance to become an integral part of a pioneering construction leader that's shaping the future! With a rich history, my client is an acclaimed construction powerhouse, renowned for iconic projects that transcend … data solutions aligned with business requirements. Design and fine-tune data storage and retrieval systems using Azure Data Lake Storage, Azure SQL Database, and other Azure DB services. Develop, optimize, and maintain data pipelines for automated data integration from … and/or BI Developer. Proficiency in Azure data tools such as Data Factory, Synapse, Databricks, SQL, Data Lake or Delta Lake. Strong data engineering and data modeling skills, with the ability to craft complex data models. more »

Retail and CPG, and Public Services. Consolidated revenues as of $13 billion. Job Description - Job Description- We are seeking a talented and experienced Data Architect to join our team. The ideal candidate will have a strong background in designing and implementing data solutions in an AWS … DevOps microservices environment using Kafka and Snowflake. As a Data Architect, you will play a crucial role in defining and implementing the data architecture, data modeling, and data integration solutions for our organization. Responsibilities : Design and implement scalable and efficient data architecture solutions in an AWS environment, leveraging microservices architecture and DevOps principles. Work closely with development teams to understand data requirements and design appropriate data models and schemas for various applications and services. Implement and maintain data pipelines using Kafka for real more »

This is an exciting opportunity for a Data Architect to join a growing Data & AI team. The successful applicant will work alongside architects and consultants to deliver high-quality data solutions to help customers become more successful in becoming a data-guided … platforms and data solutions using a wide range of Azure Services including Azure Databricks, Azure SQL Database, Azure Data Lake Storage, Azure Synapse, Azure Cosmos DB, Azure Data Factory and Azure Blob storage Building conceptual, logical and physical data models … in Data Warehouse methodologies Experience working across Azure Data Services including Azure Databricks, Azure SQL Database, Azure Data Lake Storage, Azure Cosmos DB, Azure Data Factory, Azure Blob storage and Power BI Strong experience in SQL Good understanding of Azure infrastructure more »

brand licensing through IMG; live event experiences and hospitality through On Location; full-service marketing through global cultural marketing agency 160over90; and sports data and technology through IMG ARENA and OpenBet. Endeavor is also the majority owner of TKO Group Holdings (NYSE: TKO), a premium sports and entertainment … company comprising UFC and WWE. What You’ll Do: The Data Engineer role sits within the Engineering team of the Data Services department responsible for all data applications across the company’s divisions globally from Big Data, Data Warehousing, Business … business hours and weekends when required You Have These: Exposure to, and ideally experience with, modern data architectures (e.g. data lake, lake house, data mesh) and accompanying technologies (e.g. Azure Synapse, Snowflake, Amazon Redshift) Awareness of data governance and all more »

Job Title - Data Engineer Entity - Aventum Reports to - Head of Data Location - London/Hybrid Who We Are At The Aventum Group, we’re not one for following the usual rules. We work hard at being different and that very much extends to how we look … to advance their careers at Aventum. Role Summary You will work in various settings to build systems that collect, manage, and convert raw data into usable information for data scientists and business analysts to interpret. Your ultimate goal is to make data accessible so … Fabric, Profisee), Snowflake Data Integration, Azure Service Bus, Deltalake, BigQuery, Azure DevOps, Azure Monitor, Azure Data Factory, SQL Server, Azure DataLake Storage, Azure App Service, Apache Airflow, Apache Iceberg, Apache Spark, Apache Hudi, Apache Kafka, Power BI, BigQuery, Azure ML is a plus Experience with Azure more »

Data Architect (SC Cleared/Banking), London (Hybrid) 6 Month Contract £(Apply online only)/day (Outside IR35) Data Architect needed with active SC Security Clearance , Banking domain experience , data transformation and COTS products experience on Data modernization and Transformation Programs. … work with a leading digital transformation business on large-scale IT modernisation programmes for Government clients. Key skills, experience + tasks will include: Data Architect with strong experience with COTS data flows, product architecture, and integration with data sources. Strong Banking/Financial Services … domain experience. Working Data Modernization and Transformation Programs including Data Cleansing and Migration. Good understanding of security measures and regulations in the Banking and Financial industry. Experience with COTS products and solutions for Data Integration and management. Evaluate and recommend COTS products and solutions more »

Location : London, Must have SC Clearance Certificate Mode: Hybrid Salary: Competitive Overview: We are seeking a talented Data Engineer with expertise in Azure to join our team. The ideal candidate will be responsible for developing data pipelines, building and maintaining data warehouses and data lakes, implementing data integration and transformation solutions, ensuring data quality and integrity, and optimizing data performance. Responsibilities: Develop data pipelines for ETL/ELT (Extract, Transform, Load) processes using Azure technologies. Build and maintain scalable and efficient data … and maintaining data warehouses and data lakes on Azure, utilizing services such as Azure Synapse Analytics, Azure Data Lake Storage, etc. Expertise in implementing data integration and transformation solutions using Azure services like Azure Data Factory, Azure Logic Apps more »

brand licensing through IMG; live event experiences and hospitality through On Location; full-service marketing through global cultural marketing agency 160over90; and sports data and technology through IMG ARENA and OpenBet. Endeavor is also the majority owner of TKO Group Holdings (NYSE: TKO), a premium sports and entertainment … company comprising UFC and WWE. What You’ll Do: The Lead Data Engineer role sits within the Data Services department, you will work actively to unlock the value of data by building and enabling product teams and data practitioners to use the … and needs assessment. You Have These: Team management experience Exposure to, and ideally experience with, modern data architectures (e.g. data lake, lake house, data mesh) and accompanying technologies (e.g. Azure Synapse, Snowflake, Amazon Redshift) Awareness of data governance and all more »

Senior Data Engineer - 6 month FTC London (Hybrid Role) £85,000 per annum Senior Data Engineer Opportunity An exciting opportunity has arisen for a Senior Data Engineer to join a dynamic team dedicated to shaping effortless, digital-first customer experiences. This role is pivotal … in driving value for customers through modelling, sourcing, and data transformation. The Senior Data Engineer will collaborate closely with core technology and architecture teams, delivering strategic data solutions while promoting Agile and DevOps methodologies in data engineering delivery. Responsibilities: Automate data … mentoring abilities and good critical thinking with problem-solving skills. Nice to Have: Experience with relevant technologies such as Synapse, Azure Data Lake, Azure Data Factory, and Databricks. Why Join: This is an opportunity to join a collaborative environment that encourages innovation and values each more »

As the Data Architect, you'll lead data architecture initiatives, establishing standards and practices essential for effective data management. Your focus will be on designing and implementing robust data and information systems, ensuring they meet stringent criteria for modelling, flow, lineage, quality … and security. Working closely with stakeholders and data governance teams, you'll drive adherence to established standards, facilitating data-driven decision-making processes across the organisation. This role encompasses: Developing and maintaining a comprehensive Data Architecture Framework and maturity roadmap. Enforcing best data … Azure stacks, including data pipelines, security and networking Expertise with data warehousing, data lakes and data lake houses Experience with master data management software and technologies Experience with ETL technologies including SQL Server, SSIS, Azure Data Factory more »

Job Title: AWS Data Architect Location: United Kingdom (U.K.) - Hybrid The Company: At Amber Labs, we are a cutting-edge UK and European technology consultancy dedicated to delivering exceptional value to our clients in the public sector. Our company culture promotes collaboration, autonomy, and rapid learning, empowering our … work environment focused on deliverable outcomes, along with a range of benefits to support our employees’ well-being and professional development. Key Responsibilities: Data Architecture Design: Collaborate with stakeholders to understand data requirements and design scalable and secure data architectures on AWS. Develop data models, data flow diagrams, and architectural documentation to support financial applications. Cloud Data Management: Design, implement, and manage cloud-based data solutions on AWS, including data lakes, data warehouses, and analytics platforms. Ensure the scalability, performance, and cost more »

A Data Platform Product Manager required by this leading research and analytics vendor to join an existing Data engineering/product function as they look to continually improve their product and application suite. You will be required to build strong relationships across the client and use … a strong commercial mindset and data expertise to define and execute the roadmap and backlog for the data engineering and products. Experience should include: Significant experience 3 years+ as a data business partner/manager/product owner/data analyst or … technical business analyst in a data platform domain Be an experienced subject matter expert within the data platform domain, with strong technical knowledge and demonstrable passion, including understanding on (some of): Data lakes, data marts, data warehousing (Azure Data more »

unparalleled educational experience. By attracting and retaining the finest educators worldwide, our client ensures an environment where excellence thrives. Job Purpose As the Data Platform Engineer overseeing the Data Warehouse and analytics infrastructure, you will shape the backbone of our client's business analysis and reporting … capabilities, both now and into the future. They are seeking a visionary with a proven track record of architecting enterprise-level data ecosystems, someone who thrives on pushing the boundaries of what's possible in data manipulation. Your expertise will ensure data is not … Remarkable experience with Development and maintenance of data pipeline and ETL processes using Azure data Factory, Azure Data Lake and Azure Databricks. Exceptional experience developing codes, testing for quality assurance, administering Azure Databricks platform and exposure to Unity Catalog. Exposure of Performance Tuning more »

Are you passionate about leveraging cutting-edge data technologies to drive business success? Do you thrive in dynamic environments where innovation and collaboration are valued? If so, we have an exciting opportunity for you! Position: Lead Data Engineer Responsibilities: Lead Data Management and Architecture … Spearhead the design and implementation of robust data solutions across the organization, ensuring scalability and efficiency. Next Gen Data Platform Leadership: Act as the primary data engineer supporting the development of our next-generation data platform, collaborating closely with third-party vendors … Skills and Experience: Proficiency in Microsoft Azure Data Services, including Azure SQL Database, Azure Data Factory, Azure Data Lake Storage, and Azure HDInsight. Advanced skills in SQL programming, with the ability to design, optimize, and maintain complex queries and databases. Strong experience in more »

Are you looking for an exciting opportunity in Solution Architecture ? Are you passionate about everything cloud and data ? Join us as an AWS Cloud Data Platform Solution Architect Careers at TCS: It means more TCS is a purpose-led transformation company, built on belief. We do … knowledge/experience: Experience of AWS Data platform experience including AWS-EMR Cluster Platform, Glue, EC2, S3, VPC, IAM, RDS, Redshift, Sagemaker, Lake formation and Glue catalogue. Data Storage Fundamentals. Cloud specific patterns and technologies. AWS Service Selection, Container design. Designing DevSecOps pipelines. Experience in … Pyspark. Good to have: Experience in Databricks platform. Insurance domain knowledge (Pensions & Annuity.) Understand the data platform solutions like data lake, data warehouse, data marts. AWS solution architect certifications. Rewards & Benefits: TCS is consistently voted a Top Employer in the UK more »

REMOTE Site Reliability Engineer (SRE DevOps Infrastructure AWS Python Java Go JavaScript Big Data Lake Data Mesh Kubernetes Terraform Finance Trading Glue Athena Dremio Iceberg Snowflake DBT Arrow gRPC protobuf Airflow Ignite Asset Manager Investment Management Financial Services Hedge Fund DevOps) required by our asset … Good Python, JavaScript, Java or Go Terraform SRE experience in an enterprise scale environment The following is DESIRABLE, not essential: SRE for big data Glue, Athena, Dremio, Iceberg, Snowflake, DBT … Arrow, gRPC, protobuf, Airflow, Ignite Graffana, Prometheus Role: REMOTE Site Reliability Engineer (SRE DevOps Infrastructure AWS Python Java Go JavaScript Big Data Lake Data Mesh Kubernetes Terraform Finance Trading Glue Athena Dremio Iceberg Snowflake DBT Arrow gRPC protobuf Airflow Ignite Asset Manager Investment Management Financial more »

Senior Data Engineer About us HENI is a technology company pioneering art markets and information. We work with world-leading artists and estates across various sectors, including printmaking, book publishing, NFTs, digital content, and art research – all underpinned by cutting-edge research in analytics, AI, and machine learning. … Our team comprises of talented web developers, data engineers, data scientists, and infrastructure specialists who thrive on solving complex problems. We enjoy developing machine learning tools and embracing the latest in infrastructure and coding methodologies. We maintain high standards of technical quality via thorough testing and … ingestion, Prefect for pipeline orchestration, AWS Glue for managed ETL, along with Pandas and PySpark for pipeline logic implementation. We utilize Delta Lake and PostgreSQL for data storage, emphasizing the importance of data integrity and version control in our workflows. Day-to-day more »

Senior Data Architect Joining Capco means joining an organisation that is committed to an inclusive working environment where you're encouraged to Be Yourself At Work. We celebrate individuality and recognize that diversity and inclusion, in all forms, is critical to success. It's important to us that … agile and entrepreneurial culture. Your Capco Day/Key Responsibilities You will accompany and lead complex agile consulting projects supporting our clients with data architecture design and delivery. You will work with technology leaders across Financial Services to provide best practice guidance for data management and … procedures, architectures and technologies in one or more of the following topics: Contextual Datamodelling, Entity Relationship Modelling, Logical & Physical Datamodelling and Data Lake design. Experience designing and implementing cloud data migration and storage patterns on one or more of AWS, GCP and Microsoft Azure Experience more »

partner with Deutsche Bank to support contingent recruitment processes. Job description - the role On behalf of Deutsche Bank, we are looking for a Data Governance Analyst for an initial 6 month contract based in the Banks London Office on a hybrid working model of 3 days per week … Together we're sharing new perspectives and transforming what it means to be a bank.' Purpose of the role: Transaction Monitoring & Screening Global Data Steward Office is responsible for Data Governance, Data Quality Management (DQM) & Data Programme delivery covering key functions within … Transaction Monitoring and Screening. Working as part of the DQM team the Senior Data Analyst will be responsible for supporting the Data Quality Management Framework and for contributing towards the central Data Analytics capability. Data Quality Management Framework covers Data more »

Data Architect - Consulting - FS A leading global consulting firm is looking for a Data Architect to join their team! As the successful candidate, you will be in charge of designing and implementing scalable architecture and lead a global rollout of the client's digital content platform. … office and it is offering up to £155,000 + bonus + excellent benefits! Required skills: More than 15 years of experience in Data Architecture implementing data migrations to at least one cloud platform including AWS, GCP and/or Azure Data … modernisation and data ingestion experience Experience leading at least one implementation of Databricks and/or Snowflake is required Data Lake and data storage design experience for both structure and unstructured data Experience working for a consultancy or for a financial more »

Azure Data Engineer - £60,000 + Benefits, London … weekly on-site You must have full right to work authorisation. No sponsorship available. Azure data platform technologies, including Data Lake Gen 2, Synapse, Analysis Services, and Power BI. Working in a leading UK and global business you will be a key part of the … platforms, offering near real-time process reporting solutions. Your Profile : Extensive experience with Azure data platform technologies, ideally including Data Lake Gen 2, Synapse, Analysis Services, and Power BI. Proven skills with Azure Data Factory & Databricks/Apache/Sparx, along with extensive more »

We are looking for Senior Data Solution Architect Below are details Data Solution Architect Proven experience as architect and engineering lead in Data & Analytics stream In-depth understanding of data structure principles and data platforms Problem-solving attitude, solution mindset … with implementation expertise Working experience on Modern data platforms which involves Cloud & related technologies, Graph databases, Virtualization Well-versed with the end2end data management philosophies and governance processes Has pre-sales experience and have involved in RFP/RFI/RFQ processes Creative problem-solver with … strong communication skills Excellent understanding of traditional and distributed computing paradigm Should have excellent knowledge in data warehouse/data lake technology and business intelligence concepts Should have good knowledge in Relational, No-SQL, Big Data Databases and should be able to write more »

Senior Analytics Data Engineer 🌐 Global Insurance Leader 💵 £120,000 + 20% Bonus 🏠 1-2 days in the office (Central London) 🌐 Join a Global Insurance Leader: Exceptional Opportunity for a Visionary Data Engineering Expert! Are you an experienced Senior Analytics Data Engineer who thrives on … transforming data into actionable insights? Are you ready to join a top-tier global insurer on an ambitious journey to become the industry's gold standard in data-driven decision-making? Look no further - this is your chance to make a difference and shape the future … of insurance! Our client, a prestigious global insurer, is at the forefront of leveraging data and technology to revolutionize decision-making across the organization. Their ambitious Data and Analytics strategy aims to provide a 360-degree view of data to empower every team member more »