1 to 25 of 100 Permanent Data Lake Jobs in London

Data Integration Manager A brand new and exciting opportunity has recently arisen for a Data Integration Manager with excellent knowledge in Azure and Salesforce SQL to join one of the UK's most notorious Charities. We're in search of an … experienced Data Integration Manager to lead the smooth integration of data from diverse sources into our Azure Data Lake and Salesforce NFP Cloud. The ideal candidate will oversee the entire data flow process, including cleansing, harmonization, and integration into Salesforce NFP … Cloud and Marketing Cloud. Proficiency in data management technologies, especially Azure Data Lake, Salesforce NFP, and Azure Data Factory, is essential for this role. Skills that are required for a Data Integration Manager are: Expert in Azure Experience working with data more »

to work across the full stack. Strong technical capabilities around technologies such as: Azure cloud (Azure Data Factory, Azure Data Lake, Azure Data Bricks), DevOps practices/tools, Snowflake, or other relational databases, SQL, PowerBI Ability to work both independently and within a more »

time to join this ambitious team!! WHAT YOU'LL BE DOING As Senior Machine Learning Engineer, you will sit within the company's Data Science team and report to the Director of Engineering. Their Machine Learning stack runs on on-demand containerised infrastructure – ranging from sub-second latency … TECH STACK: Python, Flask, Redis, Postgres, React, Plotly, Docker. Temporal; AWS Athena SQL, Athena & EMR Spark, ECS Fargate; Azure Synapse/Data Lake Analytics, HDInsight. KEY RESPONSIBILITIES Lead the productionisation of Monolith’s ML models and data processing pipelines, both mid-low-level system and … design and exemplary hands-on implementations using Spark and other tech stacks Shape the ML engineering culture and practices around model & data versioning, scalability, model benchmarking, ML-specific branching & release strategy Concisely break down complex high-level ML requirements into smaller deliverables (epic à task) Proactively identify blockers more »

with business, change and technology resources Create business cases detailing cost, timeline and resource estimates Coordinate and participate in detailed design exercises Identify data requirements and enhancements required to the companies enterprise data model. Requirements: Experience delivering formal business projects within private equity, investment management, investment … back-office functions, and their requirements Experience designing and producing compelling analytical solutions for Investment Professionals using qualitative and quantitative measures Knowledge of data architectures, data warehouses, data lakes etc. This is a great opportunity to join a growing change team and work across more »

Are you looking for an exciting opportunity in Solution Architecture ? Are you passionate about everything cloud and data ? Join us as an AWS Cloud Data Platform Solution Architect Careers at TCS: It means more TCS is a purpose-led transformation company, built on belief. We do … knowledge/experience: Experience of AWS Data platform experience including AWS-EMR Cluster Platform, Glue, EC2, S3, VPC, IAM, RDS, Redshift, Sagemaker, Lake formation and Glue catalogue. Data Storage Fundamentals. Cloud specific patterns and technologies. AWS Service Selection, Container design. Designing DevSecOps pipelines. Experience in … Pyspark. Good to have: Experience in Databricks platform. Insurance domain knowledge (Pensions & Annuity.) Understand the data platform solutions like data lake, data warehouse, data marts. AWS solution architect certifications. Rewards & Benefits: TCS is consistently voted a Top Employer in the UK more »

Role - Azure Data Engineer Rate - £600 per day inside Hybrid working - 1 day a week onsite (Centre of London) Duration - Initial 6 months My client is a large Financial services organisation based out of London and they are going through a huge data transformation that is … expected to run for at least 18 months. My client is seeking experienced Azure Data Engineers for a stakeholder-facing project. This role will have particular emphasis on Azure, Databricks and DevOps. Our client is in the consultancy sector but does not have a preference for previous experience … There is a massive emphasis on Pyspark and Databricks for this particular role. Technical Skills Required: Azure (ADF, Functions, Blob Storage, Data Lake Storage, Azure Data Bricks) Databricks Spark Delta Lake SQL Python PySpark ADLS Day To Day Responsibilities: Extensive experience in designing, developing more »

We are looking for Senior Data Solution Architect Below are details Data Solution Architect Proven experience as architect and engineering lead in Data & Analytics stream In-depth understanding of data structure principles and data platforms Problem-solving attitude, solution mindset … with implementation expertise Working experience on Modern data platforms which involves Cloud & related technologies, Graph databases, Virtualization Well-versed with the end2end data management philosophies and governance processes Has pre-sales experience and have involved in RFP/RFI/RFQ processes Creative problem-solver with … strong communication skills Excellent understanding of traditional and distributed computing paradigm Should have excellent knowledge in data warehouse/data lake technology and business intelligence concepts Should have good knowledge in Relational, No-SQL, Big Data Databases and should be able to write more »

My client are looking for a talented and motivated Big Data Architect (Azure, Databricks, Spark) to be based … in their London office. You'll be responsible for providing technical leadership in architecting and designing end-to-end solutions for the organisation's datalake initiatives, as they provide increasing numbers of services to consumers across multiple digital channels. You'll be responsible for managing a remote engineering team of …/experience is essential: Strong implementation experience using Spark and Databricks Strong Cloud experience (ideally Azure) Previously heavily involved in an implementation programme Data Warehouse Strong stakeholder management experience Excellent IT background, ideally within Telco/Wireless service providers but not essential Salary: Excellent + bonus + package more »

Senior Analytics Data Engineer 🌐 Global Insurance Leader 💵 £120,000 + 20% Bonus 🏠 1-2 days in the office (Central London) 🌐 Join a Global Insurance Leader: Exceptional Opportunity for a Visionary Data Engineering Expert! Are you an experienced Senior Analytics Data Engineer who thrives on … transforming data into actionable insights? Are you ready to join a top-tier global insurer on an ambitious journey to become the industry's gold standard in data-driven decision-making? Look no further - this is your chance to make a difference and shape the future … of insurance! Our client, a prestigious global insurer, is at the forefront of leveraging data and technology to revolutionize decision-making across the organization. Their ambitious Data and Analytics strategy aims to provide a 360-degree view of data to empower every team member more »

with Architecture decisions and strategy Requirements: TypeScript/JavaScript, Node.js, and React. Strong database skills (SQL) and RESTful API design experience. Working with data pipelines and data architecture Desired Skills: Experience with Python. Familiarity with AWS and Data Platforms Knowledge of Enterprise SaaS development … data lakes like snowflake This is a role that would be great for a Software Engineer that wants to drive the architecture for a product that has the potential to make a huge impact across this particular sector. Must be eligible to work in the UK and be more »

Senior Data Engineer - 6 month FTC London (Hybrid Role) £85,000 per annum Senior Data Engineer Opportunity An exciting opportunity has arisen for a Senior Data Engineer to join a dynamic team dedicated to shaping effortless, digital-first customer experiences. This role is pivotal … in driving value for customers through modelling, sourcing, and data transformation. The Senior Data Engineer will collaborate closely with core technology and architecture teams, delivering strategic data solutions while promoting Agile and DevOps methodologies in data engineering delivery. Responsibilities: Automate data … mentoring abilities and good critical thinking with problem-solving skills. Nice to Have: Experience with relevant technologies such as Synapse, Azure Data Lake, Azure Data Factory, and Databricks. Why Join: This is an opportunity to join a collaborative environment that encourages innovation and values each more »

App Services, virtual networks, Exchange Online, Intune). * Implement Azure security best practices and Microsoft Defender. * Monitor and mitigate security incidents. * Protect sensitive data and ensure data governance compliance. * Collaborate … with Data and AI teams on Azure-based solutions. * Guide on Azure services for Data and AI (Data Lake, Data Factory, OpenAI). * Integrate Data and AI solutions with existing systems. * Maintain on-premises infrastructure (Windows Server, Active Directory … CV to connor . thomas @ nextech - group . co . uk. They are interviewing IMMEDIATELY!! Keywords: Azure, Entra ID, SQL, Exchange Online, Intune, Data, Windows Server, Active Directory, Oracle, VMware, Red Hat, Enterprise, Linux, Cisco Switches, Palo Alto Firewalls, PowerShell, Python more »

Write high-quality, maintainable code using languages such as Python and SQL Establish data tools like Snowflake and Azure Data Lake Services (ADLS) Gen 2 Utilize PowerBI, Tableau, or similar tools to design and create interactive and visually appealing dashboards and reports. Work closely with … goals. Qualifications: 10+ years of overall IT experience , with 2-4 years of hands-on experience in Azure. Proficiency in at least one data visualization tool, such as PowerBI, Tableau, or similar. Knowledge of database management systems (DBMS) and proficiency in SQL Server or similar platforms. Experience with more »

have been supporting professionals from the entertainment space for the past century. Lead a team of internal and 3rd party Application, Integration and Data Engineers to deliver world-class solutions for the organisation to manage its business activities, through understanding the business requirements and translating into technical solutions. … you will excel in this role: Integration Application architecture and design Azure cloud-based infrastructure solutions Azure Integration services, Azure API management Azure Data Lakes, Azure Synapse Networking, security and storage technologies Delivering a microservices architecture Developing and implementing RESTful APIs, microservices and UIs XML, JSON, Web APIs more »

Senior Strategy Manager - Data & AI At Centrica, our purpose is to energise a greener, fairer future. Achieving this … is made possible through our family of trusted brands and businesses. Across the Centrica group we have established a vast rudimentary data lake comprising of UK & Ireland wide customer and enterprise data for group and our trusted businesses (British Gas, Bord Gais Energy, Centrica Business … Solutions, Centrica Energy and Upstream). This data source presents a huge opportunity to create significant value for our customers, for us to build innovative products, provide new services and improve the capabilities of our Centrica employees to drive our journey as we transition towards NetZero. This is more »

Senior Python Software Engineer (Architecture Programmer Developer Java Python Software Engineer Data Enterprise Engineering Developer Programmer AWS Python Athena Glue Airflow Ignite JavaScript Agile Pandas NumPy SciPy Spark Dremio Snowflake Apache Iceburg Iceberg Arrow DBT gRPC protobuf TypeScript Finance Trading Front Office Investment Banking Asset Manager Financial Services … DBT Buy-side asset management (hedge fund, asset manager, investment management) Role: Senior Python Software Engineer (Architecture Programmer Developer Java Python Software Engineer Data Enterprise Engineering Developer Programmer AWS Python Athena Glue Airflow Ignite JavaScript Agile Pandas NumPy SciPy Spark Dremio Snowflake Apache Iceburg Iceberg Arrow DBT gRPC … Office. The data is sourced from a variety of external vendors and internal departments and held in an AWS data lake, although this is being migrated to a data mesh architecture. They are working heavily with Python, Java, AWS and Iceberg. If you more »

Java Technical Lead (Senior Architecture Programmer Developer Java Python Software Engineer Data Enterprise Engineering Developer Programmer AWS Python Athena Glue Airflow Ignite JavaScript Agile Pandas NumPy SciPy Spark Dremio Snowflake Apache Iceburg Iceberg Arrow DBT gRPC protobuf TypeScript Finance Trading Front Office Investment Banking Asset Manager Financial Services … DBT Buy-side asset management (hedge fund, asset manager, investment management) Role: Java Technical Lead (Senior Architecture Programmer Developer Java Python Software Engineer Data Enterprise Engineering Developer Programmer AWS Python Athena Glue Airflow Ignite JavaScript Agile Pandas NumPy SciPy Spark Dremio Snowflake Apache Iceburg Iceberg Arrow DBT gRPC … Office. The data is sourced from a variety of external vendors and internal departments and held in an AWS data lake, although this is being migrated to a data mesh architecture. They are working heavily with Java, Python, AWS and Iceberg. If you more »

Python Technical Lead (Senior Architecture Programmer Developer Java Python Software Engineer Data Enterprise Engineering Developer Programmer AWS Python Athena Glue Airflow Ignite JavaScript Agile Pandas NumPy SciPy Spark Dremio Snowflake Apache Iceburg Iceberg Arrow DBT gRPC protobuf TypeScript Finance Trading Front Office Investment Banking Asset Manager Financial Services … DBT Buy-side asset management (hedge fund, asset manager, investment management) Role: Python Technical Lead (Senior Architecture Programmer Developer Java Python Software Engineer Data Enterprise Engineering Developer Programmer AWS Python Athena Glue Airflow Ignite JavaScript Agile Pandas NumPy SciPy Spark Dremio Snowflake Apache Iceburg Iceberg Arrow DBT gRPC … Office. The data is sourced from a variety of external vendors and internal departments and held in an AWS data lake, although this is being migrated to a data mesh architecture. They are working heavily with Python, Java, AWS and Iceberg. If you more »

As the Data Architect, you'll lead data architecture initiatives, establishing standards and practices essential for effective data management. Your focus will be on designing and implementing robust data and information systems, ensuring they meet stringent criteria for modelling, flow, lineage, quality … and security. Working closely with stakeholders and data governance teams, you'll drive adherence to established standards, facilitating data-driven decision-making processes across the organisation. This role encompasses: Developing and maintaining a comprehensive Data Architecture Framework and maturity roadmap. Enforcing best data … Azure stacks, including data pipelines, security and networking Expertise with data warehousing, data lakes and data lake houses Experience with master data management software and technologies Experience with ETL technologies including SQL Server, SSIS, Azure Data Factory more »

Job Title: AWS Data Architect Location: United Kingdom (U.K.) - Hybrid The Company: At Amber Labs, we are a cutting-edge UK and European technology consultancy dedicated to delivering exceptional value to our clients in the public sector. Our company culture promotes collaboration, autonomy, and rapid learning, empowering our … work environment focused on deliverable outcomes, along with a range of benefits to support our employees’ well-being and professional development. Key Responsibilities: Data Architecture Design: Collaborate with stakeholders to understand data requirements and design scalable and secure data architectures on AWS. Develop data models, data flow diagrams, and architectural documentation to support financial applications. Cloud Data Management: Design, implement, and manage cloud-based data solutions on AWS, including data lakes, data warehouses, and analytics platforms. Ensure the scalability, performance, and cost more »

Senior Data Engineer About us HENI is a technology company pioneering art markets and information. We work with world-leading artists and estates across various sectors, including printmaking, book publishing, NFTs, digital content, and art research – all underpinned by cutting-edge research in analytics, AI, and machine learning. … Our team comprises of talented web developers, data engineers, data scientists, and infrastructure specialists who thrive on solving complex problems. We enjoy developing machine learning tools and embracing the latest in infrastructure and coding methodologies. We maintain high standards of technical quality via thorough testing and … ingestion, Prefect for pipeline orchestration, AWS Glue for managed ETL, along with Pandas and PySpark for pipeline logic implementation. We utilize Delta Lake and PostgreSQL for data storage, emphasizing the importance of data integrity and version control in our workflows. Day-to-day more »

My leading Energy and Commodities conglomerate are currently seeking a skilled individual with experience in Python-based data pipeline, data lake house development and an appreciation for Object orientated languages such as C# or Java, using leading orchestration tools. In this role, you'll lead … the development of robust, fully tested data pipelines in Python using cutting-edge platforms like Dagster/Airflow and apply your expertise … in real-time data streaming solutions using Kafka. You'll play a key role in expanding their on prem data lake infrastructure to help enhance support for new lines of businesses they have including Systematic trading, Trading and Shipping commodities, ensuring strong data more »

Lead Data Engineer - Azure/Spark/Databricks Role: Lead Data Engineer Location: London (3 days a week) Salary: £85k - £90k A leading technology firm, renowned for connecting people globally, is looking for a skilled Lead Data Engineer to join their expanding team. In … engineering stack. Key Responsibilities: Lead the design, development, and implementation of data solutions using Azure services such as Spark, Data Lake, Synapse, and Databricks. Code in Spark and oversee the technical design aspects of projects, ensuring adherence to engineering best practices. Lead and mentor a … with remote teams, leveraging your experience in leading distributed teams to drive success. Requirements: Extensive experience with Azure services, including Spark, Data Lake, Synapse, and Databricks. Proficiency in coding with Spark and a strong understanding of design aspects. Deep understanding of engineering best practices and the ability more »

I am searching for a talented Data Architect to join my client, a leading Fintech company in London. In this exciting role, you'll be responsible for designing, developing, and implementing the data architecture that forms the backbone of their data-driven strategy. You … ll work closely with stakeholders across the business to understand their needs and translate them into robust, scalable data solutions. Job Title: Data Architect Location: London (Hybrid) Salary: £95,000 - £100,000 Industry: Fintech Start: ASAP What you'll be doing: Design and develop enterprise-level … data architectures and solutions. Select and implement appropriate data management tools and technologies. Ensure data quality, consistency, and security across the organization. Collaborate with data engineers and analysts to deliver successful data projects. Stay up-to-date on the latest more »

Data Engineering Group building greenfield pipelines, frameworks, APIs and core Data store (Data warehouse and Data Lake) to enable the ingestion and provision (upstream and downstream) of traditional and alternate datasets in partnership with Quant Research and Trading teams firmwide (X … Asset - Equities, Rates, Commodities, FX and Crypto). Key Skills required 3+ years of exp. as a Software Developer/Data Engineer with Python as your core language Extensive API and Microservices development experience 3+ years SQL Strong DB skills ideally in both SQL and NoSQL based inc. more »