1 to 25 of 61 Data Pipeline Jobs in the City of London

AWS Data Engineer - VP Level Location: London (Canary Wharf) Salary: £(phone number removed) - (phone number removed) A leading financial institution is seeking a Senior AWS Data Engineer to join a greenfield initiative focused on transforming market data access and utilization. This VP-level role offers strategic influence, technical leadership, and cross-functional collaboration across … business units. Key Responsibilities: * Design and maintain scalable data pipelines, warehouses, and lakes using AWS technologies. * Implement robust data governance, quality, and security practices. * Collaborate with data scientists to deploy machine learning models. * Lead or contribute to strategic planning, risk management, and stakeholder engagement. * Drive innovation through advanced analytics and research-based problem solving. … To be successful you should have: 10 years hands-on experience in AWS data engineering technologies, including Glue, PySpark, Athena, Iceberg, Databricks, Lake Formation, and other standard data engineering tools. Previous experience in implementing best practices for data engineering, including data governance, data quality, and data security. Proficiency More ❯

Media, Retail and CPG, and Public Services. Consolidated revenues as of 12 months ending December 2024 totaled $13.8 billion. Job Description: We are looking for a seasoned AWS Data Engineer to join our team and contribute to the development of scalable, secure, and high-performance data solutions. This role involves modernizing mainframe legacy systems, building robust … data pipelines, and enabling advanced analytics using AWS-native technologies. You’ll collaborate with cross-functional teams and play a key role in shaping our data strategy. Skills Must have : Proven experience in designing and maintaining scalable data lakes using AWS S3, with strong understanding of data lifecycle, access control, and performance … optimization. Hands-on experience migrating data from Mainframes to AWS, including transformation and validation of legacy formats. Proficiency in reading and processing VSAM data using Copybooks and the Cobrix library. Experience implementing data retention and archival strategies in cloud environments. Strong understanding and practical implementation of Medallion Architecture (Bronze, Silver, Gold layers) for structured More ❯

Media, Retail and CPG, and Public Services. Consolidated revenues as of 12 months ending December 2024 totaled $13.8 billion. Job Description: We are looking for a seasoned AWS Data Engineer to join our team and contribute to the development of scalable, secure, and high-performance data solutions. This role involves modernizing mainframe legacy systems, building robust … data pipelines, and enabling advanced analytics using AWS-native technologies. You’ll collaborate with cross-functional teams and play a key role in shaping our data strategy. Skills Must have : Proven experience in designing and maintaining scalable data lakes using AWS S3, with strong understanding of data lifecycle, access control, and performance … optimization. Hands-on experience migrating data from Mainframes to AWS, including transformation and validation of legacy formats. Proficiency in reading and processing VSAM data using Copybooks and the Cobrix library. Experience implementing data retention and archival strategies in cloud environments. Strong understanding and practical implementation of Medallion Architecture (Bronze, Silver, Gold layers) for structured More ❯

Senior Data SME - Job Description Experience : 15+ years Experience in all 4 platforms, Business Platform, Technology platform, Enablement platform, Foundation Platform Business Perspective Experience as Principal Arch in leading organizations handling customers effectively. Advanced understanding of Enterprise Architecture principles. Well-versed with the end to end data management philosophies and governance processes fit to business. Deliver … data solutions and integration pipelines in accordance with agreed organizational standards that ensure services are resilient, scalable and future-proof Collaborate with cross-functional teams to define and implement cloud strategies that align with business goals Participating in architectural discussions and decision-making processes to ensure alignment with business goals and best practices Working experience on Modern data … involves Cloud & related technologies, Graph databases, Virtualization Technology Perspective Worked in multiple large, distributed mission critical applications as an architect. Proven experience as architect and engineering lead in Data & Analytics stream. In-depth understanding of data structure principles and data platforms Used Architecture tools for modelling. Working experience on Modern data platforms More ❯

Senior Data SME - Job Description Experience : 15+ years Experience in all 4 platforms, Business Platform, Technology platform, Enablement platform, Foundation Platform Business Perspective Experience as Principal Arch in leading organizations handling customers effectively. Advanced understanding of Enterprise Architecture principles. Well-versed with the end to end data management philosophies and governance processes fit to business. Deliver … data solutions and integration pipelines in accordance with agreed organizational standards that ensure services are resilient, scalable and future-proof Collaborate with cross-functional teams to define and implement cloud strategies that align with business goals Participating in architectural discussions and decision-making processes to ensure alignment with business goals and best practices Working experience on Modern data … involves Cloud & related technologies, Graph databases, Virtualization Technology Perspective Worked in multiple large, distributed mission critical applications as an architect. Proven experience as architect and engineering lead in Data & Analytics stream. In-depth understanding of data structure principles and data platforms Used Architecture tools for modelling. Working experience on Modern data platforms More ❯

SC Cleared Data Consultant London Up to £100,000 per annum Are you a data-driven professional with a passion for delivering impactful solutions? Do you excel in translating complex data challenges into actionable insights? If so, Morela has an exciting opportunity for you. Morela is proud to partner with one of the UK … s leading data consultancies, renowned for their innovative approach to data integration and analytics. As they continue to expand, they're seeking a skilled SC Cleared Data Consultant to join their dynamic team. This role is perfect for someone who combines technical expertise with a consultative mindset, aiming to drive value across diverse projects. … operates at the intersection of advanced technology and real-world impact. Serving both public and private sectors, their services encompass strategic advisory, digital transformation, and end-to-end data engineering. As an SC Cleared Data Consultant, you'll play a crucial role in designing and implementing data solutions that empower clients to make informed More ❯

Role Title: Pre-Sales & Data Solution Architect Business Unit/Segment: Data Management Location: London, United Kingdom (Flexible hybrid working) Employment Type: Permanent Summary Our client, a leading global data and AI consultancy, is seeking a Pre-Sales & Data Solution Architect Lead - hybrid role to become one of their leaders in this … area in their Data Management practice with a focus on banking clients . This is a strategic, client-facing role where you will partner with senior stakeholders to modernise data platforms, accelerate cloud adoption, and drive measurable business value. The role combines technical leadership with consulting and pre-sales expertise . You will design enterprise data … as a trusted advisor, leading workshops and influencing senior stakeholders on strategy, roadmaps, and innovation. ETL/ELT & Data Engineering – Provide oversight on robust data pipeline and integration designs using SSIS, ADF, Informatica, or IBM DataStage. Data Governance & Quality – Define governance, metadata, and quality frameworks using tools such as Unity Catalog, Profisee, Alation More ❯

Role Title: Pre-Sales & Data Solution Architect Business Unit/Segment: Data Management Location: London, United Kingdom (Flexible hybrid working) Employment Type: Permanent Summary Our client, a leading global data and AI consultancy, is seeking a Pre-Sales & Data Solution Architect Lead - hybrid role to become one of their leaders in this … area in their Data Management practice with a focus on banking clients . This is a strategic, client-facing role where you will partner with senior stakeholders to modernise data platforms, accelerate cloud adoption, and drive measurable business value. The role combines technical leadership with consulting and pre-sales expertise . You will design enterprise data … as a trusted advisor, leading workshops and influencing senior stakeholders on strategy, roadmaps, and innovation. ETL/ELT & Data Engineering – Provide oversight on robust data pipeline and integration designs using SSIS, ADF, Informatica, or IBM DataStage. Data Governance & Quality – Define governance, metadata, and quality frameworks using tools such as Unity Catalog, Profisee, Alation More ❯

Data Engineering Technical Lead Global Investment Bank London - Hybrid Permanent - Excellent Package + Benefits We are working with one of the world's leading banking groups, who we have partnered with for 15 years. We are seeking an experienced Data Architect/EDM Developer/Data Engineering Lead to join their International Technology team … in London. You will be a key part of the Architecture, Middleware, Data & Enterprise Services (AMD) division, driving data engineering, integration and automation initiatives across MUFG's EMEA banking and securities entities. This is a hands-on leadership role, combining technical expertise with mentoring and team leadership. Key Responsibilities Architect, design and deliver enterprise-wide EDM … and data solutions. Lead and mentor EDM developers, ensuring high-quality, cost-effective delivery. Drive data innovation, automation and best practices across EMEA. Translate business requirements into functional and technical designs. Ensure compliance with SDLC, governance, and risk policies. Skills & Experience - Essential Strong SQL Server or Snowflake skills. Advanced knowledge of low-code/no-code More ❯

As a Data Specialist at this tech scaleup in London, you will analyse, collect, sort, and create data solutions that integrate across multiple products. You will steer decisions regarding data and build the models that power their products. You will take client insights and requirements and make them a reality, by developing datasets and … to drive innovation into the product development process. You will interpret your clients objectives, desires, and preferences to help the wider technical team understand the opportunities and apply data engineering responsibilities consistently. You will be working on an interesting range of projects that deliver to National Security customers and as such, you will have to hold the highest … pipelines that are tested and robust using a variety of technologies and techniques depending on the available technologies (Nifi, Spark) Build analytics tools that utilise the data pipeline to provide actionable insights into client requirements, operational efficiency, and other key business performance metrics. Complete onsite client visits and provide excellent customer support service. Problem-solve in a More ❯

VCCP are looking for a Head of Data to build our data function from the ground up, architect our MS Fabric warehouse, stand up secure pipelines across our internal systems/TM1/Sage/ATS/HubSpot, lead a lean team, and deliver high-impact dashboards for senior leadership. Role Overview: We are seeking a … hands-on Head of Data to architect and lead the data function from the ground up. This is a newly created role in a fast-paced environment, where data is central to unlocking operational efficiency, commercial insight, and strategic decision-making. The successful candidate will be responsible for building the data warehouse … integrating strategic data sources, and delivering high-impact dashboards and analytics to senior leadership. They must be confident operating in a high-pressure, fast-evolving environment and capable of influencing senior stakeholders across the business. Key Responsibilities: Build and deploy a centralised data warehouse using Microsoft Fabric, integrating key systems including (some of these are internal More ❯

VCCP are looking for a Head of Data to build our data function from the ground up, architect our MS Fabric warehouse, stand up secure pipelines across our internal systems/TM1/Sage/ATS/HubSpot, lead a lean team, and deliver high-impact dashboards for senior leadership. Role Overview: We are seeking a … hands-on Head of Data to architect and lead the data function from the ground up. This is a newly created role in a fast-paced environment, where data is central to unlocking operational efficiency, commercial insight, and strategic decision-making. The successful candidate will be responsible for building the data warehouse … integrating strategic data sources, and delivering high-impact dashboards and analytics to senior leadership. They must be confident operating in a high-pressure, fast-evolving environment and capable of influencing senior stakeholders across the business. Key Responsibilities: Build and deploy a centralised data warehouse using Microsoft Fabric, integrating key systems including (some of these are internal More ❯

evolving Lloyds managing agent thats leading the way across multiple lines, including Property, Casualty, and Specialty. As part of a major transformation, theyre building a modern, business-driven data platform from the ground up and theyre looking for a Senior Data Engineer to help shape it. About the Business This Lloyds managing agent is authorised and … regulated by the PRA and FCA, managing a high-performing syndicate. They are known for underwriting excellence and are investing heavily in data and technology to support their long-term growth. Why this role? You'll play a key role in delivering a robust, scalable data platform that unlocks better MI, improves data quality … and drives automation across the business. Youll work closely with teams across Actuarial, Finance, and IT helping to bring their data needs to life. This is a genuinely exciting opportunity to influence the data direction of a Lloyds business, not just maintain it. Key responsibilities: Design and build Data Lake and Data More ❯

Our Client has a requirement for a Data Engineer , who will be required to work on a contract basis Remote working . Role Purpose: We are … seeking an experienced Data Engineer to design, build, and optimize data solutions using the Microsoft technology stack. The role focuses on developing scalable ETL pipelines, data models, and cloud-based solutions leveraging Azure Data Factory, Azure Synapse, Azure SQL, and Azure Functions . The ideal candidate will have a proven background in … data engineering, cloud architecture, and enterprise data systems . Job Role Responsibilities: Design and implement robust ETL pipelines using Azure Data Factory . Develop and maintain data models and warehouses with Azure Synapse and Azure SQL . Build scalable data processing solutions using Azure Functions . Create interactive dashboards and More ❯

Data Scientist - Property London – Hybrid Up to £65,000 About the Role A leading London-based data and technology company in the property space is looking for a Data Scientist to join their growing product-focused team. This is an exciting opportunity to work as a full stack/end to end Data Scientist, driving real business impact. You’ll collaborate closely with stakeholders to build and deploy ML models and data-driven tools that are core to the company’s commercial offerings. Key Responsibilities Identify and scope opportunities where machine learning and data science can enhance product offerings. Design and run experiments to test hypotheses and … support product development. Develop robust data pipelines to support science experimentation and production. Deploy, maintain, and monitor predictive models in live environments. Communicate findings and technical concepts clearly to both technical and non-technical audiences. Contribute to agile delivery workflows and cross-functional collaboration. What We’re Looking For Solid understanding of core data science techniques More ❯

Senior Data Engineer MUST HAVE SNOWFLAKE Salary - £90-100k with 15% bonus Hybrid working - couple of days in the office City of London We are looking for: Good understanding of data engineering principles A good technical grasp of Snowflake and automating it, and transforming complex datasets AWS Skillset Delivery experience Building solutions in snowflake implementing … data warehousing solutions using Snowflake and AWS Hands-on experience with AWS services such as Glue (Spark), Lambda, Step Functions, ECS, Redshift, and SageMaker. Enthusiasm for cross-functional work and adaptability beyond traditional data engineering. Examples like building APIs, integrating with microservices, or contributing to backend systems - not just data pipelines or data … Advanced SQL skills for complex data queries and transformations Proficiency in at least one programming language, with Python strongly preferred for data processing, automation, and pipeline development AWS or Snowflake certifications are a plus Hands-on experience with AWS services such as Glue (Spark), Lambda, Step Functions, ECS, Redshift, and SageMaker. Enthusiasm for cross-functional More ❯

Senior AWS Data Engineer - London - £125,000 Please note - this role will require you to work from the London based office. You must have the unrestricted right to work in the UK to be eligible for this role. This organisation is not able to offer sponsorship. An exciting opportunity to join a greenfield initiative focused on transforming how … market data is accessed and utilised. As a Senior AWS Data Engineer, you'll play a key role in designing and building a cutting-edge data platform using technologies like Databricks, Snowflake, and AWS Glue. Key Responsibilities: Build and maintain scalable data pipelines, warehouses, and lakes. Design secure, high-performance data architectures. Develop processing and analysis algorithms for complex data sets. Collaborate with data scientists to deploy machine learning models. Contribute to strategy, planning, and continuous improvement. Required Experience: Hands-on experience with AWS data tools: Glue, PySpark, Athena, Iceberg, Lake Formation. Strong Python and SQL skills for data processing and More ❯

Data Consultant – Aviation Industry Specialist - £800 - £1000 Our client is seeking a data-focused consultant with deep aviation expertise (ideally within airlines, low-cost carriers, or airports). The consultant will work directly with business units and senior leadership to identify and deliver measurable operational and commercial improvements through data-driven initiatives . This … opportunity is ideal for a professional who can blend strong aviation domain knowledge with modern data engineering and analytics experience , while confidently engaging stakeholders at all levels of the organisation. Candidate Profile Industry Expertise Proven aviation experience (airlines, LCCs, airports; adjacent industries considered). Strong track record of delivering successful data products and projects. Evidence of … generating tangible value (£ millions in opportunities or savings). Understanding of flight operations, ground handling, network planning, revenue management, or MRO. Data & Technology Knowledge (~10% focus) Familiarity with Data Mesh principles and decentralised data governance. Experience with Databricks for data pipelines, analytics, and ML environments. Knowledge of data as a More ❯

Data Consultant – Aviation Industry Specialist - £800 - £1000 Our client is seeking a data-focused consultant with deep aviation expertise (ideally within airlines, low-cost carriers, or airports). The consultant will work directly with business units and senior leadership to identify and deliver measurable operational and commercial improvements through data-driven initiatives . This … opportunity is ideal for a professional who can blend strong aviation domain knowledge with modern data engineering and analytics experience , while confidently engaging stakeholders at all levels of the organisation. Candidate Profile Industry Expertise Proven aviation experience (airlines, LCCs, airports; adjacent industries considered). Strong track record of delivering successful data products and projects. Evidence of … generating tangible value (£ millions in opportunities or savings). Understanding of flight operations, ground handling, network planning, revenue management, or MRO. Data & Technology Knowledge (~10% focus) Familiarity with Data Mesh principles and decentralised data governance. Experience with Databricks for data pipelines, analytics, and ML environments. Knowledge of data as a More ❯

Senior Data Engineer – AI & Neuroscience Location: London (Hybrid, Kings Cross) or San Francisco (Onsite/Hybrid) Employment: Full-time About the Company We are partnered with a pioneering biotech building the world’s first brain foundation models — large-scale AI systems designed to deeply understand, protect, and enhance the human brain. By generating our own data … direct influence on the technical systems that will underpin scientific and clinical breakthroughs. The work is ambitious, interdisciplinary, and sits at the cutting edge of AI, biology, and data engineering. The Role We are seeking a Senior Data Engineer to lead the design and scaling of the company’s core data infrastructure. You will … work will provide the backbone for training next-generation AI models, enabling researchers to extract real-world insights from raw biological data. Key Responsibilities Design and implement distributed data pipelines for multi-omic, neuroscience, and clinical datasets Build a unified feature store to serve ML training and downstream biological analysis Develop scalable storage, ingestion, and validation systems with More ❯

Senior Data Engineer – AI & Neuroscience Location: London (Hybrid, Kings Cross) or San Francisco (Onsite/Hybrid) Employment: Full-time About the Company We are partnered with a pioneering biotech building the world’s first brain foundation models — large-scale AI systems designed to deeply understand, protect, and enhance the human brain. By generating our own data … direct influence on the technical systems that will underpin scientific and clinical breakthroughs. The work is ambitious, interdisciplinary, and sits at the cutting edge of AI, biology, and data engineering. The Role We are seeking a Senior Data Engineer to lead the design and scaling of the company’s core data infrastructure. You will … work will provide the backbone for training next-generation AI models, enabling researchers to extract real-world insights from raw biological data. Key Responsibilities Design and implement distributed data pipelines for multi-omic, neuroscience, and clinical datasets Build a unified feature store to serve ML training and downstream biological analysis Develop scalable storage, ingestion, and validation systems with More ❯

Lead Azure Data Consultant/Engineer Azure/Microsoft Fabric/Synapse/Databricks Permanent Must be based in the UK and SC Cleared/SC Clearable (no sponsorship offered) In this role, you’ll be leading enterprise-scale data projects with a preferred Microsoft Partner that specialises in modern data platforms. You’ll … be working with well-known customers, helping them transform the way they use and manage data to deliver real business impact. As a Lead Data Engineer , you will be designing and delivering cutting-edge solutions across Azure Data services, Microsoft Fabric, and Databricks. This is a hands-on role where you’ll lead from … the front, working on enterprise projects that shape the future of data platforms. What you’ll be doing: Leading the design and implementation of modern data platform solutions for enterprise customers. Building and optimising data pipelines using Azure Data Factory , Microsoft Fabric, Synapse and Databricks . Guiding best practices in data More ❯

Lead Azure Data Consultant/Engineer Azure/Microsoft Fabric/Synapse/Databricks Permanent Must be based in the UK and SC Cleared/SC Clearable (no sponsorship offered) In this role, you’ll be leading enterprise-scale data projects with a preferred Microsoft Partner that specialises in modern data platforms. You’ll … be working with well-known customers, helping them transform the way they use and manage data to deliver real business impact. As a Lead Data Engineer , you will be designing and delivering cutting-edge solutions across Azure Data services, Microsoft Fabric, and Databricks. This is a hands-on role where you’ll lead from … the front, working on enterprise projects that shape the future of data platforms. What you’ll be doing: Leading the design and implementation of modern data platform solutions for enterprise customers. Building and optimising data pipelines using Azure Data Factory , Microsoft Fabric, Synapse and Databricks . Guiding best practices in data More ❯

day (Inside IR35) Start Date: 01/10/2025 - contract running until 31/12/2025 Company Introduction We are currently looking for an experienced cloud data developer to join a forward-thinking team on a long-term engagement. This is an excellent opportunity for someone who enjoys designing data pipelines and working with … modern AWS technologies to deliver scalable and efficient solutions. Job Responsibilities/Objectives: Creating and maintaining ETL/ELT workflows with AWS Glue to process data from a variety of sources. Setting up and administering Redshift clusters, ensuring cost-effective scaling and high performance. Developing robust … SQL queries, procedures, and transformations to support reporting and analytics. Ensuring accuracy, quality, and governance standards are met across all data processes. Investigating and resolving issues affecting pipeline performance or data integrity. Contributing to continuous improvements in architecture by keeping pace with AWS innovations. Required Skills/Experience: Strong practical knowledge of AWS Glue and More ❯

Cyber Insights Analyst City of London (Hybrid) Up to £55,000 + 20% Bonus Are you passionate about using data to solve complex problems and deliver real business impact? A global leader within Insurtech are looking for a Cyber Insights Analyst to join their global Cyber Insights & Analytics team. This is a hands-on role where you ll … transform diverse data sources. Ranging from policy and claims to cyber threat intelligence and incident response, into actionable insights that strengthen our underwriting decisions and frameworks. With their data strategy making strong progress, this is an exciting opportunity for someone with a few years experience in analytics or data science to take the next … manage reporting for underwriting class and product leaders. • Investigate underwriting performance queries and provide actionable insights. • Design, develop, and validate predictive models to inform underwriting decisions. • Work with data engineering teams to design and maintain Azure Databricks pipelines. • Support the automation of data-driven deliverables for proactive services. • Identify and resolve data issues, recommending More ❯