26 to 50 of 276 Data Pipeline Jobs in London

Consolidated revenues as of 12 months ending December 2024 totaled $13.8 billion. Responsibilities: Design, develop, and maintain robust and scalable ETL/ELT pipelines using Snowflake as the data warehouse. Implement data transformations and build analytical data models using dbt (data build tool). Develop and optimize complex SQL queries for data extraction, transformation, and loading in Snowflake. Orchestrate and schedule data workflows using Apache Airflow, including developing custom Python operators and DAGs. Write efficient and maintainable Python scripts for data processing, automation, and integration with various data sources and APIs. Ensure data quality, integrity, and governance throughout the data lifecycle. Collaborate with data analysts, data scientists, and business stakeholders to understand requirements and deliver data solutions. Implement and maintain CI/CD pipelines for data engineering processes, including version control with Git. Monitor data pipelines, troubleshoot issues, and optimize performance for efficiency and cost-effectiveness. Qualifications: Strong More ❯

Consolidated revenues as of 12 months ending December 2024 totaled $13.8 billion. Responsibilities: Design, develop, and maintain robust and scalable ETL/ELT pipelines using Snowflake as the data warehouse. Implement data transformations and build analytical data models using dbt (data build tool). Develop and optimize complex SQL queries for data extraction, transformation, and loading in Snowflake. Orchestrate and schedule data workflows using Apache Airflow, including developing custom Python operators and DAGs. Write efficient and maintainable Python scripts for data processing, automation, and integration with various data sources and APIs. Ensure data quality, integrity, and governance throughout the data lifecycle. Collaborate with data analysts, data scientists, and business stakeholders to understand requirements and deliver data solutions. Implement and maintain CI/CD pipelines for data engineering processes, including version control with Git. Monitor data pipelines, troubleshoot issues, and optimize performance for efficiency and cost-effectiveness. Qualifications: Strong More ❯

Who We Are BCG Expand is a specialist firm providing unique data-driven business intelligence as a trusted advisor to senior executives across the world's leading financial services firms to help them to operate more effectively. Expand's activities are focused on data benchmarking in technology, operations, FinTech and market data, with expertise … our professional, supportive and entrepreneurial working culture that we all enjoy. What You'll Do We're looking for a highly skilled Principal Engineer with a passion for data, cloud technologies, and building robust, scalable solutions - who also brings strategic product thinking and cross-functional leadership . In this role, you'll not only lead the design and … implementation of the data infrastructure that powers advanced analytics and data science across our platforms, but also contribute to the strategic direction, roadmap, and execution of data products in collaboration with senior stakeholders. Architect & Build Scalable Data Solutions Collaborate closely with senior product stakeholders to understand data needs and architect More ❯

Join Barclays as a Digital Customer Data Engineer, where you'll be leading the development for a new shared data platform that is transforming how individuals/organisations interact with the bank. Your role involves using the latest data and AI engineering skills to deliver a digital capability. As part of our team, you … ll also benefit from being part of a vast professional network, collaborating with industry mentors and experts. To be successful you should have experience with: Modern data engineering practices, inc. event-driven and data lake. Digital platform development, inc. experience developing colleague and customer facing applications. Banking software development. Deep technical awareness of cloud and data technology, such as Kafka, Data Bricks, microservices, API. Some other highly valued skills may include: Technical design: Analysis and documentation of system characteristics, inc. designing solutions to meet multi-faceted non-functional requirements. Communication Skills: Ability to clearly convey project goals to senior leadership, updates, and technical details to stakeholders at all levels, including non-technical More ❯

Snr Data Engineer - Global Accounts, Professional Services, AWSI-SDT-APJ-Japan Job ID: Amazon Web Services Japan GK AWS Global Services includes experts from across AWS who help our customers design, build, operate, and secure their cloud environments. Customers innovate with AWS Professional Services, upskill with AWS Training and Certification, optimize with AWS Support and Managed Services, and … practices and conduct skills transfer workshops. AWS consultants collaborate with customers and partners to address security and compliance, performance and scale, availability and manageability. They advise customers on data platforms using the full range of AWS services. They also assist with the non-technical change management work on policies, processes and people changes. At AWS, we're looking … tuning skills using Unix profiling tools and SQL - Experience leading large-scale data warehousing and analytics projects, including using AWS technologies - Redshift, S3, EC2, Data-pipeline and other big data technologies - Linux/UNIX including to process large data sets. Our inclusive culture empowers Amazonians to deliver the best results for More ❯

Role Title: Pre-Sales & Data Solution Architect Business Unit/Segment: Data Management Location: London, United Kingdom (Flexible hybrid working) Employment Type: Permanent Summary Our client, a leading global data and AI consultancy, is seeking a Pre-Sales & Data Solution Architect Lead - hybrid role to become one of their leaders in this … area in their Data Management practice with a focus on banking clients . This is a strategic, client-facing role where you will partner with senior stakeholders to modernise data platforms, accelerate cloud adoption, and drive measurable business value. The role combines technical leadership with consulting and pre-sales expertise . You will design enterprise data … as a trusted advisor, leading workshops and influencing senior stakeholders on strategy, roadmaps, and innovation. ETL/ELT & Data Engineering – Provide oversight on robust data pipeline and integration designs using SSIS, ADF, Informatica, or IBM DataStage. Data Governance & Quality – Define governance, metadata, and quality frameworks using tools such as Unity Catalog, Profisee, Alation More ❯

Role Title: Pre-Sales & Data Solution Architect Business Unit/Segment: Data Management Location: London, United Kingdom (Flexible hybrid working) Employment Type: Permanent Summary Our client, a leading global data and AI consultancy, is seeking a Pre-Sales & Data Solution Architect Lead - hybrid role to become one of their leaders in this … area in their Data Management practice with a focus on banking clients . This is a strategic, client-facing role where you will partner with senior stakeholders to modernise data platforms, accelerate cloud adoption, and drive measurable business value. The role combines technical leadership with consulting and pre-sales expertise . You will design enterprise data … as a trusted advisor, leading workshops and influencing senior stakeholders on strategy, roadmaps, and innovation. ETL/ELT & Data Engineering – Provide oversight on robust data pipeline and integration designs using SSIS, ADF, Informatica, or IBM DataStage. Data Governance & Quality – Define governance, metadata, and quality frameworks using tools such as Unity Catalog, Profisee, Alation More ❯

Role Title: Pre-Sales & Data Solution Architect Business Unit/Segment: Data Management Location: London, United Kingdom (Flexible hybrid working) Employment Type: Permanent Summary Our client, a leading global data and AI consultancy, is seeking a Pre-Sales & Data Solution Architect Lead - hybrid role to become one of their leaders in this … area in their Data Management practice with a focus on banking clients . This is a strategic, client-facing role where you will partner with senior stakeholders to modernise data platforms, accelerate cloud adoption, and drive measurable business value. The role combines technical leadership with consulting and pre-sales expertise . You will design enterprise data … as a trusted advisor, leading workshops and influencing senior stakeholders on strategy, roadmaps, and innovation. ETL/ELT & Data Engineering – Provide oversight on robust data pipeline and integration designs using SSIS, ADF, Informatica, or IBM DataStage. Data Governance & Quality – Define governance, metadata, and quality frameworks using tools such as Unity Catalog, Profisee, Alation More ❯

Role Title: Pre-Sales & Data Solution Architect Business Unit/Segment: Data Management Location: London, United Kingdom (Flexible hybrid working) Employment Type: Permanent Summary Our client, a leading global data and AI consultancy, is seeking a Pre-Sales & Data Solution Architect Lead - hybrid role to become one of their leaders in this … area in their Data Management practice with a focus on banking clients . This is a strategic, client-facing role where you will partner with senior stakeholders to modernise data platforms, accelerate cloud adoption, and drive measurable business value. The role combines technical leadership with consulting and pre-sales expertise . You will design enterprise data … as a trusted advisor, leading workshops and influencing senior stakeholders on strategy, roadmaps, and innovation. ETL/ELT & Data Engineering – Provide oversight on robust data pipeline and integration designs using SSIS, ADF, Informatica, or IBM DataStage. Data Governance & Quality – Define governance, metadata, and quality frameworks using tools such as Unity Catalog, Profisee, Alation More ❯

Location: Central London (Hybrid - 3 days in office) Salary: Up to £62,000 per annum Contract Type: Fixed Term (18 months) Are you a skilled Data Engineer looking to make a real impact in a world-renowned cultural institution? We're working with a prestigious organisation undergoing a major digital and data transformation - and they're … looking for a hands-on Data Engineer to help build the foundations of their future data estate. This is a unique opportunity to join a high-impact transformation programme at a pivotal moment. You'll play a key role in designing and building robust, scalable data pipelines that bring together data from … across the organisation - including CRM, HR, Finance, and Commercial systems - into clean, trusted, and AI-ready datasets. You'll be part of a growing data team, working closely with architects, analysts, and business stakeholders to unlock insights and enable innovation. What You'll Be Doing: Design and build robust, automated data pipelines using Azure Data More ❯

Location: Central London (Hybrid - 3 days in office) Salary: Up to £62,000 per annum Contract Type: Fixed Term (18 months) Are you a skilled Data Engineer looking to make a real impact in a world-renowned cultural institution? We're working with a prestigious organisation undergoing a major digital and data transformation - and they're … looking for a hands-on Data Engineer to help build the foundations of their future data estate. This is a unique opportunity to join a high-impact transformation programme at a pivotal moment. You'll play a key role in designing and building robust, scalable data pipelines that bring together data from … across the organisation - including CRM, HR, Finance, and Commercial systems - into clean, trusted, and AI-ready datasets. You'll be part of a growing data team, working closely with architects, analysts, and business stakeholders to unlock insights and enable innovation. What You'll Be Doing: Design and build robust, automated data pipelines using Azure Data More ❯

Location: Central London (Hybrid - 3 days in office) Salary: Up to £62,000 per annum Contract Type: Fixed Term (18 months) Are you a skilled Data Engineer looking to make a real impact in a world-renowned cultural institution? We're working with a prestigious organisation undergoing a major digital and data transformation - and they're … looking for a hands-on Data Engineer to help build the foundations of their future data estate. This is a unique opportunity to join a high-impact transformation programme at a pivotal moment. You'll play a key role in designing and building robust, scalable data pipelines that bring together data from … across the organisation - including CRM, HR, Finance, and Commercial systems - into clean, trusted, and AI-ready datasets. You'll be part of a growing data team, working closely with architects, analysts, and business stakeholders to unlock insights and enable innovation. What You'll Be Doing: Design and build robust, automated data pipelines using Azure Data More ❯

Location: Central London (Hybrid - 3 days in office) Salary: Up to £62,000 per annum Contract Type: Fixed Term (18 months) Are you a skilled Data Engineer looking to make a real impact in a world-renowned cultural institution? We're working with a prestigious organisation undergoing a major digital and data transformation - and they're … looking for a hands-on Data Engineer to help build the foundations of their future data estate. This is a unique opportunity to join a high-impact transformation programme at a pivotal moment. You'll play a key role in designing and building robust, scalable data pipelines that bring together data from … across the organisation - including CRM, HR, Finance, and Commercial systems - into clean, trusted, and AI-ready datasets. You'll be part of a growing data team, working closely with architects, analysts, and business stakeholders to unlock insights and enable innovation. What You'll Be Doing: Design and build robust, automated data pipelines using Azure Data More ❯

Location: Central London (Hybrid - 3 days in office) Salary: Up to £62,000 per annum Contract Type: Fixed Term (18 months) Are you a skilled Data Engineer looking to make a real impact in a world-renowned cultural institution? We're working with a prestigious organisation undergoing a major digital and data transformation - and they're … looking for a hands-on Data Engineer to help build the foundations of their future data estate. This is a unique opportunity to join a high-impact transformation programme at a pivotal moment. You'll play a key role in designing and building robust, scalable data pipelines that bring together data from … across the organisation - including CRM, HR, Finance, and Commercial systems - into clean, trusted, and AI-ready datasets. You'll be part of a growing data team, working closely with architects, analysts, and business stakeholders to unlock insights and enable innovation. What You'll Be Doing: Design and build robust, automated data pipelines using Azure Data More ❯

Location: Central London (Hybrid - 3 days in office) Salary: Up to £62,000 per annum Contract Type: Fixed Term (18 months) Are you a skilled Data Engineer looking to make a real impact in a world-renowned cultural institution? We're working with a prestigious organisation undergoing a major digital and data transformation - and they're … looking for a hands-on Data Engineer to help build the foundations of their future data estate. This is a unique opportunity to join a high-impact transformation programme at a pivotal moment. You'll play a key role in designing and building robust, scalable data pipelines that bring together data from … across the organisation - including CRM, HR, Finance, and Commercial systems - into clean, trusted, and AI-ready datasets. You'll be part of a growing data team, working closely with architects, analysts, and business stakeholders to unlock insights and enable innovation. What You'll Be Doing: Design and build robust, automated data pipelines using Azure Data More ❯

Senior AWS Data Engineer I am working with a leading organisation that is embarking on a greenfield initiative to revolutionise how their data is accessed and utilised. As part of this transformation, they're looking for a highly skilled Senior AWS Data Engineer to join their team and help design and build a cutting … edge data platform. This is a rare opportunity to work on a … high-impact project, collaborating with multiple business units and leveraging modern technologies such as Databricks and Snowflake. You'll be at the forefront of solving complex data pipeline challenges while contributing to strategic data architecture decisions. As part of this role, you will be responsible for some of the following areas: Design and build scalable More ❯

Senior AWS Data Engineer I am working with a leading organisation that is embarking on a greenfield initiative to revolutionise how their data is accessed and utilised. As part of this transformation, they're looking for a highly skilled Senior AWS Data Engineer to join their team and help design and build a cutting … edge data platform. This is a rare opportunity to work on a … high-impact project, collaborating with multiple business units and leveraging modern technologies such as Databricks and Snowflake. You'll be at the forefront of solving complex data pipeline challenges while contributing to strategic data architecture decisions. As part of this role, you will be responsible for some of the following areas: Design and build scalable More ❯

Senior AWS Data Engineer - Greenfield Market Data Platform | London | Vice President Level Location: London - Canary Wharf (Hybrid: 2-3 days/week in office) Salary: Up to around £120,000 + Bonus + Benefits Employment Type: Permanent Industry: Financial Services We're working with a prestigious financial services client on a greenfield data engineering … opportunity that's set to transform how market data is accessed and used across the business. This is a rare chance to join a high-impact initiative at its inception, designing and building a cutting-edge Market Data Store (MDS) platform using the latest cloud technologies. … This individual contributor role is ideal for a hands-on Senior Data Engineer who thrives in technically complex environments and enjoys solving large-scale data pipeline challenges. You'll work with tools like AWS Glue, PySpark, Iceberg, Databricks, and Snowflake , collaborating with data scientists and stakeholders across multiple business units. Key Responsibilities: Design More ❯

Senior AWS Data Engineer - Greenfield Market Data Platform | London | Vice President Level Location: London - Canary Wharf (Hybrid: 2-3 days/week in office) Salary: Up to around £120,000 + Bonus + Benefits Employment Type: Permanent Industry: Financial Services We're working with a prestigious financial services client on a greenfield data engineering … opportunity that's set to transform how market data is accessed and used across the business. This is a rare chance to join a high-impact initiative at its inception, designing and building a cutting-edge Market Data Store (MDS) platform using the latest cloud technologies. … This individual contributor role is ideal for a hands-on Senior Data Engineer who thrives in technically complex environments and enjoys solving large-scale data pipeline challenges. You'll work with tools like AWS Glue, PySpark, Iceberg, Databricks, and Snowflake , collaborating with data scientists and stakeholders across multiple business units. Key Responsibilities: Design More ❯

Data Engineer (one day a week in the central London office) Sports Analytics This role is a unique opportunity for a Data Engineer to combine technical challenges with creativity in a collaborative, high-standard work environment. By joining this team, youll not only be part of a creative and open work culture focused on innovation and … it an ideal setting for those passionate about leveraging cutting-edge technology to make meaningful contributions in the world of sports analytics. Key responsibilities for the role of Data Engineer include: Design, construct, install, test, and maintain highly scalable data management systems. Ensure systems meet requirements and industry practices for data quality and integrity. … Integrate data management technologies and software tools into existing structures. Create data tools to support sport data modelling, prediction and analytics. Work with data and analytics experts to strive for greater functionality in our data systems. As the selected Data Engineer, your background will include: 3+ years industry More ❯

Senior AWS Data Engineer - Greenfield Market Data Platform | London | Vice President Level Location: London - Canary Wharf (Hybrid: 2-3 days/week in office) Salary: Up to around £120,000 + Bonus + Benefits Employment Type: Permanent Industry: Financial Services We're working with a prestigious financial services client on a greenfield data engineering … opportunity that's set to transform how market data is accessed and used across the business. This is a rare chance to join a high-impact initiative at its inception, designing and building a cutting-edge Market Data Store (MDS) platform using the latest cloud technologies. … This individual contributor role is ideal for a hands-on Senior Data Engineer who thrives in technically complex environments and enjoys solving large-scale data pipeline challenges. You'll work with tools like AWS Glue, PySpark, Iceberg, Databricks, and Snowflake , collaborating with data scientists and stakeholders across multiple business units. Key Responsibilities: Design More ❯

Senior AWS Data Engineer - Greenfield Market Data Platform | London | Vice President Level Location: London - Canary Wharf (Hybrid: 2-3 days/week in office) Salary: Up to around £120,000 + Bonus + Benefits Employment Type: Permanent Industry: Financial Services We're working with a prestigious financial services client on a greenfield data engineering … opportunity that's set to transform how market data is accessed and used across the business. This is a rare chance to join a high-impact initiative at its inception, designing and building a cutting-edge Market Data Store (MDS) platform using the latest cloud technologies. … This individual contributor role is ideal for a hands-on Senior Data Engineer who thrives in technically complex environments and enjoys solving large-scale data pipeline challenges. You'll work with tools like AWS Glue, PySpark, Iceberg, Databricks, and Snowflake , collaborating with data scientists and stakeholders across multiple business units. Key Responsibilities: Design More ❯

Contract Opportunity: Lead Azure Data Engineer (Remote - 500/day Outside IR35) We're hiring for a Lead Azure Data Engineer to join our team on a hybrid contract based in London, supporting key finance stakeholders and transforming our data platform. This role offers the chance to shape the future of financial reporting through … cutting-edge cloud engineering and data architecture. Location : Central London (UK-based candidates only) Rate : £500/day IR35 Status : Outside IR35 Start Date : ASAP - Interviews next week Duration : 6 months with potential for extension The Role You'll lead the design and delivery of scalable data products that support financial analysis and decision-making. Working … platform Explore new technologies to improve data ingestion and self-service Essential Skills Azure Databricks : Expert in Spark (SQL, PySpark), Databricks Workflows Data Pipeline Design : Proven experience in scalable ETL/ELT development Azure Services : Data Lake, Blob Storage, Synapse Data Governance : Unity Catalog, access control, metadata management Performance More ❯

Contract Opportunity: Lead Azure Data Engineer (Remote - 500/day Outside IR35) We're hiring for a Lead Azure Data Engineer to join our team on a hybrid contract based in London, supporting key finance stakeholders and transforming our data platform. This role offers the chance to shape the future of financial reporting through … cutting-edge cloud engineering and data architecture. Location : Central London (UK-based candidates only) Rate : £500/day IR35 Status : Outside IR35 Start Date : ASAP - Interviews next week Duration : 6 months with potential for extension The Role You'll lead the design and delivery of scalable data products that support financial analysis and decision-making. Working … platform Explore new technologies to improve data ingestion and self-service Essential Skills Azure Databricks : Expert in Spark (SQL, PySpark), Databricks Workflows Data Pipeline Design : Proven experience in scalable ETL/ELT development Azure Services : Data Lake, Blob Storage, Synapse Data Governance : Unity Catalog, access control, metadata management Performance More ❯

Senior Data Engineer | London hybrid | Insurance industry l Upto 100k The Opportunity Are you passionate about building cutting-edge data platforms that drive business growth? Our client who is a prominent London Market insurer is looking for a skilled and motivated Senior Data Engineer to play a key role in the development of a … new data platform within the Azure ecosystem. This is an exciting opportunity to be at the forefront of data innovation, working for a long standing London Market Insurer. You’ll help shape the data engineering roadmap, improve data quality, and empower the business to make data-driven decisions. As a … Senior Data Engineer, you'll work closely with both technical and business stakeholders, leveraging your expertise to design, develop, and optimize a high-performance data platform built on Azure. This platform will be built to scale, incorporating the latest advancements in data intelligence while supporting strategic business objectives. Key Responsibilities 🔹 Build & Develop – Design and More ❯