1 to 25 of 76 Data Pipeline Jobs in the City of London

Radius is seeking a highly talented Data Architect and Governance lead for my client going through a digital transformation. Team Leadership Data Strategy Define and lead enterprise wide data strategy Design and maintain a scalable, secure data architecture Deep understanding of Data Governance, Modern data platforms: Cloud Data quality Management, metadata, lineage and data modelling Experience with Data integration, master data management (MDM) and designing scalable data pipelines Data Privacy, regulatory compliance, and security best practises BI Tools Power BI, Tableau, Looker proficient in data modelling, SQL, and cloud platforms - Azure, AWS, GCP and … their Data services Key Responsibilities Define and implement data architecture strategies in line with the Customer Journey programme Guide the data-driven decision-making process, ensuring that data is effectively utilised to inform business strategies and initiatives. Develop data plan to address data quality issues in line with More ❯

Radius is seeking a highly talented Data Architect and Governance lead for my client going through a digital transformation. Team Leadership Data Strategy Define and lead enterprise wide data strategy Design and maintain a scalable, secure data architecture Deep understanding of Data Governance, Modern data platforms: Cloud Data quality Management, metadata, lineage and data modelling Experience with Data integration, master data management (MDM) and designing scalable data pipelines Data Privacy, regulatory compliance, and security best practises BI Tools Power BI, Tableau, Looker proficient in data modelling, SQL, and cloud platforms - Azure, AWS, GCP and … their Data services Key Responsibilities Define and implement data architecture strategies in line with the Customer Journey programme Guide the data-driven decision-making process, ensuring that data is effectively utilised to inform business strategies and initiatives. Develop data plan to address data quality issues in line with More ❯

Consolidated revenues as of 12 months ending December 2024 totaled $13.8 billion. Responsibilities: Design, develop, and maintain robust and scalable ETL/ELT pipelines using Snowflake as the data warehouse. Implement data transformations and build analytical data models using dbt (data build tool). Develop and optimize complex SQL queries for data extraction, transformation, and loading in Snowflake. Orchestrate and schedule data workflows using Apache Airflow, including developing custom Python operators and DAGs. Write efficient and maintainable Python scripts for data processing, automation, and integration with various data sources and APIs. Ensure data quality, integrity, and governance throughout the data lifecycle. Collaborate with data analysts, data scientists, and business stakeholders to understand requirements and deliver data solutions. Implement and maintain CI/CD pipelines for data engineering processes, including version control with Git. Monitor data pipelines, troubleshoot issues, and optimize performance for efficiency and cost-effectiveness. Qualifications: Strong More ❯

Consolidated revenues as of 12 months ending December 2024 totaled $13.8 billion. Responsibilities: Design, develop, and maintain robust and scalable ETL/ELT pipelines using Snowflake as the data warehouse. Implement data transformations and build analytical data models using dbt (data build tool). Develop and optimize complex SQL queries for data extraction, transformation, and loading in Snowflake. Orchestrate and schedule data workflows using Apache Airflow, including developing custom Python operators and DAGs. Write efficient and maintainable Python scripts for data processing, automation, and integration with various data sources and APIs. Ensure data quality, integrity, and governance throughout the data lifecycle. Collaborate with data analysts, data scientists, and business stakeholders to understand requirements and deliver data solutions. Implement and maintain CI/CD pipelines for data engineering processes, including version control with Git. Monitor data pipelines, troubleshoot issues, and optimize performance for efficiency and cost-effectiveness. Qualifications: Strong More ❯

Media, Retail and CPG, and Public Services. Consolidated revenues as of 12 months ending December 2024 totaled $13.8 billion. Job Description: We are looking for a seasoned AWS Data Engineer to join our team and contribute to the development of scalable, secure, and high-performance data solutions. This role involves modernizing mainframe legacy systems, building robust … data pipelines, and enabling advanced analytics using AWS-native technologies. You’ll collaborate with cross-functional teams and play a key role in shaping our data strategy. Skills Must have : Proven experience in designing and maintaining scalable data lakes using AWS S3, with strong understanding of data lifecycle, access control, and performance … optimization. Hands-on experience migrating data from Mainframes to AWS, including transformation and validation of legacy formats. Proficiency in reading and processing VSAM data using Copybooks and the Cobrix library. Experience implementing data retention and archival strategies in cloud environments. Strong understanding and practical implementation of Medallion Architecture (Bronze, Silver, Gold layers) for structured More ❯

Media, Retail and CPG, and Public Services. Consolidated revenues as of 12 months ending December 2024 totaled $13.8 billion. Job Description: We are looking for a seasoned AWS Data Engineer to join our team and contribute to the development of scalable, secure, and high-performance data solutions. This role involves modernizing mainframe legacy systems, building robust … data pipelines, and enabling advanced analytics using AWS-native technologies. You’ll collaborate with cross-functional teams and play a key role in shaping our data strategy. Skills Must have : Proven experience in designing and maintaining scalable data lakes using AWS S3, with strong understanding of data lifecycle, access control, and performance … optimization. Hands-on experience migrating data from Mainframes to AWS, including transformation and validation of legacy formats. Proficiency in reading and processing VSAM data using Copybooks and the Cobrix library. Experience implementing data retention and archival strategies in cloud environments. Strong understanding and practical implementation of Medallion Architecture (Bronze, Silver, Gold layers) for structured More ❯

Role Title: Pre-Sales & Data Solution Architect Business Unit/Segment: Data Management Location: London, United Kingdom (Flexible hybrid working) Employment Type: Permanent Summary Our client, a leading global data and AI consultancy, is seeking a Pre-Sales & Data Solution Architect Lead - hybrid role to become one of their leaders in this … area in their Data Management practice with a focus on banking clients . This is a strategic, client-facing role where you will partner with senior stakeholders to modernise data platforms, accelerate cloud adoption, and drive measurable business value. The role combines technical leadership with consulting and pre-sales expertise . You will design enterprise data … as a trusted advisor, leading workshops and influencing senior stakeholders on strategy, roadmaps, and innovation. ETL/ELT & Data Engineering – Provide oversight on robust data pipeline and integration designs using SSIS, ADF, Informatica, or IBM DataStage. Data Governance & Quality – Define governance, metadata, and quality frameworks using tools such as Unity Catalog, Profisee, Alation More ❯

Role Title: Pre-Sales & Data Solution Architect Business Unit/Segment: Data Management Location: London, United Kingdom (Flexible hybrid working) Employment Type: Permanent Summary Our client, a leading global data and AI consultancy, is seeking a Pre-Sales & Data Solution Architect Lead - hybrid role to become one of their leaders in this … area in their Data Management practice with a focus on banking clients . This is a strategic, client-facing role where you will partner with senior stakeholders to modernise data platforms, accelerate cloud adoption, and drive measurable business value. The role combines technical leadership with consulting and pre-sales expertise . You will design enterprise data … as a trusted advisor, leading workshops and influencing senior stakeholders on strategy, roadmaps, and innovation. ETL/ELT & Data Engineering – Provide oversight on robust data pipeline and integration designs using SSIS, ADF, Informatica, or IBM DataStage. Data Governance & Quality – Define governance, metadata, and quality frameworks using tools such as Unity Catalog, Profisee, Alation More ❯

Senior AWS Data Engineer I am working with a leading organisation that is embarking on a greenfield initiative to revolutionise how their data is accessed and utilised. As part of this transformation, they're looking for a highly skilled Senior AWS Data Engineer to join their team and help design and build a cutting … edge data platform. This is a rare opportunity to work on a … high-impact project, collaborating with multiple business units and leveraging modern technologies such as Databricks and Snowflake. You'll be at the forefront of solving complex data pipeline challenges while contributing to strategic data architecture decisions. As part of this role, you will be responsible for some of the following areas: Design and build scalable More ❯

As a Data Specialist at this tech scaleup in London, you will analyse, collect, sort, and create data solutions that integrate across multiple products. You will steer decisions regarding data and build the models that power their products. You will take client insights and requirements and make them a reality, by developing datasets and … to drive innovation into the product development process. You will interpret your clients objectives, desires, and preferences to help the wider technical team understand the opportunities and apply data engineering responsibilities consistently. You will be working on an interesting range of projects that deliver to National Security customers and as such, you will have to hold the highest … pipelines that are tested and robust using a variety of technologies and techniques depending on the available technologies (Nifi, Spark) Build analytics tools that utilise the data pipeline to provide actionable insights into client requirements, operational efficiency, and other key business performance metrics. Complete onsite client visits and provide excellent customer support service. Problem-solve in a More ❯

VCCP are looking for a Head of Data to build our data function from the ground up, architect our MS Fabric warehouse, stand up secure pipelines across our internal systems/TM1/Sage/ATS/HubSpot, lead a lean team, and deliver high-impact dashboards for senior leadership. Role Overview: We are seeking a … hands-on Head of Data to architect and lead the data function from the ground up. This is a newly created role in a fast-paced environment, where data is central to unlocking operational efficiency, commercial insight, and strategic decision-making. The successful candidate will be responsible for building the data warehouse … integrating strategic data sources, and delivering high-impact dashboards and analytics to senior leadership. They must be confident operating in a high-pressure, fast-evolving environment and capable of influencing senior stakeholders across the business. Key Responsibilities: Build and deploy a centralised data warehouse using Microsoft Fabric, integrating key systems including (some of these are internal More ❯

VCCP are looking for a Head of Data to build our data function from the ground up, architect our MS Fabric warehouse, stand up secure pipelines across our internal systems/TM1/Sage/ATS/HubSpot, lead a lean team, and deliver high-impact dashboards for senior leadership. Role Overview: We are seeking a … hands-on Head of Data to architect and lead the data function from the ground up. This is a newly created role in a fast-paced environment, where data is central to unlocking operational efficiency, commercial insight, and strategic decision-making. The successful candidate will be responsible for building the data warehouse … integrating strategic data sources, and delivering high-impact dashboards and analytics to senior leadership. They must be confident operating in a high-pressure, fast-evolving environment and capable of influencing senior stakeholders across the business. Key Responsibilities: Build and deploy a centralised data warehouse using Microsoft Fabric, integrating key systems including (some of these are internal More ❯

evolving Lloyds managing agent thats leading the way across multiple lines, including Property, Casualty, and Specialty. As part of a major transformation, theyre building a modern, business-driven data platform from the ground up and theyre looking for a Senior Data Engineer to help shape it. About the Business This Lloyds managing agent is authorised and … regulated by the PRA and FCA, managing a high-performing syndicate. They are known for underwriting excellence and are investing heavily in data and technology to support their long-term growth. Why this role? You'll play a key role in delivering a robust, scalable data platform that unlocks better MI, improves data quality … and drives automation across the business. Youll work closely with teams across Actuarial, Finance, and IT helping to bring their data needs to life. This is a genuinely exciting opportunity to influence the data direction of a Lloyds business, not just maintain it. Key responsibilities: Design and build Data Lake and Data More ❯

VP AWS Data Engineer - Strategic Data Leadership - London - Hybrid Are you a visionary data engineering leader ready to architect the future of market data infrastructure? A global financial institution is seeking a Vice President - AWS Data Engineering to spearhead the development of a next-generation Market Data Store … initiative offers a rare opportunity to shape the platform from the ground up, leveraging technologies like Databricks, Snowflake, and AWS Glue to solve one of the most complex data challenges in banking. What makes this role stand out? Strategic ownership of a high-impact data platform initiative. Influence cross-functional teams and senior stakeholders. Drive innovation … in data architecture, governance, and analytics. Collaborate with data scientists to deploy machine learning models. Gain visibility across multiple business units and sectors. Key responsibilities: Lead the design and implementation of scalable, secure data pipelines and architectures. Define and enforce best practices in data governance, quality, and security. Guide the development of More ❯

Sahaj Software is an artisanal software engineering firm built on the values of trust, respect, curiosity, and craftsmanship, and delivering purpose-built solutions to drive data-led transformation for organisations. Our emphasis is on craft as we create purpose-built solution, leveraging Data Engineering, Platform Engineering and Data Science with a razor-sharp focus … to solve complex business and technology challenges, and provide customers with a competitive edge. As a Senior Data Engineer, you’ll feel at home if you are hands on, grounded, opinionated and passionate about delivering comprehensive data solutions that align with modern data architecture approaches. Your work will range from building a full data platform to building data pipelines or helping with data architecture and strategy. This role is ideal for those looking to have a large impact and huge scope for growth, while still being hands on with technology. We aim to allow growth without becoming “post-technical”. Responsibilities: Collaborate with Data Scientists and More ❯

Sahaj Software is an artisanal software engineering firm built on the values of trust, respect, curiosity, and craftsmanship, and delivering purpose-built solutions to drive data-led transformation for organisations. Our emphasis is on craft as we create purpose-built solution, leveraging Data Engineering, Platform Engineering and Data Science with a razor-sharp focus … to solve complex business and technology challenges, and provide customers with a competitive edge. As a Senior Data Engineer, you’ll feel at home if you are hands on, grounded, opinionated and passionate about delivering comprehensive data solutions that align with modern data architecture approaches. Your work will range from building a full data platform to building data pipelines or helping with data architecture and strategy. This role is ideal for those looking to have a large impact and huge scope for growth, while still being hands on with technology. We aim to allow growth without becoming “post-technical”. Responsibilities: Collaborate with Data Scientists and More ❯

Lead AWS Data Engineer/Architect - Databricks - London I'm working with a globally renowned financial services client that are looking for a seasons Data professional. My client are seen as leaders and pioneers within their relative field but also, a very well known house hold name. They have operations in near enough every country and … best companies to work for along side many other outstanding accolades. This is a hands on technical role. The successful applicant will be building and maintaining AWS Data pipelines and infrastructure. This is while working with cross functional teams to design best in Class data solutions across the business. As a business, they are very mature … within their data services however, there is always room for improvements so we're looking for an expert to not look to be involved in new processes but equally, help to reverse engineer the existing platform to increase efficiency and performance. This is a salaried position paying up to £130k as a base salary. It's hybrid working More ❯

remote Benefits: Healthcare and great holidays The Role As a Machine Learning Operations Engineer, you will be instrumental in designing, deploying, and maintaining machine learning systems that drive data-driven decision-making in the pensions and financial services sector. Using Azure's cloud ecosystem, you will build scalable ML workflows that support actuarial modelling, risk prediction, and client … classification, regression, ensemble methods, survival models) to improve predictions and automate core processes. 2. MLOps & Model Lifecycle Management * Own the end-to-end ML lifecycle using Azure ML pipelines, Data Factory (ADF), and ML tracking tools. * Implement CI/CD pipelines with Azure DevOps, ensuring robust version control, testing, and seamless deployment. * Monitor production ML systems for performance … data drift, and anomalies using Azure Monitor or other observability tools. * Schedule and automate model retraining pipelines to maintain performance over time. 3. Data Engineering & Preprocessing * Develop and maintain scalable ETL/ELT data pipelines using Azure Data Factory, Data Lake, and SQL. * Prepare and preprocess large datasets to ensure More ❯

Our Client has a requirement for a Data Engineer , who will be required to work on a contract basis Remote working . Role Purpose: We are … seeking an experienced Data Engineer to design, build, and optimize data solutions using the Microsoft technology stack. The role focuses on developing scalable ETL pipelines, data models, and cloud-based solutions leveraging Azure Data Factory, Azure Synapse, Azure SQL, and Azure Functions . The ideal candidate will have a proven background in … data engineering, cloud architecture, and enterprise data systems . Job Role Responsibilities: Design and implement robust ETL pipelines using Azure Data Factory . Develop and maintain data models and warehouses with Azure Synapse and Azure SQL . Build scalable data processing solutions using Azure Functions . Create interactive dashboards and More ❯

Head Of Data Position A fast-growing, digitally transformed business is seeking a founding member of their data team to build and implement the infrastructure that will support their continued growth and success. Over the past two years, this organisation has adopted a wide range of cloud-based systems across finance, operations, project management, and sales. … experienced candidate to architect the pipelines and reporting structures that bring this vision to life. Working directly with senior leadership, you’ll have the opportunity to shape the data function from the ground up, with a clear path to building and leading your own team in the future. Key Responsibilities Data Architecture & Integration (50%) Design and … implement scalable data pipelines across multiple platforms (ERP, CRM, project management tools, etc.) Develop and maintain ETL/ELT processes to unify cross-functional datasets. Build foundational datasets and collaborate with senior leadership to define data architecture. Design for future scalability and cloud warehouse implementation Data Visualisation & Storytelling (25%) Build dynamic dashboards and reports More ❯

Head Of Data Position A fast-growing, digitally transformed business is seeking a founding member of their data team to build and implement the infrastructure that will support their continued growth and success. Over the past two years, this organisation has adopted a wide range of cloud-based systems across finance, operations, project management, and sales. … experienced candidate to architect the pipelines and reporting structures that bring this vision to life. Working directly with senior leadership, you’ll have the opportunity to shape the data function from the ground up, with a clear path to building and leading your own team in the future. Key Responsibilities Data Architecture & Integration (50%) Design and … implement scalable data pipelines across multiple platforms (ERP, CRM, project management tools, etc.) Develop and maintain ETL/ELT processes to unify cross-functional datasets. Build foundational datasets and collaborate with senior leadership to define data architecture. Design for future scalability and cloud warehouse implementation Data Visualisation & Storytelling (25%) Build dynamic dashboards and reports More ❯

Data Scientist - Property London – Hybrid Up to £65,000 About the Role A leading London-based data and technology company in the property space is looking for a Data Scientist to join their growing product-focused team. This is an exciting opportunity to work as a full stack/end to end Data Scientist, driving real business impact. You’ll collaborate closely with stakeholders to build and deploy ML models and data-driven tools that are core to the company’s commercial offerings. Key Responsibilities Identify and scope opportunities where machine learning and data science can enhance product offerings. Design and run experiments to test hypotheses and … support product development. Develop robust data pipelines to support science experimentation and production. Deploy, maintain, and monitor predictive models in live environments. Communicate findings and technical concepts clearly to both technical and non-technical audiences. Contribute to agile delivery workflows and cross-functional collaboration. What We’re Looking For Solid understanding of core data science techniques More ❯

Location: Canary Wharf, London Working model: Hybrid - 3 days onsite My client, a leading financial services company, is seeking a skilled Backend Developer with a strong focus on data engineering. The successful candidate will be responsible for building and maintaining robust data pipelines that ingest data from various sources, including Snowflake, PostgreSQL databases, and … Kafka topics. This data will be transformed and fed into a real-time visualization platform. The role requires close collaboration with frontend developers to ensure accurate data processing and seamless integration. Key Responsibilities: Design and implement scalable data pipelines for real-time data processing and visualization. Integrate data from multiple … sources, including Snowflake, PostgreSQL, and Kafka. Perform data transformations to structure and optimize data for visualization. Ensure the quality and integrity of data throughout the pipeline. Develop APIs (RESTful, GraphQL, and remote procedure calls) to provide data to the frontend application. Implement secure authentication mechanisms for external customer access to data. Collaborate More ❯

We are partnered with an elite systematic investment fund in the UAE seeking Data Scientists and Data Researchers with a strong interest in AI and Finance to join their growing team. This role sits at the intersection of quantitative research and data science, with a focus on advancing systematic investing through innovative data solutions. Key Responsibilities: Data Ingestion & Cleansing: Develop pipelines to integrate and clean large, complex datasets (commodities, equities, alternative data). Data Analysis: Apply statistical and machine learning techniques to uncover actionable insights for investment strategies. Data Infrastructure: Maintain and optimise automated systems for data collection, storage, and retrieval … the financial accuracy and relevance of AI-driven outputs. Key Requirements: Bachelor’s degree in Finance, Computer Science, or a related discipline 2+ years of experience in finance, data science, or a closely related field Strong understanding of macroeconomics, financial markets, and investment principles Proficiency in Python, with experience in data manipulation libraries such as pandas More ❯

We are partnered with an elite systematic investment fund in the UAE seeking Data Scientists and Data Researchers with a strong interest in AI and Finance to join their growing team. This role sits at the intersection of quantitative research and data science, with a focus on advancing systematic investing through innovative data solutions. Key Responsibilities: Data Ingestion & Cleansing: Develop pipelines to integrate and clean large, complex datasets (commodities, equities, alternative data). Data Analysis: Apply statistical and machine learning techniques to uncover actionable insights for investment strategies. Data Infrastructure: Maintain and optimise automated systems for data collection, storage, and retrieval … the financial accuracy and relevance of AI-driven outputs. Key Requirements: Bachelor’s degree in Finance, Computer Science, or a related discipline 2+ years of experience in finance, data science, or a closely related field Strong understanding of macroeconomics, financial markets, and investment principles Proficiency in Python, with experience in data manipulation libraries such as pandas More ❯