26 to 50 of 164 Data Pipeline Jobs in Central London

Exp: 10 + yrs Job Summary The Senior Test Specialist/Architect in Cloud Automation Testing focuses on automating testing processes in ETL (Extract, Transform, Load) and Data Warehousing environments to ensure the quality and reliability of data pipelines. The role involves designing, developing, and implementing automation scripts for testing data transformations, data loading processes, and data quality verification. Key Responsibilities 1. Develop and execute automated test scripts for etl processes and data pipelines. 2. Collaborate with cross functional teams to design and implement automated testing frameworks. 3. Create test plans and test cases for etl and data warehouse testing. 4. Identify and troubleshoot issues … in data transformations and data loading processes. 5. Conduct performance testing and ensure scalability of data pipelines. 6. Implement best practices for data quality assurance in etl environments. Skill Requirements 1. Proficiency in cloud-based automation testing tools like selenium, appium, or similar. 2. Experience in automation testing of etl (extract, transform More ❯

A leading London Market Underwriter is looking for a Senior Data Business Analyst to lead the discovery, definition, and delivery of data-focused projects across a well established Change and Transformation function. This role offers the opportunity to take ownership of data processes, reporting, and business intelligence initiatives, while influencing the evolution of the … organisation’s Business Analysis framework. Key Responsibilities: Lead the initiation, discovery, and delivery of multiple data and BI projects, ensuring scope, budget, and timelines are controlled. Play a key role in developing the data business analysis function , with future opportunities to take on leadership and line management responsibilities as the team grows. Collaborate with technical and … business stakeholders to capture clear and actionable requirements using techniques such as data models, data flows, report specifications, and wireframes. Partner with the Data Engineering and BI teams to define and implement robust data pipelines, reporting solutions, and analytical models that support strategic decision-making and operational efficiency. Analyse data More ❯

A leading London Market Underwriter is looking for a Senior Data Business Analyst to lead the discovery, definition, and delivery of data-focused projects across a well established Change and Transformation function. This role offers the opportunity to take ownership of data processes, reporting, and business intelligence initiatives, while influencing the evolution of the … organisation’s Business Analysis framework. Key Responsibilities: Lead the initiation, discovery, and delivery of multiple data and BI projects, ensuring scope, budget, and timelines are controlled. Play a key role in developing the data business analysis function , with future opportunities to take on leadership and line management responsibilities as the team grows. Collaborate with technical and … business stakeholders to capture clear and actionable requirements using techniques such as data models, data flows, report specifications, and wireframes. Partner with the Data Engineering and BI teams to define and implement robust data pipelines, reporting solutions, and analytical models that support strategic decision-making and operational efficiency. Analyse data More ❯

Senior Data Engineer | Modern Data Platforms | AWS | Generative AI Location: Flexible London-based | Hybrid No sponsorship Are you passionate about shaping the future of data infrastructure and driving digital transformation? We’re partnering with a leading organisation at the forefront of innovation, leveraging cutting-edge technologies to modernise their data landscape and … build next-generation data platforms. They’re now looking for an experienced Senior Data Engineer to join a forward-thinking team and play a key role in advancing their cloud and analytics capabilities across multiple business domains, including Finance, Regulatory, Legal and Sustainability. What You’ll Be Doing Design, develop, test and support robust data … technical solutions, working closely with architects and cross-functional teams. Build sophisticated reporting solutions and drive the adoption of generative AI to enable intuitive self-service analytics. Enhance data engineering processes through automation, CI/CD pipelines and optimized ETL/ELT workflows. Experiment with new tools, technologies and approaches to support the organisation’s analytics and insights More ❯

Senior Data Engineer | Modern Data Platforms | AWS | Generative AI Location: Flexible London-based | Hybrid No sponsorship Are you passionate about shaping the future of data infrastructure and driving digital transformation? We’re partnering with a leading organisation at the forefront of innovation, leveraging cutting-edge technologies to modernise their data landscape and … build next-generation data platforms. They’re now looking for an experienced Senior Data Engineer to join a forward-thinking team and play a key role in advancing their cloud and analytics capabilities across multiple business domains, including Finance, Regulatory, Legal and Sustainability. What You’ll Be Doing Design, develop, test and support robust data … technical solutions, working closely with architects and cross-functional teams. Build sophisticated reporting solutions and drive the adoption of generative AI to enable intuitive self-service analytics. Enhance data engineering processes through automation, CI/CD pipelines and optimized ETL/ELT workflows. Experiment with new tools, technologies and approaches to support the organisation’s analytics and insights More ❯

Senior Data Engineer (GCP BigQuery Python SQL) London/WFH to £85k Are you a tech savvy Data Engineer with Google Cloud expertise combined with client facing skills? You could be joining a global technology consultancy with a range of banking, financial services and insurance clients in a senior, hands-on Data Engineer role. … As a Senior Data Engineer you will design and build end-to-end real-time data pipelines using GCP services such as BigQuery, Dataflow and Dataproc, collaborating closely with clients to define business requirements, architect fit-for-purpose data solutions and support the migration of legacy on-premise systems to cloud-native architectures. You … ll collaborate directly with clients to analyse requirements, define solutions and deliver production grade systems, leading the development of robust, well tested and fault tolerant data engineering solutions. Location/WFH: There's a hybrid work from home model with two days a week in the London, City office (or at client site in London). About you More ❯

Contract Role Remote Working £600 - £700, Inside IR35 We're partnering with a Global IT Partner who are looking for a Databricks Solution Architect to provide guidance on data architecture and design and optimise Databricks solutions. As a Databricks Solution Architect, you will be responsible for: Lead end-to-end architecture of data lakes to Azure … Databricks Design and implement data pipelines in Azure Databricks, optimizing for performance, cost, and scalability Advise of data lake migration, data modelling and pipeline optimisation Refactor and modernize existing data models to integrate with Power BI Coordinate phased migrations to ensure zero disruption to live operations Replace legacy tools by consolidating … experience with Azure Data Lake Storage Gen2, Azure Data Factory, and Azure Synapse Analytics Proficiency in SQL and Python for data transformation and pipeline logic Please note: you will be working in a US time zone. Please contract/01676545407 to discuss in more detail. ECS Recruitment Group Ltd is acting as an More ❯

the best talent to strengthen our reputation as both a design practice and an employer of choice. We are seeking a motivated Junior BI Analyst to join our data analytics team. This role is perfect for an entry-level professional looking to build expertise in business intelligence, data management, and emerging AI technologies. The successful candidate … will support data-driven decision making across the organization while developing technical skills in data analysis and visualization. Key Responsibilities: Data Management & Quality Perform data cleansing, validation, and quality assurance processes Identify and resolve data inconsistencies and anomalies Document data sources, transformations, and cleansing procedures Monitor data pipelines and ensure timely data updates Business Intelligence & Reporting Develop and maintain standard business reports and dashboards Create ad-hoc reports based on stakeholder requirements Support the creation of KPIs and performance metrics Assist in data visualization best practices implementation AI Integration & Prompt Engineering Create and optimize AI prompts for various business applications Collaborate More ❯

the best talent to strengthen our reputation as both a design practice and an employer of choice. We are seeking a motivated Junior BI Analyst to join our data analytics team. This role is perfect for an entry-level professional looking to build expertise in business intelligence, data management, and emerging AI technologies. The successful candidate … will support data-driven decision making across the organization while developing technical skills in data analysis and visualization. Key Responsibilities: Data Management & Quality Perform data cleansing, validation, and quality assurance processes Identify and resolve data inconsistencies and anomalies Document data sources, transformations, and cleansing procedures Monitor data pipelines and ensure timely data updates Business Intelligence & Reporting Develop and maintain standard business reports and dashboards Create ad-hoc reports based on stakeholder requirements Support the creation of KPIs and performance metrics Assist in data visualization best practices implementation AI Integration & Prompt Engineering Create and optimize AI prompts for various business applications Collaborate More ❯

/Engineer Location: 3 Days p/w Central London Rate: £450 Outside IR35 Duration: 6 Months + Extensions This role is responsible for migrating and developing modern data and analytics solutions using Microsoft Fabric and Azure technologies. The position supports enterprise-wide BI initiatives, delivering data models, pipelines, dashboards and reporting assets that enable data-driven decision-making. Key Responsibilities Lead and implement data migration into Microsoft Fabric for analytics and reporting use cases Design and build data models from multiple sources to generate actionable insights Maintain and optimise data warehouse platforms; identify and resolve issues Develop data pipelines using Fabric Pipelines, Azure Data … Lakehouse and Warehouse environments Implement CI/CD processes using Azure DevOps for secure, reliable deployment Technical Skills: Strong expertise in: Power BI and paginated reporting SQL and data architecture Dimensional modelling (star schema, snowflake, denormalised structures, SCD handling) DAX, Visual Studio and data transformation logic Azure Fabric, Azure Data Factory, Synapse, Data More ❯

/Engineer Location: 3 Days p/w Central London Rate: £450 Outside IR35 Duration: 6 Months + Extensions This role is responsible for migrating and developing modern data and analytics solutions using Microsoft Fabric and Azure technologies. The position supports enterprise-wide BI initiatives, delivering data models, pipelines, dashboards and reporting assets that enable data-driven decision-making. Key Responsibilities Lead and implement data migration into Microsoft Fabric for analytics and reporting use cases Design and build data models from multiple sources to generate actionable insights Maintain and optimise data warehouse platforms; identify and resolve issues Develop data pipelines using Fabric Pipelines, Azure Data … Lakehouse and Warehouse environments Implement CI/CD processes using Azure DevOps for secure, reliable deployment Technical Skills: Strong expertise in: Power BI and paginated reporting SQL and data architecture Dimensional modelling (star schema, snowflake, denormalised structures, SCD handling) DAX, Visual Studio and data transformation logic Azure Fabric, Azure Data Factory, Synapse, Data More ❯

Data Delivery and Integration Manager - Full time permanent role - Retail industry - Remote/home-based - Salary up to £75,000 plus bonus, private healthcare and more! A large and growing retail business are looking to expand their IT function with the addition of a Data Integration Manager. This role will focus on end to end data engineering and integration solutions for a company wide modernisation programme. Duties will include: - Managing the processes for data engineering and data integration in order to align to business needs - Leading a team across engineering and integration - Work to ensure the data infrastructure of the business effectively supports analytics, AL, machine learning ML … and reporting - Designing of data pipelines, ETL, ELT processes and architecture - Work with internal stakeholders and multiple 3rd parties to seamlessly integrate systems, platforms and applications To be considered suitable for this Data Engineering Manager role you will need to have experience across the following: - Team leadership and guidance - Data engineering and aligning of More ❯

Senior Data Engineer – Insurance Sector Location: Hybrid (London-based with flexible working) Salary: £85K -£100K – this is dependent on skills and experience Join a growing data team that’s transforming how the insurance industry uses data to make smarter, faster decisions. As a Senior Data Engineer , you’ll design and deliver scalable … high-performing data solutions that power analytics, reporting, and risk insights across the business. You’ll work closely with analysts, actuaries, and underwriters to build pipelines, models, and architectures that bring structure, speed, and accuracy to complex insurance data. This is a fantastic opportunity to make a real impact while shaping the future of data engineering … within a forward-thinking organisation. What You’ll Be Doing Partnering with analysts and product owners to turn business needs into scalable data solutions. Designing and maintaining robust data pipelines and models for analytics and reporting. Applying best practices in data architecture, security, and performance. Working with modern tools including Azure, Databricks, Python, Azure More ❯

Senior Data Engineer – Insurance Sector Location: Hybrid (London-based with flexible working) Salary: £85K -£100K – this is dependent on skills and experience Join a growing data team that’s transforming how the insurance industry uses data to make smarter, faster decisions. As a Senior Data Engineer , you’ll design and deliver scalable … high-performing data solutions that power analytics, reporting, and risk insights across the business. You’ll work closely with analysts, actuaries, and underwriters to build pipelines, models, and architectures that bring structure, speed, and accuracy to complex insurance data. This is a fantastic opportunity to make a real impact while shaping the future of data engineering … within a forward-thinking organisation. What You’ll Be Doing Partnering with analysts and product owners to turn business needs into scalable data solutions. Designing and maintaining robust data pipelines and models for analytics and reporting. Applying best practices in data architecture, security, and performance. Working with modern tools including Azure, Databricks, Python, Azure More ❯

has big plans for its team and the wider business. I am seeking an exceptional Engineering Manager within Analytics to lead high-performing teams at the crossroads of data engineering, analytics, and real-world business impact. This role offers the chance to shape the architecture, culture, and strategy of a modern analytics ecosystem , driving innovation, scalability, and measurable … different types of projects at one time and adaptable in leading teams and upskilling those around you. In this role, you will: Lead, mentor, and inspire analytics and data engineering teams, fostering technical excellence and a culture of innovation. Architect and optimise ELT pipelines and workflows using dbt, Airflow, SQL, and Python. Oversee the design and evolution of … are able to bring: Bring 5+ years of experience in analytics and data engineering, including proven leadership within technical teams. Demonstrate deep expertise in data pipeline architecture, schema design, and transformation frameworks. Have strong proficiency across cloud-based data platforms and scalable infrastructure. Excel at developing teams, driving process optimisation, and championing collaboration More ❯

has big plans for its team and the wider business. I am seeking an exceptional Engineering Manager within Analytics to lead high-performing teams at the crossroads of data engineering, analytics, and real-world business impact. This role offers the chance to shape the architecture, culture, and strategy of a modern analytics ecosystem , driving innovation, scalability, and measurable … different types of projects at one time and adaptable in leading teams and upskilling those around you. In this role, you will: Lead, mentor, and inspire analytics and data engineering teams, fostering technical excellence and a culture of innovation. Architect and optimise ELT pipelines and workflows using dbt, Airflow, SQL, and Python. Oversee the design and evolution of … are able to bring: Bring 5+ years of experience in analytics and data engineering, including proven leadership within technical teams. Demonstrate deep expertise in data pipeline architecture, schema design, and transformation frameworks. Have strong proficiency across cloud-based data platforms and scalable infrastructure. Excel at developing teams, driving process optimisation, and championing collaboration More ❯

Data QA Engineer We are seeking a QA Engineer with a strong background in data and analytics to help ensure the accuracy and reliability of both data pipelines and machine learning models . This role combines hands-on testing, automation, and close collaboration with engineering and data teams. About the Role You … ll take ownership of quality assurance for data pipelines; you’ll have a strong understanding of SQL to verify data consistency and accuracy. You’ll support with automated QA solutions for data pipelines and testing the outputs of ML models. Responsibilities Design, develop, and execute comprehensive test plans and test cases for data pipelines and ML models. Implement and maintain automated testing frameworks for data and ML quality assurance Perform thorough manual testing during initial development and complex feature releases Collaborate closely with Data Engineers, Data Scientists, and Product Managers Proactively identify, document, and track bugs and quality issues. Participate in code reviews and provide More ❯

Data QA Engineer We are seeking a QA Engineer with a strong background in data and analytics to help ensure the accuracy and reliability of both data pipelines and machine learning models . This role combines hands-on testing, automation, and close collaboration with engineering and data teams. About the Role You … ll take ownership of quality assurance for data pipelines; you’ll have a strong understanding of SQL to verify data consistency and accuracy. You’ll support with automated QA solutions for data pipelines and testing the outputs of ML models. Responsibilities Design, develop, and execute comprehensive test plans and test cases for data pipelines and ML models. Implement and maintain automated testing frameworks for data and ML quality assurance Perform thorough manual testing during initial development and complex feature releases Collaborate closely with Data Engineers, Data Scientists, and Product Managers Proactively identify, document, and track bugs and quality issues. Participate in code reviews and provide More ❯

MI & Data Integration Analyst (Azure/Databricks) 📍 Hybrid – Central London | 💰 £43,000 pro rata | ⏳ up to 6-month FTC (potential to extend) We’re looking for a technically strong and commercially aware MI & Data Integration Analyst to join a leading UK retail and technology business on an initial 3-month fixed-term contract . This is … a great opportunity for someone who enjoys combining hands-on data analysis, reporting, and Azure integration work — helping build robust data foundations and self-serve reporting capability for an evolving business area. 🔍 The Opportunity You’ll be supporting a small, forward-thinking team to improve how they use data — helping them access accurate and … timely MI, streamline processes, and migrate to a new Azure-based data environment. You’ll take ownership of reporting improvements, work closely with third-party partners on offshoring data processes, and support the move of key datasets into the Unity Catalog environment. 💼 Key Responsibilities Produce and enhance accurate, timely, and insightful MI reports to support business More ❯

MI & Data Integration Analyst (Azure/Databricks) 📍 Hybrid – Central London | 💰 £43,000 pro rata | ⏳ up to 6-month FTC (potential to extend) We’re looking for a technically strong and commercially aware MI & Data Integration Analyst to join a leading UK retail and technology business on an initial 3-month fixed-term contract . This is … a great opportunity for someone who enjoys combining hands-on data analysis, reporting, and Azure integration work — helping build robust data foundations and self-serve reporting capability for an evolving business area. 🔍 The Opportunity You’ll be supporting a small, forward-thinking team to improve how they use data — helping them access accurate and … timely MI, streamline processes, and migrate to a new Azure-based data environment. You’ll take ownership of reporting improvements, work closely with third-party partners on offshoring data processes, and support the move of key datasets into the Unity Catalog environment. 💼 Key Responsibilities Produce and enhance accurate, timely, and insightful MI reports to support business More ❯

Data Engineer — Hedge Fund (L/S Equity) | London A fundamentally driven long/short equity fund focused on the European Industrials sector is seeking a Data Engineer for a variety of greenfield development. Founded by a former Citadel portfolio manager, the team brings together expertise from leading global institutions such as Citadel, DE Shaw and … Millennium. They combine rigorous data analysis with industry insight to anticipate industrial cycles ahead of the market, acting decisively when conviction and evidence align. Role Overview We are seeking a highly skilled Data Engineer to design, build and maintain the core data infrastructure powering the investment and risk processes. This is a fully hands … in Python and modern ETL frameworks Experience designing and maintaining cloud-based data pipelines (e.g. AWS, Airflow, Snowflake) Deep understanding of data modelling, validation, and pipeline resilience Familiarity with financial or alternative datasets preferred More ❯

Data Engineer — Hedge Fund (L/S Equity) | London A fundamentally driven long/short equity fund focused on the European Industrials sector is seeking a Data Engineer for a variety of greenfield development. Founded by a former Citadel portfolio manager, the team brings together expertise from leading global institutions such as Citadel, DE Shaw and … Millennium. They combine rigorous data analysis with industry insight to anticipate industrial cycles ahead of the market, acting decisively when conviction and evidence align. Role Overview We are seeking a highly skilled Data Engineer to design, build and maintain the core data infrastructure powering the investment and risk processes. This is a fully hands … in Python and modern ETL frameworks Experience designing and maintaining cloud-based data pipelines (e.g. AWS, Airflow, Snowflake) Deep understanding of data modelling, validation, and pipeline resilience Familiarity with financial or alternative datasets preferred More ❯

Fimador is currently recruiting for a Principal Data Engineer to join a team developing cutting-edge … data-driven products. This role is helping the firm pioneer AI and ML in the global environment. This role is central to designing and maintaining scalable pipelines, data platforms, and integrations, while ensuring solutions meet regulatory standards and align with architectural best practices. Key Responsibilities: Build and optimise scalable data pipelines using Databricks and … backlog refinement. Promote engineering best practices including CI/CD, code reviews, and testing. Research and introduce new tools, technologies, and methodologies. Ideal experience: Experience with efficient, reliable data pipelines that improve time-to-insight. Knowledge of secure, auditable, and compliant data workflows. Know how on optimising performance and reducing costs through Spark and Databricks tuning. More ❯

Fimador is currently recruiting for a Principal Data Engineer to join a team developing cutting-edge … data-driven products. This role is helping the firm pioneer AI and ML in the global environment. This role is central to designing and maintaining scalable pipelines, data platforms, and integrations, while ensuring solutions meet regulatory standards and align with architectural best practices. Key Responsibilities: Build and optimise scalable data pipelines using Databricks and … backlog refinement. Promote engineering best practices including CI/CD, code reviews, and testing. Research and introduce new tools, technologies, and methodologies. Ideal experience: Experience with efficient, reliable data pipelines that improve time-to-insight. Knowledge of secure, auditable, and compliant data workflows. Know how on optimising performance and reducing costs through Spark and Databricks tuning. More ❯

around £500/day Starting ASAP for 6 months - likely to extend About the Role We’re partnering with a financial client that’s undertaking a large-scale data transformation, modernising legacy systems and building a unified analytics platform in the cloud. As part of this initiative, you’ll play a key role in developing and optimising a … Snowflake-based data warehouse and delivering high-quality visualisations and reporting through Power BI. The role offers the chance to work on greenfield architecture, migrate legacy SQL Server solutions and collaborate closely with senior stakeholders on complex, high-impact data initiatives. What You’ll Be Doing Design, develop and maintain data pipelines and models … using Snowflake, dbt and Python. Enhance and support Power BI dashboards and data models, ensuring performance, accuracy and user accessibility. Contribute to the migration of legacy SQL Server (SSIS/SSAS/SSRS) systems into a modern Snowflake/Power BI environment. Work collaboratively with business users to understand requirements, translate them into technical solutions and deliver robust More ❯