101 to 125 of 276 Data Pipeline Jobs in London

Lead Data Engineer Salary/Rate : £100,000 - £110,000 per annum + Bonus Location : North London Company : Retelligence About Retelligence Retelligence is partnering with a high-growth, forward-thinking organization that specializes in digital innovation and marketing across international markets. The company is on an exciting journey, rapidly scaling its capabilities and leveraging advanced technology to deliver … cutting-edge solutions. Join a dynamic team within a business that values innovation, supports professional development, and offers exceptional career progression. The Role Retelligence is seeking a Lead Data Engineer to take a hands-on role in designing and delivering robust, real-time data pipelines and infrastructure in a Google Cloud Platform (GCP) environment. The company … Build and optimize data models for querying and analytics use cases. Develop fault-tolerant, highly available data ingestion and processing pipelines. Continuously monitor and improve pipeline performance for low-latency and high-throughput operations. Ensure data quality, integrity, and security across all systems. Implement effective monitoring, logging, and alerting mechanisms. Collaborate with product More ❯

DATA/PYTHON ENGINEER £500 - £600 per day Outside IR35 Full Remote 3-month Company Overview We are working with a fast-growing digital marketing consultancy working at the intersection of data, media, and technology. Their mission is to help brands understand their customers better and drive performance through data-driven decision-making. With … a diverse team of technologists, analysts, and marketing specialists, they deliver cutting-edge solutions across iGaming, Sports, Retail, and Financial Services. Role Overview & Responsibilities We are looking for Data Engineers to join a central data engineering function, supporting a high-priority data project following recent team changes. You'll be working end-to-end … high-quality data products for dashboards and reporting. Technical Skills & Experience We're looking for candidates who bring: Strong hands-on experience with Python for API ingestion, pipeline automation, and data transformation. Solid SQL skills with Snowflake (or similar cloud data warehouses). Experience with Airflow or other orchestration tools. Knowledge of data More ❯

and consulting company with a growing team and a solid foundation of project-based revenue. It works with leading organisations across the UK rail industry, helping them harness data to solve complex operational challenges. Data Engineers are key to this mission - building robust data infrastructure and tooling that powers insights, analytics, and software products … used across the rail network. The Role As a Data Engineer, you'll be part of a collaborative technical team, working across the data lifecycle: from designing ETL pipelines and integrating real-time data streams, to developing APIs and backend systems that deliver rail data securely and reliably. You'll work closely … with engineers, consultants, and project managers to translate real-world rail problems into scalable technical solutions. This role sits at the intersection of software engineering, data architecture, and delivery. Key Responsibilities Data Engineering & Infrastructure • Design and implement robust data pipelines (batch and real-time) for ingesting, transforming, and serving rail-related datasets. • Develop and More ❯

Lead Data Architect £75,000 - 83,000 GBP Hybrid WORKING Location: Central London, Greater London - United Kingdom Type: Permanent Lead Data Architect Locations: Leeds, Manchester, Birmingham, Bristol, Glasgow, or London (Hybrid - up to 2 days on-site per week, remainder remote) Salary: £67,000 - £76,000 (non-London)/Up to £83,000 (London) + Benefits … A respected digital consultancy is hiring a Lead Data Architect to join its growing Data Engineering practice. Known for delivering modern, human-centred services across public and private sectors, this organisation supports clients through meaningful transformation by combining data, design, and engineering. This role is ideal for experienced architects who enjoy shaping cloud-native … data platforms, working in multidisciplinary teams, and embedding data governance and architectural rigour across complex service ecosystems. The Role: As a Lead Data Architect, you'll play a key role in designing scalable, secure, and accessible data platforms that enable insight-driven decision-making. You'll work closely with engineering teams and More ❯

Head of Data Engineering | High-Growth InsurTech London - Hybrid (Once per week in office)Up to £120,000 (Maybe £130,000) + Bonus + Share Options Are you a data leader who loves building high-performing teams and scalable platforms? I'm hiring for a Head of Data Engineering at a rapidly growing InsurTech … insurance works-making it simpler, fairer, and more personalised. This is a hands-on leadership role where you'll set the strategy, guide architecture, and empower a talented data engineering team to deliver world-class data systems in a fast-paced, product-led environment. Why This Role Stands Out Competitive salary + company share options Hybrid … engineering roadmap aligned with business goals Own the architecture, reliability, and scalability of the data platform end-to-end Drive best practices in ETL/ELT pipelines, data modelling, and real-time processing Partner with stakeholders to ensure data solutions are fit for purpose and future-proof Introduce and champion modern engineering practices (testing More ❯

great opportunity for an Azure Databricks Engineer to join a renewable energy firm based in London. You'll play a hands-on role in developing and optimising modern data lakehouse solutions on Azure, while supporting critical analytics and data delivery systems. The environment encourages technical ownership, collaboration, and the chance to tackle complex cloud-native engineering … challenges. THE COMPANY This is a leading organisation within the renewable energy sector, dedicated to sustainable innovation and data-driven operations. The business is undergoing rapid digital transformation, investing in cloud-based technologies to optimise performance, forecasting, and environmental impact. With operations across multiple regions, their data initiatives play a key role in supporting clean energy … and efficient data platforms on Microsoft Azure. Your work will directly support migration initiatives, analytics enablement, and platform reliability. You'll be responsible for data pipeline development, resource deployment, and ongoing optimisation of cloud-native systems. Your responsibilities will include: Designing and implementing scalable data lakehouse architectures using Databricks on Azure. Building efficient More ❯

We are partnered with an elite systematic investment fund in the UAE seeking Data Scientists and Data Researchers with a strong interest in AI and Finance to join their growing team. This role sits at the intersection of quantitative research and data science, with a focus on advancing systematic investing through innovative data solutions. Key Responsibilities: Data Ingestion & Cleansing: Develop pipelines to integrate and clean large, complex datasets (commodities, equities, alternative data). Data Analysis: Apply statistical and machine learning techniques to uncover actionable insights for investment strategies. Data Infrastructure: Maintain and optimise automated systems for data collection, storage, and retrieval … the financial accuracy and relevance of AI-driven outputs. Key Requirements: Bachelor’s degree in Finance, Computer Science, or a related discipline 2+ years of experience in finance, data science, or a closely related field Strong understanding of macroeconomics, financial markets, and investment principles Proficiency in Python, with experience in data manipulation libraries such as pandas More ❯

We are partnered with an elite systematic investment fund in the UAE seeking Data Scientists and Data Researchers with a strong interest in AI and Finance to join their growing team. This role sits at the intersection of quantitative research and data science, with a focus on advancing systematic investing through innovative data solutions. Key Responsibilities: Data Ingestion & Cleansing: Develop pipelines to integrate and clean large, complex datasets (commodities, equities, alternative data). Data Analysis: Apply statistical and machine learning techniques to uncover actionable insights for investment strategies. Data Infrastructure: Maintain and optimise automated systems for data collection, storage, and retrieval … the financial accuracy and relevance of AI-driven outputs. Key Requirements: Bachelor’s degree in Finance, Computer Science, or a related discipline 2+ years of experience in finance, data science, or a closely related field Strong understanding of macroeconomics, financial markets, and investment principles Proficiency in Python, with experience in data manipulation libraries such as pandas More ❯

We are partnered with an elite systematic investment fund in the UAE seeking Data Scientists and Data Researchers with a strong interest in AI and Finance to join their growing team. This role sits at the intersection of quantitative research and data science, with a focus on advancing systematic investing through innovative data solutions. Key Responsibilities: Data Ingestion & Cleansing: Develop pipelines to integrate and clean large, complex datasets (commodities, equities, alternative data). Data Analysis: Apply statistical and machine learning techniques to uncover actionable insights for investment strategies. Data Infrastructure: Maintain and optimise automated systems for data collection, storage, and retrieval … the financial accuracy and relevance of AI-driven outputs. Key Requirements: Bachelor’s degree in Finance, Computer Science, or a related discipline 2+ years of experience in finance, data science, or a closely related field Strong understanding of macroeconomics, financial markets, and investment principles Proficiency in Python, with experience in data manipulation libraries such as pandas More ❯

We are partnered with an elite systematic investment fund in the UAE seeking Data Scientists and Data Researchers with a strong interest in AI and Finance to join their growing team. This role sits at the intersection of quantitative research and data science, with a focus on advancing systematic investing through innovative data solutions. Key Responsibilities: Data Ingestion & Cleansing: Develop pipelines to integrate and clean large, complex datasets (commodities, equities, alternative data). Data Analysis: Apply statistical and machine learning techniques to uncover actionable insights for investment strategies. Data Infrastructure: Maintain and optimise automated systems for data collection, storage, and retrieval … the financial accuracy and relevance of AI-driven outputs. Key Requirements: Bachelor’s degree in Finance, Computer Science, or a related discipline 2+ years of experience in finance, data science, or a closely related field Strong understanding of macroeconomics, financial markets, and investment principles Proficiency in Python, with experience in data manipulation libraries such as pandas More ❯

Location: dmg media HQ - High Street Kensington, London, W8 Position: Permanent About the Role We are seeking a motivated ML Engineer/Data Engineer to join our Data Engineering team and help evolve our data and machine learning platforms. In this role, you'll work at the intersection of machine learning and data engineering - designing and maintaining scalable data pipelines, supporting analysts and stakeholders with reliable datasets, and deploying ML models into production at scale. You'll play a critical role in ensuring both the data foundations and the ML systems are robust, efficient, and impactful.The ideal candidate is pragmatic, collaborative, and passionate about solving complex problems … with data and machine learning. Main Responsibilities Collaborate with data analysts to create ML models from prototype to production Design, build, and maintain data pipelines and feature engineering workflows Deploy, monitor, and optimize ML models and pipelines in production Implement MLOps best practices including CI/CD, experiment tracking, and model monitoring Support and More ❯

Full Stack Python Developer - Data Engineering & AI Location: UK-wide - Remote based. Employment Type: Permanent Salary: £75,000 - £85,000 per annum, dependent on experience Context Recruitment is working in partnership with a leading IT Services provider to appoint a skilled Full Stack Python Developer with demonstrable experience in data engineering . This role offers the … Responsibilities: Design, build and maintain full-stack applications using Python (Django, Flask or FastAPI) and modern JavaScript frameworks (such as React, Vue or Angular) Develop and optimise ETL pipelines , data workflows and API integrations Collaborate with data engineers and analysts to ensure efficient data architecture and accessibility Work with relational and non-relational databases … Candidate Requirements: Proven experience as a Full Stack Developer with strong proficiency in Python Solid understanding of data engineering principles , including ETL processes and data pipeline design Front-end development experience using JavaScript and associated frameworks Familiarity with cloud infrastructure , containerisation , and Kubernetes Must have a keen interest in Artificial Intelligence and Machine Learning , with More ❯

Full Stack Python Developer - Data Engineering & AI Location: UK-wide - Remote based. Employment Type: Permanent Salary: £75,000 - £85,000 per annum, dependent on experience Context Recruitment is working in partnership with a leading IT Services provider to appoint a skilled Full Stack Python Developer with demonstrable experience in data engineering . This role offers the … Responsibilities: Design, build and maintain full-stack applications using Python (Django, Flask or FastAPI) and modern JavaScript frameworks (such as React, Vue or Angular) Develop and optimise ETL pipelines , data workflows and API integrations Collaborate with data engineers and analysts to ensure efficient data architecture and accessibility Work with relational and non-relational databases … Candidate Requirements: Proven experience as a Full Stack Developer with strong proficiency in Python Solid understanding of data engineering principles , including ETL processes and data pipeline design Front-end development experience using JavaScript and associated frameworks Familiarity with cloud infrastructure , containerisation , and Kubernetes Must have a keen interest in Artificial Intelligence and Machine Learning , with More ❯

Company Description At The Stepstone Group, we have a simple yet very important mission: The right job for everyone. Using our data, platform, and technology, we create opportunities for job seekers and companies around the world to find a perfect match, in fair and equitable way. With over 20 brands across 30+ countries, we strive for fair and … the worlds leading job-tech platform. Job Description The role of Product Analytics Manager in our company is at the intersection of several disciplines: Product Analytics, Insights Analytics, Data Engineering and Data Science. Your teams projects will cover a broad range of topics, from working with engineers to implement event tracking on our platforms, all the … way to generating data insights and sometimes building predictive models, ultimately helping our Product organisation deliver great products, which solve meaningful user problems. You and your team are part of a larger Product Analytics group, and will be working closely with Product Directors, Heads of Product, Data Engineers, Data Scientists and many other stakeholders More ❯

chain leaders from a range of leading global businesses including Nike, Walmart, Unilever, Pfizer, Google, Honeywell, Volvo Cars and many others. Clients use the company's peer networking, data, research and advisory services to shape their global supply chain strategies and accelerate progress on long-term digitization and sustainability initiatives. Headquartered in London, UK, Zero100's members include … Nike, Walmart, CVS, Unilever, Pfizer, Google, Volvo Cars, Honeywell, Ecolab, McDonald's, and more. The role As a Senior Full-Stack Data Scientist at Zero100, you will play a central role in building and scaling the data foundations that power our research, product, and member-facing teams. You'll be hands-on across the full data … collaborate closely with colleagues in research, product, membership, and revenue operations, thriving in an environment where you can wear many hats, balance engineering with analytics, and help shape data practices as we scale. Responsibilities Develop and operate scalable, reliable data pipelines and infrastructure, with a focus on efficiency, maintainability, and smooth end-to-end data More ❯

CPG, and Public Services. Consolidated revenues as of 12 months ending December 2024 totaled $13.8 billion. Job Summary: We are seeking a highly skilled and experienced AWS Lead Data Engineer, who will build, and lead the development of scalable data pipelines and platforms on AWS. The ideal candidate will have deep expertise in PySpark, Glue, Athena … AWS LakeFormation, data modelling, DBT, Airflow, Docker and will be responsible for driving best practices in data engineering, governance, and DevOps. Key Responsibilities: • Lead the design and implementation of scalable, secure, and high-performance data pipelines using PySpark and AWS Glue. • Architect and manage data lakes using AWS Lake Formation, ensuring proper … access control and data governance. • Develop and optimize data models (dimensional and normalized) to support analytics and reporting. • Collaborate with analysts and business stakeholders to understand data requirements and deliver robust solutions. • Implement and maintain CI/CD pipelines for data workflows using tools like AWS CodePipeline, Git, GitHub Actions. • Ensure data More ❯

CPG, and Public Services. Consolidated revenues as of 12 months ending December 2024 totaled $13.8 billion. Job Summary: We are seeking a highly skilled and experienced AWS Lead Data Engineer, who will build, and lead the development of scalable data pipelines and platforms on AWS. The ideal candidate will have deep expertise in PySpark, Glue, Athena … AWS LakeFormation, data modelling, DBT, Airflow, Docker and will be responsible for driving best practices in data engineering, governance, and DevOps. Key Responsibilities: • Lead the design and implementation of scalable, secure, and high-performance data pipelines using PySpark and AWS Glue. • Architect and manage data lakes using AWS Lake Formation, ensuring proper … access control and data governance. • Develop and optimize data models (dimensional and normalized) to support analytics and reporting. • Collaborate with analysts and business stakeholders to understand data requirements and deliver robust solutions. • Implement and maintain CI/CD pipelines for data workflows using tools like AWS CodePipeline, Git, GitHub Actions. • Ensure data More ❯

CPG, and Public Services. Consolidated revenues as of 12 months ending December 2024 totaled $13.8 billion. Job Summary: We are seeking a highly skilled and experienced AWS Lead Data Engineer, who will build, and lead the development of scalable data pipelines and platforms on AWS. The ideal candidate will have deep expertise in PySpark, Glue, Athena … AWS LakeFormation, data modelling, DBT, Airflow, Docker and will be responsible for driving best practices in data engineering, governance, and DevOps. Key Responsibilities: • Lead the design and implementation of scalable, secure, and high-performance data pipelines using PySpark and AWS Glue. • Architect and manage data lakes using AWS Lake Formation, ensuring proper … access control and data governance. • Develop and optimize data models (dimensional and normalized) to support analytics and reporting. • Collaborate with analysts and business stakeholders to understand data requirements and deliver robust solutions. • Implement and maintain CI/CD pipelines for data workflows using tools like AWS CodePipeline, Git, GitHub Actions. • Ensure data More ❯

CPG, and Public Services. Consolidated revenues as of 12 months ending December 2024 totaled $13.8 billion. Job Summary: We are seeking a highly skilled and experienced AWS Lead Data Engineer, who will build, and lead the development of scalable data pipelines and platforms on AWS. The ideal candidate will have deep expertise in PySpark, Glue, Athena … AWS LakeFormation, data modelling, DBT, Airflow, Docker and will be responsible for driving best practices in data engineering, governance, and DevOps. Key Responsibilities: • Lead the design and implementation of scalable, secure, and high-performance data pipelines using PySpark and AWS Glue. • Architect and manage data lakes using AWS Lake Formation, ensuring proper … access control and data governance. • Develop and optimize data models (dimensional and normalized) to support analytics and reporting. • Collaborate with analysts and business stakeholders to understand data requirements and deliver robust solutions. • Implement and maintain CI/CD pipelines for data workflows using tools like AWS CodePipeline, Git, GitHub Actions. • Ensure data More ❯

leading lenders, brokers, and financial institutions make smarter decisions with data. Our reporting and MI products are at the centre of that mission, delivering the dashboards, insights, and data quality our customers rely on every day. We're looking for a Senior Product Owner - Data & Reporting to drive efficient and effective delivery of our MI and … You'll work hand-in-hand with engineers and analysts to prioritise, scope, and deliver reporting features that meet customer and internal needs. With your strong background in data and MI, you'll ensure reporting requirements are clearly defined, data quality is assured, and delivery stays on track. This role is all about turning requirements into … analysts and engineers are working to clear goals and delivery milestones. Work closely with internal stakeholders (commercial, product, analytics) to refine requirements, manage expectations, and communicate progress. Oversee data validation and quality checks, ensuring delivered dashboards and reports are accurate and reliable. Support analysts and engineers by removing blockers, clarifying requirements, and making delivery decisions quickly. Track progress More ❯

Data Consultant – Aviation Industry Specialist - £800 - £1000 Our client is seeking a data-focused consultant with deep aviation expertise (ideally within airlines, low-cost carriers, or airports). The consultant will work directly with business units and senior leadership to identify and deliver measurable operational and commercial improvements through data-driven initiatives . This … opportunity is ideal for a professional who can blend strong aviation domain knowledge with modern data engineering and analytics experience , while confidently engaging stakeholders at all levels of the organisation. Candidate Profile Industry Expertise Proven aviation experience (airlines, LCCs, airports; adjacent industries considered). Strong track record of delivering successful data products and projects. Evidence of … generating tangible value (£ millions in opportunities or savings). Understanding of flight operations, ground handling, network planning, revenue management, or MRO. Data & Technology Knowledge (~10% focus) Familiarity with Data Mesh principles and decentralised data governance. Experience with Databricks for data pipelines, analytics, and ML environments. Knowledge of data as a More ❯

Data Consultant – Aviation Industry Specialist - £800 - £1000 Our client is seeking a data-focused consultant with deep aviation expertise (ideally within airlines, low-cost carriers, or airports). The consultant will work directly with business units and senior leadership to identify and deliver measurable operational and commercial improvements through data-driven initiatives . This … opportunity is ideal for a professional who can blend strong aviation domain knowledge with modern data engineering and analytics experience , while confidently engaging stakeholders at all levels of the organisation. Candidate Profile Industry Expertise Proven aviation experience (airlines, LCCs, airports; adjacent industries considered). Strong track record of delivering successful data products and projects. Evidence of … generating tangible value (£ millions in opportunities or savings). Understanding of flight operations, ground handling, network planning, revenue management, or MRO. Data & Technology Knowledge (~10% focus) Familiarity with Data Mesh principles and decentralised data governance. Experience with Databricks for data pipelines, analytics, and ML environments. Knowledge of data as a More ❯

Data Consultant – Aviation Industry Specialist - £800 - £1000 Our client is seeking a data-focused consultant with deep aviation expertise (ideally within airlines, low-cost carriers, or airports). The consultant will work directly with business units and senior leadership to identify and deliver measurable operational and commercial improvements through data-driven initiatives . This … opportunity is ideal for a professional who can blend strong aviation domain knowledge with modern data engineering and analytics experience , while confidently engaging stakeholders at all levels of the organisation. Candidate Profile Industry Expertise Proven aviation experience (airlines, LCCs, airports; adjacent industries considered). Strong track record of delivering successful data products and projects. Evidence of … generating tangible value (£ millions in opportunities or savings). Understanding of flight operations, ground handling, network planning, revenue management, or MRO. Data & Technology Knowledge (~10% focus) Familiarity with Data Mesh principles and decentralised data governance. Experience with Databricks for data pipelines, analytics, and ML environments. Knowledge of data as a More ❯

Data Consultant – Aviation Industry Specialist - £800 - £1000 Our client is seeking a data-focused consultant with deep aviation expertise (ideally within airlines, low-cost carriers, or airports). The consultant will work directly with business units and senior leadership to identify and deliver measurable operational and commercial improvements through data-driven initiatives . This … opportunity is ideal for a professional who can blend strong aviation domain knowledge with modern data engineering and analytics experience , while confidently engaging stakeholders at all levels of the organisation. Candidate Profile Industry Expertise Proven aviation experience (airlines, LCCs, airports; adjacent industries considered). Strong track record of delivering successful data products and projects. Evidence of … generating tangible value (£ millions in opportunities or savings). Understanding of flight operations, ground handling, network planning, revenue management, or MRO. Data & Technology Knowledge (~10% focus) Familiarity with Data Mesh principles and decentralised data governance. Experience with Databricks for data pipelines, analytics, and ML environments. Knowledge of data as a More ❯

BI – SC Cleared We’re standing up a brand-new enterprise reporting solution within a large-scale government programme and we need a hands-on Technical Lead (Lead Data Engineer) to shape the technical design from day one. This role goes beyond standard dashboarding. You’ll define the end-to-end data architecture (sourcing, ETL, orchestration … data modelling) that underpins critical KPIs for monitoring performance across multiple services. Your design will directly inform how senior leaders track and improve delivery at scale. You’ll be working closely with architects, product teams, and business analysts leading a small, focused team (BA, Power BI Analyst, Delivery Manager) to turn complex, cross-departmental … data into actionable insight. What you’ll be doing Define the high-level technical design for a new KPI reporting solution. Architect the full data pipeline: sourcing, ETL/ELT, database layer, orchestration, and models. Guide the Power BI build and integration, ensuring performance and governance are baked in. Partner with senior stakeholders to align More ❯