76 to 100 of 1,923 Data Pipeline Jobs in London

strong background in cloud technologies and infrastructure, excellent problem-solving skills, and a passion for staying up to date with the latest advancements in cloud computing. Having sound data science, programming and analytics skills are desirable to support data-driven decision-making and understand the client journey. The ideal candidate will be an engineer at heart … such Azure hands-on experience is essential. Job Responsibilities Design, size, cost, deploy, and maintain cloud infrastructure solutions based on business and technical customer requirements. Collaborate with development & data science teams to provide guidance on cloud architecture best practices, ensuring scalability, reliability, and security. Implement and manage cloud services, including compute, storage, networking, and databases. Monitor and optimise … cloud infrastructure performance, addressing issues related to scalability, availability, and efficiency. Ensure data integrity and security by implementing appropriate access controls, encryption, and backup solutions. Automate infrastructure, provisioning, configuration, and deployment processes using infrastructure-as-code (IaC) tools. Conduct regular security assessments and implement necessary measures to protect cloud-based systems and data. Collaborate with the ISx4 team More ❯

be based on your skills and experience — talk with your recruiter to learn more. Base pay range Direct message the job poster from In Technology Group Head of Data, BI, and AI @ In Technology Group. Job Title: Machine Learning Engineer Our Data Analytics business continues to grow and we are now looking for an experienced and … provide wider ranging support to insurance companies in the life and bulk annuities sector. The Team The client is a specialist and multi-disciplinary team consisting of actuaries, data scientist and developers. Our role in this mission is to pioneer advancements in the field of pensions and beyond, leveraging state-of-the-art technology to extract valuable and … designing, deploying, maintaining and refining statistical and machine learning models using Azure ML. Optimize model performance and computational efficiency. Ensure that applications run smoothly and handle large-scare data efficiently. Implement and maintain monitoring of model drifts, data-quality alerts, scheduled r-training pipelines. Data Management and Preprocessing. Collect, clean and preprocess large datasets More ❯

be based on your skills and experience — talk with your recruiter to learn more. Base pay range Direct message the job poster from In Technology Group Head of Data, BI, and AI @ In Technology Group. Job Title: Machine Learning Engineer Our Data Analytics business continues to grow and we are now looking for an experienced and … provide wider ranging support to insurance companies in the life and bulk annuities sector. The Team The client is a specialist and multi-disciplinary team consisting of actuaries, data scientist and developers. Our role in this mission is to pioneer advancements in the field of pensions and beyond, leveraging state-of-the-art technology to extract valuable and … designing, deploying, maintaining and refining statistical and machine learning models using Azure ML. Optimize model performance and computational efficiency. Ensure that applications run smoothly and handle large-scare data efficiently. Implement and maintain monitoring of model drifts, data-quality alerts, scheduled r-training pipelines. Data Management and Preprocessing. Collect, clean and preprocess large datasets More ❯

Join us at the heart of trust. As an Analytics Engineer at Trustpilot, you'll be a crucial bridge between our business and technical teams, directly influencing Analytics, Data Science, Data Engineering, and Product. You'll leverage our cutting-edge modern data stack to expand and enhance our data warehouse, empowering data-driven decisions across the organisation. This role offers the exciting opportunity to shape our data landscape and contribute to innovative analytics solutions. What You’ll Be Doing Maintain and evolve our data warehouse and business intelligence tool, providing reliable data access to users throughout Trustpilot Design, build, maintain, and rigorously monitor robust … data pipelines and transformative models using our modern data stack, including GCP, Airflow, dbt, and potentially emerging technologies like real-time streaming platforms Develop and manage reverse ETL processes to seamlessly integrate data with our commercial systems, ensuring operational efficiency Maintain and optimise our Customer Data Platform (CDP, ensuring effective data More ❯

Social network you want to login/join with: Senior Data Engineer Whitehall Resources are looking for a Senior Data Engineer. This role is hybrid working with 3 days per week on site in Hounslow, and the remainder remote working, for an initial 3 month contract. ***Inside IR35*** Job Description: As a senior data … ready code • Participate in code reviews, ensuring adherence to best practices and high engineering standards • Investigate data quality issues and implement monitoring and alterting systems to ensure pipeline reliability • Document workflows, data lineage and technical designs to support maintainability and knowledge sharing • Champion a culture of continuous improvement, experimentation and technical excellence within the team … tools (Airflow, DBT, Glue etc.) and working with large volumes of structures and semi-structured data • Demonstrated experience with SQL and Python for data manipulation, pipeline development and workflow orchestration • Strong grasp of data modelling, data warehousing concepts and performance optimization techniques • Hands-on exposure to cloud platforms, especially AWS • Experience More ❯

As a Senior Data Engineer, you will be responsible for the development of complex data sources and pipelines into our data platform (i.e. Snowflake) along with other data applications (i.e. Azure, Airflow, etc.) and automation. The Senior Data Engineer will work closely with the data, Architecture, Business Analyst … Data Stewards to integrate and align requirements, specifications and constraints of each element of the requirement. They will also need to help identify gaps in resources, technology, or capabilities required, and work with the data engineering team to identify and implement solutions where appropriate. Work type: Contract Length: initial 6 months Work structure: hybrid 2 days … requirements and design optimal pipelines to fulfil the use-cases Creating logical & physical data models to ensure data integrity is maintained CI CD pipeline creation & automation using GIT & GIT Actions Tuning and optimizing data processes Qualifications Required Qualifications: · Bachelor's degree in Computer Science or a related field. · Proven hands-on More ❯

As a Senior Data Engineer, you will be responsible for the development of complex data sources and pipelines into our data platform (i.e. Snowflake) along with other data applications (i.e. Azure, Airflow, etc.) and automation. The Senior Data Engineer will work closely with the data, Architecture, Business Analyst … Data Stewards to integrate and align requirements, specifications and constraints of each element of the requirement. They will also need to help identify gaps in resources, technology, or capabilities required, and work with the data engineering team to identify and implement solutions where appropriate. Work type: Contract Length: initial 6 months Work structure: hybrid 2 days … requirements and design optimal pipelines to fulfil the use-cases Creating logical & physical data models to ensure data integrity is maintained CI CD pipeline creation & automation using GIT & GIT Actions Tuning and optimizing data processes Qualifications Required Qualifications: · Bachelor's degree in Computer Science or a related field. · Proven hands-on More ❯

resources. We help companies make strategic, sustainable decisions by bringing transparency to markets across the world. We create a comprehensive view of commodities markets, providing companies with the data and intelligence to successfully navigate across global value chains every day. Our customers benefit from instant access to price assessments, reports and forecasts, a dedicated news channel and supply … within a dynamic market-leading global business. We have a supportive culture with a keen focus on innovation, technical excellence, career development and mutual support. Responsibilities Data Pipeline Development:Design, develop, and optimize robust data pipelines and ETL processes to ensure efficient data flow and integration Data Infrastructure Management:Manage and … possible data, to create high value analytical services Ownership of assigned data products, including data model design, end-to-end data pipeline delivery, data product quality monitoring, requirements analysis and issue resolution Enthusiastic attitude to explore and implement advanced data technologies and tools Working with other tech More ❯

Join to apply for the Lead Data Engineer/Architect role at Insurance Office of America 1 day ago Be among the first 25 applicants Join to apply for the Lead Data Engineer/Architect role at Insurance Office of America Get AI-powered advice on this job and more exclusive features. Insurance Office of America … Leading the design and implementation of data platforms, including data lakes, data warehouses, and other data processing systems. Data Pipeline Development: Designing, building, and maintaining robust data pipelines for data ingestion, transformation, and loading. Assemble large, complex data sets that meet functional/… process improvements: automating manual processes, optimizing data delivery, and assisting in re-designing infrastructure for greater scalability, etc. Build analytics tools that utilize the data pipeline to provide actionable insights into customer acquisition, operational efficiency, and other key business performance metrics. Work with stakeholders including the IT operations, automation and development teams to assist with More ❯

support EQ's transition into a stronger, more profitable, product-led business, driven by real insights and the needs of its customers. We are looking for a skilled Data Engineer to join our Data team. As a Data Engineer, you will be responsible for designing, building, and maintaining large-scale data pipelines … using Microsoft Fabric and Databricks. You will work closely with our Product and Engineering teams to ensure data flow and integration across our data ecosystem. Your expertise will help us to improve our data infrastructure, improve data quality, and enable data-driven decision-making across the organization. Core Duties and … experience in data engineering, with a focus on cloud-based data pipelines and architectures Strong expertise in Microsoft Fabric and Databricks, including data pipeline development, data warehousing, and data lake management Proficiency in Python, SQL, Scala, or Java Experience with data processing frameworks such as Apache Spark More ❯

support EQ's transition into a stronger, more profitable, product-led business, driven by real insights and the needs of its customers. We are looking for a skilled Data Engineer to join our Data team. As a Data Engineer, you will be responsible for designing, building, and maintaining large-scale data pipelines … using Microsoft Fabric and Databricks. You will work closely with our Product and Engineering teams to ensure data flow and integration across our data ecosystem. Your expertise will help us to improve our data infrastructure, improve data quality, and enable data-driven decision-making across the organization. Core Duties and … experience in data engineering, with a focus on cloud-based data pipelines and architectures Strong expertise in Microsoft Fabric and Databricks, including data pipeline development, data warehousing, and data lake management Proficiency in Python, SQL, Scala, or Java Experience with data processing frameworks such as Apache Spark More ❯

Job Description Job Title: Data Engineering Consultant Locations: London/Bristol/Manchester Salary: Competitive salary and package (Depending on level of experience) Please Note: Any offer of employment is subject to satisfactory BPSS and SC security clearance which requires 5 years continuous UK address history (typically including no periods of 30 consecutive days or more spent outside … and the impact we have with our clients and communities. It is personal to all of us." – Julie Sweet, Accenture CEO Key responsibilities Implement ETL pipelines and orchestrate data flows using batch and streaming technologies based on software engineering best practices Define, document, and iterate data mappings based on data modelling concepts and principles … at least one programming language (Python, Java, or Scala) Experience with cloud platforms (AWS, GCP, or Azure) Experience with data warehousing and lake architectures ETL/ELT pipeline development SQL and NoSQL databases Distributed computing frameworks (Spark, Kinesis, etc.) Software development best practices including CI/CD, TDD, and version control Strong understanding of data More ❯

with Google Continue with Google Continue with Google Continue with Google Direct message the job poster from Mastek Location: Westminster, London Type: Full-time Department: Quality Assurance/Data Engineering Reporting to: Data Engineering Lead Job Summary: We are looking for a skilled and detail-oriented QA Tester with hands-on experience in ETL testing (Informatica … or Microsoft Fabric) and a strong understanding of Azure Data Engineering tools such as Azure SQL Database , Azure Blob Storage , and Azure Data Factory . The ideal candidate will possess good hands-on knowledge of … writing and executing SQL queries to validate data across systems. Experience in Microsoft Fabric is considered a strong bonus. Key Responsibilities: Perform ETL/data pipeline testing using Informatica PowerCenter or IICS . Validate data ingestion , transformation , and loading processes across Azure services. Execute source-to-target data validation , data More ❯

London – Senior Data Engineer, Consultant Ekimetrics UK Job Description About Ekimetrics Ekimetrics is a leader in data science and AI-powered solutions. Since 2006, we’ve pioneered the use of AI and data science applied to unified marketing performance, measurement and holistic business optimization aligned with sustainability goals. Our goal: Combine high-impact AI … and data science solutions for sustainable business performance. At Ekimetrics, we help companies seize their data opportunities by creating new data and analytics capabilities and deploying data science solutions that power-up their revenue strategy and improve their business operations. Key figures about Ekimetrics 400+ data science experts globally 1000+ … once you’re with us) Experience building/operating highly available, distributed data pipelines for Data Science or BI teams Experience building a robust ML Pipeline or at least have a good conceptual understanding of it Experience providing technical leadership and mentoring other engineers for the best practices in the data engineering space More ❯

Data Engineer Bond street 70k ROLE OBJECTIVE This role requires a strong understanding of data architecture, data modelling, and performance optimisation. The role will contribute significantly to the company’s success. There are opportunities for professional development, such as training programs, certifications, and career advancement paths. KEY RESPONSIBILITIES Design, develop, and maintain scalable data … their data needs, scoping and implementing solutions Optimising and cleaning of data warehouse, cleaning existing codebase and creating documentation. Monitor and troubleshoot data pipeline issues Provide technical support and guidance to junior team members Provide ongoing support for business applications. Provide help with data analysis using SQL, Qube, Tableau, Hubspot and … SKILLS AND EXPERIENCE Technical Educated to Degree Level, or equivalent, in Computer Science, Information Technology or related discipline. Proven experience in software support and technical/business/data analysis. Able to communicate effectively and clearly in writing and verbally, including drafting correspondence and documents and good presentation skills. Experience with business intelligence and reporting tools like Tableau More ❯

Data Engineer Bond street 50-70k ROLE OBJECTIVE This role requires a strong understanding of data architecture, data modelling, and performance optimisation. The role will contribute significantly to the company's success. There are opportunities for professional development, such as training programs, certifications, and career advancement paths. KEY RESPONSIBILITIES Design, develop, and maintain … scalable data pipelines SQL, Azure ADF, Azure Functions, DBT Collaborate with analysts and stakeholders to understand … their data needs, scoping and implementing solutions Optimising and cleaning of data warehouse, cleaning existing codebase and creating documentation. Monitor and troubleshoot data pipeline issues Provide technical support and guidance to junior team members Provide ongoing support for business applications. Provide help with data analysis using SQL, Qube, Tableau, Hubspot and More ❯

Data Engineer Bond street 50-70k ROLE OBJECTIVE This role requires a strong understanding of data architecture, data modelling, and performance optimisation. The role will contribute significantly to the company's success. There are opportunities for professional development, such as training programs, certifications, and career advancement paths. KEY RESPONSIBILITIES Design, develop, and maintain … scalable data pipelines SQL, Azure ADF, Azure Functions, DBT Collaborate with analysts and stakeholders to understand … their data needs, scoping and implementing solutions Optimising and cleaning of data warehouse, cleaning existing codebase and creating documentation. Monitor and troubleshoot data pipeline issues Provide technical support and guidance to junior team members Provide ongoing support for business applications. Provide help with data analysis using SQL, Qube, Tableau, Hubspot and More ❯

Join to apply for the Data Analytics & Technology Lead role at LifeArc Join to apply for the Data Analytics & Technology Lead role at LifeArc Direct message the job poster from LifeArc Job Title: Data Analytics & Technology Lead Location: London Job Type: Permanent Agencies: We will only work with recruitment partners on our preferred supplier … life changing and delivering impact for patients? We want to hear from you. About the role: The purpose of this key role is to build and evolve our data platform ecosystem, aligned with business outcomes and enabling business value in line with Technology strategy. This role will lead the establishment of a new data and knowledge … data platform technologies – programming skills (e.g. SQL, Python, Terraform), cloud platforms (Azure preferred), and data warehousing products (Snowflake preferred). Familiarity with data pipeline orchestration, metadata management, containerisation (Docker), orchestration (Kubernetes) and Infrastructure as Code (e.g. Terraform) Experience in setting up and maintaining CI/CD pipelines and DevOps practices, and experience of More ❯

the first 25 applicants Direct message the job poster from Methods Location: London, hybrid working. Can be onsite one day a week Methods are seeking a SC cleared Data Engineer in London, responsible for designing, building, and maintaining scalable data pipelines and infrastructure. The ideal candidate will have expertise in Microsoft Fabric, Azure, Python, and C#. … Key Responsibilities: - Design, develop, and optimize data pipelines using Microsoft Fabric and Azure services (eg, Azure Data Factory, Azure Synapse Analytics, Azure Databricks). - Build and maintain scalable, high-performance data architectures to support analytics, reporting, and machine learning workloads. - Write clean, efficient, and maintainable code in Python and C# for data processing, transformation, and integration tasks. - Implement data ingestion, ETL/ELT processes, and data warehousing solutions. - Collaborate with data scientists, analysts, and stakeholders to understand data requirements and deliver solutions. - Ensure data quality, security, and compliance with organizational and regulatory standards. - Monitor and troubleshoot data pipelines More ❯

reimagining what's possible. Join us to help leading organizations unlock the value of technology and build a more sustainable, inclusive world. Your Role: Design and implement robust data pipelines using Python and Snowflake. Develop and maintain ETL/ELT workflows supporting analytics and reporting. Collaborate with data scientists, analysts, and business stakeholders to understand data requirements. Optimize data models and queries for performance and scalability. Ensure data quality, integrity, and security across processes. Document technical solutions and maintain best practices in data engineering. Your Profile: Proven experience in Python programming for data engineering. Hands-on expertise with Snowflake, including data modeling, performance tuning … and SQL scripting. Strong understanding of data warehousing concepts and cloud data platforms. Proficiency with version control systems like Git. Familiarity with CI/CD pipelines for development workflows. Excellent analytical, problem-solving, and communication skills. Exposure to domain-specific languages such as Slang. Experience with orchestration tools like Apache Airflow or dbt. Understanding of data More ❯

Technologies: Data Engineer ETL SQL Power BI Azure Data Warehouse We have an exciting Hybrid working opportunity for a Data Engineer to join our clients growing Data team, playing a key role in surfacing data within their fast-growing Finance business on the South Coast The role Responsibility for designing … building, and implementing a robust and scalable data warehouse that meets and functions with the Microsoft Azure Platform. This requires defining data models and ensuring data integrity and accuracy and enabling integrations with other platforms. Automating data pipelines to extract, transform, and load data from various sources into the data warehouse, identifying and implementing appropriate tools such as data bricks, designing, and testing data transformations, and ensuring data quality and consistency. Having a good understanding of the available data warehousing and data integration tools and be able to make informed decisions on the most suitable tools for each More ❯

Metyis is growing! We are looking for a Data Science Principal with +8 years of experience to join our Data and Analytics team in London. Who we are Metyis is a global and forward-thinking firm operating across a wide range of industries, developing and delivering Big Data, Digital Commerce, Marketing & Design solutions and … management oversight required to meet client expectations. Responsibilities may include overall project delivery quality, development of presentations, and managing of external relationships. Function as a sounding board for data and analytics related questions of client business leadership. Seek compelling opportunities beyond the scope of the current project. Provide guidance to more junior members of the team, while orchestrating … consulting experience or relevant industry experience, with at least 3+ years at a project lead level. Master (or Ph.D.) degree in a quantitative field with a focus on data analysis (e.g. econometrics, AI, mathematics, physics, and other exact sciences). Sharp thinking ability to be able to quickly differentiate major contributors from irrelevant details and to hierarchically structure More ❯

our people get to enjoy working on some of the most exciting projects in the world. Job Description Job Description Due to growing demand for our Infrastructure Digital - Data and Analytics services, we are seeking to recruit a data engineer with excellent technical knowledge of designing, building, and maintaining scalable data pipelines that will … enable our clients to extract value from their data assets. The Role As part of the role you will: Work with developers, managers, and business stakeholders to understand and define components of the data landscape and how it relates to the data strategy Understand and translate end-user requirements into designs and delivery plans … for effective data solutions Design, develop, execute, and maintain highly automated data pipelines using cloud technologies Analyse and resolve data quality issues Build, maintain, and support data workflows and the necessary infrastructure as part of a data and analytics delivery team Identify opportunities for improvement and promote theses through the More ❯

our people get to enjoy working on some of the most exciting projects in the world. Job Description Job Description Due to growing demand for our Infrastructure Digital - Data and Analytics services, we are seeking to recruit a data engineer with excellent technical knowledge of designing, building, and maintaining scalable data pipelines that will … enable our clients to extract value from their data assets. The Role As part of the role you will: Work with developers, managers, and business stakeholders to understand and define components of the data landscape and how it relates to the data strategy Understand and translate end-user requirements into designs and delivery plans … for effective data solutions Design, develop, execute, and maintain highly automated data pipelines using cloud technologies Analyse and resolve data quality issues Build, maintain, and support data workflows and the necessary infrastructure as part of a data and analytics delivery team Identify opportunities for improvement and promote theses through the More ❯

we simulate and predict how garments will perform on real bodies, eliminating poor fit before a single item is made. It’s a shift from subjective guesswork to data-driven decision-making that reduces returns, cuts waste, boosts conversions, and drives loyalty. We’re building the Fit Operating System for the modern fashion supply chain and we're … looking for a talented and experienced Senior Data Engineer to help power this transformation. THE ROLE: As a Senior Data Engineer at Fit Collective, you’ll build the infrastructure that fuels everything from AI-powered predictions to insights delivered directly to our customers. If you love designing clean, scalable pipelines and want to see your work … across the fashion supply chain. Over the next 12 months, we’re launching a Shopify app that automatically updates product fit recommendations and descriptions, powered directly by manufacturing data, helping brands reduce returns and improve their bottom line. Our Current Stack: Data Warehouse and Tools: DBT, Snowflake, Airbyte Languages: Python, TypeScript, Node.js , React Cloud & Infra: AWS More ❯