51 to 75 of 265 Data Pipeline Jobs in London

Job DescriptionSenior Data EngineerLondon (Hybrid Role)£80,000 per annumAbout:We are partnering with a hyper-growth financial services firm. This firm, highly regarded in the industry, is seeking a Senior Data Engineer to join their dynamic IT team and play a pivotal role in shaping … the futureRole Overview:As a Senior Data Engineer, you will contribute significantly to the development and maintenance of data infrastructure and analytics solutions, driving data-based decision-making across the organisation. Working in collaboration with cross-functional teams, including Heads of Departments, Analysts, and … Product Managers, you will be at the forefront of designing and implementing scalable, efficient, and reliable data pipelines and systems.Responsibilities:Design, build, and maintain scalable data pipelines and systems.Develop and optimize data integration, transformation, and cleansing processes.Collaborate with cross-functional teams to understand data more »

Job DescriptionJob Title: Data EngineerJob Type: Full Time, PermanentWorking location: London, HybridRole Purpose At Travelex we are developing modern data services, which will be at the heart of our relationship with our customers. Our data architecture is becoming more service-oriented and more real … problems for you to solve. Our work is fast-paced although our team remains collaborative and flexible even when delivering fast.Your role as data engineer has a wide remit: working with product owners, subject matter experts and managers across the company, to design and deliver new components of … to the traditional data engineering type!Significant hands-on experience with AWS, especially AWS services focused on data flow, pipelines, data transformation, storage and streaming.Excellent data engineering skills, for example with SQL, Python, DBT and Airflow.Good understanding of service-oriented architecture; experience more »

AWS Data Architect - Insight & Data Services - Permanent Salary guideline: £90,000 - £100,000 pa (DOE) + 10% Bonus, Pension up to 6% contributory, Health Insurance, Life Assurance etc. Base Location: UK Wide - Hybrid model – Onsite/Remote The Role: The Cloud Data Platforms team … is part of the Insights and Data Global Practice and has seen strong growth and continued success across a variety of projects and sectors. Cloud Data Platforms is the home of the Data Engineers, Platform Engineers, Solutions Architects and Business Analysts who are focused … on driving our customers digital and data transformation journey using the modern cloud platforms. We specialise on using the latest frameworks, reference architectures and technologies using AWS, Azure and GCP. Essential Skills and Experience: • AWS (e.g., Athena, Redshift, Glue, EMR) • Strong AWS Data Solution Architect Experience more »

Senior Data Engineer London (Hybrid Role) £80,000 per annum About: We are partnering with a hyper-growth financial services firm. This firm, highly regarded in the industry, is seeking a Senior Data Engineer to join their dynamic IT team and play a pivotal role in … shaping the future Role Overview: As a Senior Data Engineer, you will contribute significantly to the development and maintenance of data infrastructure and analytics solutions, driving data-based decision-making across the organisation. Working in collaboration with cross-functional teams, including Heads of Departments … Analysts, and Product Managers, you will be at the forefront of designing and implementing scalable, efficient, and reliable data pipelines and systems. Responsibilities: Design, build, and maintain scalable data pipelines and systems. Develop and optimize data integration, transformation, and cleansing processes. Collaborate with cross more »

Lead Data Engineer. What’s it all about? Information, Systems and Change and Data and Digital Solutions. You will be responsible for leading the technical design and implementation of data integration projects using Microsoft Azure and Integration Services alongside other cloud components. As the … Lead Data Engineer you will also oversee the quality, performance, and security of the data integration solutions. How you’ll make a difference? By delivering data integration solutions on their data platform, you will contribute to the continuous improvement of data … that improve the services the client provides for their customers. How you’ll do it? Lead the Engineering Team in the development of data and integration solutions leveraging the latest technologies, tools, processes, and best practice. Create and provide data products following the data more »

our award-winning team at Sparta Global. About the Job You'll become versatile in a wide array of tools across topics covering data focused coding, data visualisation, Cloud Services, Big Data and Machine learning. You will be equipped to work within a variety … of roles including but not limited to; Data Engineer, Data Analyst, Data Scientist, Information Engineer. You'll be designing, building, maintaining, and troubleshooting the data pipelines that enable organizations to store, process, and analyse their data, and ensuring the data is reliable through testing and debugging. Looking for efficiencies and optimising the data pipelines for scalability and performance will be a focus. You'll be handling and working with large sets of structured and unstructured data and will be responsible for ensuring that the more »

Job DescriptionLead Data Engineer.What’s it all about? Information, Systems and Change and Data and Digital Solutions. You will be responsible for leading the technical design and implementation of data integration projects using Microsoft Azure and Integration Services alongside other cloud components. As the … Lead Data Engineer you will also oversee the quality, performance, and security of the data integration solutions.How you’ll make a difference? By delivering data integration solutions on their data platform, you will contribute to the continuous improvement of data … decisions that improve the services the client provides for their customers.How you’ll do it?Lead the Engineering Team in the development of data and integration solutions leveraging the latest technologies, tools, processes, and best practice.Create and provide data products following the data development more »

Data Engineer/Scientist London (Hybrid) £50,000-£60,000 (+bens) 83DATA are partered with a leading Internet provider seeking the skills of a Data Engineer/Scientist. The successful candidate can expect to be tasked to connect data sources to pre-existing Azure … data infrastructure. You will lead projects with a data centric approach, for this reason, a passion and aptitude for critical analysis of large and complex datasets is vital. The role will directly report to the Head of Data - a high degree of resilience is … required to provide solutions to the executive board thus enabling data driven decisions. Responsibilites: Build data engineering pipelines to ingest, transform, store, and serve high quality data for Data Scientists Writing and tuning complex data pipelines, workflows, and data more »

Data Engineer/Scientist London (Hybrid) £50,000-£60,000 (+bens) 83DATA are partnered with a leading Internet provider seeking the skills of a Data Engineer/Scientist. The successful candidate can expect to be tasked to connect data sources to pre-existing Azure … data infrastructure. You will lead projects with a data centric approach, for this reason, a passion and aptitude for critical analysis of large and complex datasets is vital. Responsibilities: Build data engineering pipelines to ingest, transform, store, and serve high quality data for Data Scientists Writing and tuning complex data pipelines, workflows, and data structures. Identify opportunities for increasing performance, minimising costs and increase platform monitoring. Experience with components such as Azure Active Directory, Management Groups, Subscriptions, vNets, Route Tables, NSG’s and Azure more »

Senior Data Platform Engineer Up to £80,000 1 day a week in office (flexible) Xcede are delighted to be working with a well-renowned insurance company based in Hertfordshire. The company have an excellent Data Science and Machine Learning unit supported by a group of … Data Engineers who are responsible for Data Pipelines. As a bonus - Data Engineers are also able to help with ML model deployment and MLOps. Responsibilities Design and implement Data Pipelines for both structured and unstructured data. Support and improve ML infrastructure – particularly … Azure ML. Data Engineering: Guarantee data access, Data quality, ETL, Data pipelines Platform administration: Help set up accesses, infrastructure, create new storage, enforce best practices Production: Help the preparation of the data science pipelines, interact with data scientists more »

Lead Data Engineer. What's it all about? Information, Systems and Change and Data and Digital Solutions. You will be responsible for leading the technical design and implementation of data integration projects using Microsoft Azure and Integration Services alongside other cloud components. The Lead … Data Engineer will also oversee the quality, performance, and security of the data integration solutions. How you'll make a difference By delivering data integration solutions on their data platform, you will contribute to the continuous improvement of data-driven … that improve the services the client provides for their customers. How you'll do it Lead the Engineering Team in the development of data and integration solutions leveraging the latest technologies, tools, processes, and best practice. Create and provide data products following the data more »

Senior Data Engineer - Azure Paying up to 70K Hybrid: 2 days onsite About the Role My food retail business client is looking for a senior Data Engineer to join the team. My client operate on a global scale, but their Data and Analytics function … career progression opportunities as part of our technology transformation journey. Paying up to £70K Hybrid (2 days onsite) Main Duties Design and deliver data pipelines. Build and manage Azure Data Factory data flows. Ingest data into an Azure Data Lake. … Design data models for large datasets, including creating dimensional multivariate models. Utilize non-relational data storage solutions such as MongoDB. Work with SQL Server Integration Services (SSIS) and other ETL tools. Develop solutions with SSIS, SSRS, T-SQL (On-prem/Cloud). Manage NoSQL environments more »

Job Title: Data Engineer Job Type: Full Time, Permanent Working location: London, Hybrid Role Purpose At Travelex we are developing modern data services, which will be at the heart of our relationship with our customers. Our data architecture is becoming more service-oriented and … for you to solve. Our work is fast-paced although our team remains collaborative and flexible even when delivering fast. Your role as data engineer has a wide remit: working with product owners, subject matter experts and managers across the company, to design and deliver new components of … to the traditional data engineering type! Significant hands-on experience with AWS, especially AWS services focused on data flow, pipelines, data transformation, storage and streaming. Excellent data engineering skills, for example with SQL, Python, DBT and Airflow. Good understanding of service-oriented more »

New 1 year contract opportunity for a mid-senior (outside IR35) Data Engineer to join an independent Civil Society with immediate effect. Data Engineer Responsibilities Design and implement scalable data architectures for large-scale web-scraping tools. Develop and optimize data pipelines … for ETL from online platforms, including textual and unstructured data, and REST APIs. Test, validate, and secure acquired data to ensure accuracy and quality. Work closely with the AI/ML Engineer to ensure seamless integration and alignment of data and AI … models. Data Engineer Required Experience Data engineering experience (2+ years) Cloud platform proficiency (e.g., AWS, Azure, GCP) Data pipeline development (e.g., Airflow, Apache Spark) SQL proficiency, database design Visualization tools knowledge (e.g., Tableau, PowerBI, Looker) Data Engineer Application Process This is more »

My leading Energy and Commodities conglomerate are currently seeking a skilled individual with experience in Python-based data pipeline, data lake house development and an appreciation for Object orientated languages such as C# or Java, using leading orchestration tools. In this role, you'll lead … the development of robust, fully tested data pipelines in Python using cutting-edge platforms like Dagster/Airflow and apply your expertise in real-time data streaming solutions using Kafka. You'll play a key role in expanding their on prem data lake infrastructure … to help enhance support for new lines of businesses they have including Systematic trading, Trading and Shipping commodities, ensuring strong data lineage. The successful candidate will have: A strong background in data engineering using Python Strong understanding of Test Driven Development Experience building Star Schemer Databases more »

Job DescriptionJob Title: Data Operations AnalystCompany: Global Asset ManagerLocation: City of London, 3 days per week in officeSalary: £85,000 per annum + BenefitsAbout the Company:Join a prestigious global asset manager with over $200 billion in assets under management. Our firm is renowned for its innovative approach … excellence. Based in the heart of London's financial district, you'll be part of a dynamic team shaping the future of financial data operations.Role Overview:As a Data Operations Analyst, you will play a pivotal role in implementing a cutting-edge global platform aimed at … centralizing data management processes. Leveraging your expertise in R, Python, SQL, and Enterprise Data Management (EDM), you'll contribute to streamlining data workflows and optimizing data quality across the organization.Key Responsibilities:Collaborate with cross-functional teams to understand data requirements more »

Data Engineer Location: London – Hybrid Employment: Full Time All applicants must hold a DV (Developed Vetting) clearance to apply. The Opportunity: For the Ministry of Defence (MOD), maintaining National Security is a 24-hour-a-day, 365-day-a-year mission. For this reason, it partners with this … Big 4 Firm to support its multidisciplinary AI & Cloud Data Engineering Capabilities. The Role: As an AI & Cloud Data Engineering practitioner, you’ll build large-scale data pipelines and infrastructure, utilising data integration tools to optimise complex data solutions. Collaborating … with high-performing Data Science and ML Engineering teams to drive innovation and insight through AI and Data across the Defence & Security sector. Required Skills/Experience: British Passport Holder with active DV (Developed Vetting Clearance) End-to-end development with data pipelines, ETL more »

consultancy client. The successful candidate will be responsible for designing, developing, and maintaining KDB databases, as well as creating and optimizing real-time data analytics solutions. You will work closely with the technology team to enhance our data management systems, ensure data accuracy and … integrity, and help us make data-driven decisions. Key Responsibilities: Design and develop KDB databases and data processing systems to support real-time data analysis and reporting. Optimize database performance, troubleshoot issues, and implement improvements as needed. Collaborate with data analysts and … other teams to understand data requirements and develop efficient data retrieval processes. Write and maintain KDB Q scripts, ensuring code is well-documented and follows best practices. Monitor data quality and accuracy, implementing data validation and cleansing processes. Develop and maintain data more »

DATA ENGINEER LONDON HYBRID - 3 DAYS IN THE OFFICE UP TO £60K This role does not have the ability to sponsor THE COMPANY A leading provider in accommodation focused on taking advantage of their raw data sources to provide a nurturing environment facilitating personal growth and … vibrant communities THE ROLE AND RESPONSIBILITIES You will be responsible for bringing together all data sources into an Azure Cloud Data Warehouse to be leveraged for BI reporting and analytics In this role you will: Designing, implementing, and maintaining data pipelines across multiple data pipelines across multiple cloud SQL environments to ensure data integrity, security, and performance are maintained to a high standard Work with the BI team to design and build best-in-class data warehouse components Designing, building, and maintaining quality ETLs and ELTs to make more »

Data Engineer with Azure & Power BI Permanent £65-70k + Great Bens Hybrid Flexible Working - London area Talented and experienced Data Engineer/Power BI Developer, strong in both interpersonal communication skills, as well as key technical experience including; strong SQL, Azure Synapse Analytics and … Power BI (DAX & PowerQuery). This is a key technical role to join a small dynamic and friendly team, responsible for the data that enables our client to deliver premium entertainment to a Global network of over a billion airline passengers. You will … play a crucial part in the continued development and maintenance of the existing data warehouse solution, including data analysis, integration pipeline creation, SQL JSON querying and reporting using Power BI. The cloud based systems leverage data to support operational activities and inform business more »

Job Role: Azure Data Engineer Location: Central London - Hybrid/3 Days in Office Salary: £60,000 - £70,000 + 12.5% Bonus Chapman Tate is collaborating with a leading property management company that is revolutionizing the industry through innovative data-driven solutions. As they continue to … expand their operations, they are seeking a talented Azure Data Engineer with strong SQL expertise to join their dynamic team. Are you passionate about leveraging data to drive business insights and enhance operational efficiency? Do you have experience working with Azure data technologies and … so, this is an exciting opportunity to contribute to the success of a forward-thinking property management company. Responsibilities: Design, develop, and maintain data pipelines and ETL processes using Azure Data Factory. Optimize data storage and retrieval processes to ensure scalability, reliability, and performance. more »

I'm looking for a Data Engineer with skills in Databricks, Azure Data Factory and SQL Server, for a Fixed Term Contract role until August 2025. It's with an international organisation headquartered in London, and this role will require you to be in their offices … near Victoria station 3 days per week. You will be joining a small cross-functional data team, where you will focus on the development of their newly-created Azure Data Platform - this will involve designing and developing ETL pipelines using Azure Data Factory, and … utilising Databricks to support the ongoing development of the cutting-edge Data Platform. You'll seek opportunities for improvement where possible, and will collaborate with both business and technology stakeholders to ensure that the solution is fit for purpose. You will also design and develop data more »

Job Title: Data Operations Analyst Company: Global Asset Manager Location: City of London, 3 days per week in office Salary: £85,000 per annum + Benefits About the Company: Join a prestigious global asset manager with over $200 billion in assets under management. Our firm is renowned for … excellence. Based in the heart of London's financial district, you'll be part of a dynamic team shaping the future of financial data operations. Role Overview: As a Data Operations Analyst, you will play a pivotal role in implementing a cutting-edge global platform aimed … at centralizing data management processes. Leveraging your expertise in R, Python, SQL, and Enterprise Data Management (EDM), you'll contribute to streamlining data workflows and optimizing data quality across the organization. Key Responsibilities: Collaborate with cross-functional teams to understand data more »

We are currently seeking a Snr Data Engineer who has experience with the Azure Data stack to join the Business Intelligence team. The team is part of a wider team that includes the MS D365 implementation team (on-going project), Oracle ERP team and Digital Applications … to deliver MI reporting to our Finance stakeholders. Main responsibilities Collaborate with business stakeholders, process owners and BAs to understand requirements. To build data pipelines, reading data from various on-premise or cloud services to the Azure datalake. To design and build data models … suitable for reporting. To design and build Reports. Implementation of Data Quality, Data Governance, Data Security processes as part of the solution design About the Candidate Essential experience Experience with the following technologies Azure Synapse, Data Factory, Databricks, SQL Db, Datalake, Key more »

A global boutique (re)insurance broker are looking for an experienced Azure Data Engineer to join their growing team, to act as the lead Engineer on the support and development of their brand-new next-generation data platform. This is a permanent role with hybrid working … incredibly digitally-driven and are fully operating on Azure, with no legacy systems. You'll be joining their small (but growing) Cloud and Data team as their sole Data Engineer, making this a role with huge growth potential. They're currently working with an external partner … to build a brand-new Snowflake data warehouse, which brings a huge scope of work for you to lead on internally, including the development of scalable data pipelines, overseeing the integration of new data management technologies, and implementing data storage solutions. This more »