1 to 25 of 239 Data Pipeline Jobs in London

Key Responsibilities As a Data Architect, you will support the clients in their journey to become data-driven, by capturing and understanding their business needs, and defining and implementing Data and Analytics reference architectures, solutions and roadmaps. Furthermore, you will actively help our practice … in its objective to stay a Leader in Data & Analytics. • Define, guide and advise clients in developing data strategies • Engage with the clients to understand their business needs, pain points and realities • Gather, challenge and prioritize clients data requirements • Define and design innovative Data … architectures and solutions that leverage the right trade-off between clients realities, latest and disruptive technologies, current and future business needs, budget constraints, data availability and resource requirements. • Oversee the design of the data architecture and the technical architecture for the old-fashion and modern data more »

Lead Data Engineer Data Insights Permanent Office location: ITV White City Hiring range: £85,000 - £95,000 Please note: This is to assist candidates in assessing their suitability for the role. The range does not represent a longer term salary commitment for the role. For more … TV and the course of your career. Don’t just watch it. Be part of it. Join ITV. Your impact sends ripples. Our data team is responsible for helping us realise the value of data. That could mean anything from up-skilling and training our people, to increasing … Lead Data Engineer, both directly and via your team, you will be responsible for developing, maintaining, and optimising our data pipeline infrastructure using our proprietary central data platform (Symphony), which is based on Databricks. You will collaborate with cross-functional teams to design more »

the development and deployment of data pipelines. The developer will set up and own day-to-day management of the Database Automation pipeline while working closely with various stakeholders. You as a middleware developer will work at the intersection of data analysis, data … Engineer will be responsible for the complete maintenance and development of the database, including knowledge of ETL, Big Data, and pipeline deployment. You should be able to identify and solve problems creatively while keeping security and efficiency in mind. We are looking for a multi … skilled candidate with a quantitative background for the position of Data Engineer with experience in pipeline design, automation, and deployment. The Data Engineer will be responsible for efficient, reusable well-designed, and high-quality stored procedures, and automation script to support frontend and middleware to more »

Job summary The Senior Data Engineer will have direct oversight and responsibility for the development, implementation, review, and establishment of data and techniques used in the Advanced Data Analytics Portfolio (DAP). As a Senior Data Engineer, you will work with project … managers, data scientists, counter fraud experts, stakeholders, and senior management across the full extent of the NHSCFA and our wider stakeholders in pursuit of the above. Your primary responsibilities will include developing data pipelines, processing data, and preparing data suitable for analysis. … data wrangling methods, models, data structures, and data formats. oJSON, SQL, and XML oWriting robust data pipeline code that can run unattended. oMachine learning for engineering practices, such as meta driven intelligent ETL and pipeline processes. oStrong skills in relevant more »

seizing energy investment opportunities. Job Summary The job holder will be responsible for expanding and optimising our data and data pipeline architecture, as well as optimising data flow and collection for cross functional teams. The Data Engineer will support our software … developers, database architects, and Trading teams with data initiatives that will ensure optimal data delivery architecture is consistent throughout ongoing projects. This role requires the jobholder to be self-directed and comfortable supporting the data needs of multiple teams, systems and products. They will … architecture to support our next generation of products and data initiatives. Key Responsibilities: Create and maintain optimal data pipeline architecture Build analytics tools that utilize the data pipeline to provide actionable insights into customer acquisition, operational efficiency and other key more »

Job Title - Data Engineer Entity - Aventum Reports to - Head of Data Location - London/Hybrid Who We Are At The Aventum Group, we’re not one for following the usual rules. We work hard at being different and that very much extends to how we look … to advance their careers at Aventum. Role Summary You will work in various settings to build systems that collect, manage, and convert raw data into usable information for data scientists and business analysts to interpret. Your ultimate goal is to make data accessible so … lifecycle design, data ontology alignment, annotated datasets, and elastic search approaches Developing, creating and maintaining a reliable data pipeline and schemas that feed other data processes; includes both technical processes and business logic to transform data from disparate sources more »

Retail and CPG, and Public Services. Consolidated revenues as of $13 billion. Job Description - Job Description- We are seeking a talented and experienced Data Architect to join our team. The ideal candidate will have a strong background in designing and implementing data solutions in an AWS … DevOps microservices environment using Kafka and Snowflake. As a Data Architect, you will play a crucial role in defining and implementing the data architecture, data modeling, and data integration solutions for our organization. Responsibilities : Design and implement scalable and efficient data architecture solutions in an AWS environment, leveraging microservices architecture and DevOps principles. Work closely with development teams to understand data requirements and design appropriate data models and schemas for various applications and services. Implement and maintain data pipelines using Kafka for real more »

Job DescriptionThe Role The Data Pipeline Engineers ensure that our customer work with updated and correct data in our data analytics tool. You are the first to respond to data health failures on key pipelines. You will be comfortable reading code … that pipelines run effectively. Setting up and maintaining health checks on different data pipelines Respond, Triage, and debug the data pipeline when broken. Reading code and writing code changes and/or modifying the monitoring set-up Knowing and understanding how to navigate the data … providers when data is incorrect or not received on time. Communicating outages with the end users of a data pipeline What We Value Comfortable reading and writing code in Python, Pyspark and Java. Basic understanding of Spark and interested in learning the basics of more »

Azure Synapse Data Engineer - £75K - Remote (London) My client a market leading provider grants clients access to bespoke data insights and analytics specific to their needs, providing research, data and expertise to support clients and all lines of business (Advisory, Digital, Search). They … and loading of data from a wide variety of data sources using Azure Synapse tools and technologies. Data pipeline development/testing. Test the reliability and performance of each part of a system during the development phase. Identify, design, and implement process improvements … automating manual processes, optimizing data delivery, and re-designing infrastructure for greater scalability and data quality. Track pipeline stability. Monitor the overall performance and stability of the system and ensure automated parts/scheduled tasks are monitored. Manage structured and unstructured data and more »

Truth Builds Trust If our mission, values, and purpose align with your own, we would love to hear from you! Your opportunity The Data Engineer is a crucial member of the Investments Data, Technology team, tasked with developing, managing, and optimizing data pipelines within … our data ecosystem. Reporting to the Investments Data Lead, this role is instrumental in implementing the data strategy that supports front office stakeholders, systems, and clients. The Data Engineer will leverage cutting-edge technology in Snowflake, Python, SQL, and Azure to enhance … within our subsidized onsite canteen Must have skills Proven experience as a Data Engineer, with a strong background in data pipeline construction, data architecture, and data warehousing. Expertise in Snowflake, Python, SQL, and cloud-native ETL/ELT tools. Familiarity with more »

Job DescriptionAre you ready to take your data engineering expertise to new heights? This is your chance to become an integral part of a pioneering construction leader that's shaping the future!With a rich history, my client is an acclaimed construction powerhouse, renowned for iconic projects that … to surpassing expectations drives them to explore every avenue for growth, harnessing opportunities that benefit their teams, projects, communities, and industry.As an Azure Data Engineer, you will play a pivotal role in the data engineering and business intelligence journey of global and regional operations. Collaborating with … cross-functional teams, you'll lead the charge in designing and implementing scalable azure data engineering and Power BI data visualisation solutions that meet diverse business needs. Your experience as an Azure Data Engineer and BI Developer will be key to optimizing data more »

business in the coming years. We’re looking for an engineer to help us build BPP’s platforms, products and user experiences. The Data Engineering team will enable BPP to leverage data, to inform and optimise how we deliver scalable, secure and performant experiences that delight … and engage learners during their time studying with BPP and beyond, throughout their working lives. As a Data Engineer, you will report to the Data Engineering Manager. Collaborating closely with teams across BPP P&T department (Data, Product Management, Product Design & Engineering), you will … help deliver and evolve BPP Education’s vision to become more customer centred, design and data informed, to build products that meet and exceed our users’ needs across our education ecosystem. Key responsibilities Run data pipelines to extract data from various source systems (log more »

Job Purpose As a Data Engineer, you'll be at the forefront of enhancing our client's data capabilities, ensuring every bit is optimised for success. You will be collaborating closely with their visionary Data Scientists, crafting intricate data models and stunning … visualisations that will drive strategic decision-making to new heights. Your expertise in programming languages and data analysis tools will be the key to unlocking their data's full potential. Your contributions will be pivotal in driving strategic decision-making, fueling various impactful use cases. Harness … your prowess in programming languages and data analysis tools, while showcasing your formidable analytical and communication skills. Responsibilities Support the design and maintenance of data integrity checks across our client's various systems. Collaborate with the Data Scientists to understand data needs more »

Are you ready to take your data engineering expertise to new heights? This is your chance to become an integral part of a pioneering construction leader that's shaping the future! With a rich history, my client is an acclaimed construction powerhouse, renowned for iconic projects that transcend … surpassing expectations drives them to explore every avenue for growth, harnessing opportunities that benefit their teams, projects, communities, and industry. As an Azure Data Engineer, you will play a pivotal role in the data engineering and business intelligence journey of global and regional operations. Collaborating with … cross-functional teams, you'll lead the charge in designing and implementing scalable azure data engineering and Power BI data visualisation solutions that meet diverse business needs. Your experience as an Azure Data Engineer and BI Developer will be key to optimizing data more »

Engineer to spearhead their Data Engineering team in London! Interesting and varied role where you can work across the whole pipeline on a breadth of impactful projects! Great scope for career progression & you’ll be leading and managing a small team of Data … from the Data Engineering work to contribute towards the Data Science/Analytics. Work across the whole Data pipeline (Data Engineering, Data Warehousing, Analytics) Help shape the strategy and roadmap for whole Data function and contribute towards … Engineering pipeline. Preferably someone who is capable of ‘wearing many hats’ and getting hands on in other areas across the Data Engineering pipeline if necessary! Experience leading team of Data Engineers/Scientists. Min. Bachelors’ degree in STEM subject (ideally from UK university). THE more »

Senior Data Engineer - Python/Hadoop/Spark - sought by leading investment bank based in London - Hybrid - contract *inside IR35 - umbrella* Key Responsibilities: Design and implement scalable data pipelines that extract, transform and load data from various sources into the data Lakehouse. … Help teams push the boundaries of analytical insights, creating new product features using data. Develop and automate large scale, high performance data processing systems(batch and real time) to drive growth and improve product experience. Develop and maintain infrastructure tooling for our data systems. Collaborate with … software teams and business analysts to understand their data requirements and deliver quality fit for purpose data solutions. Ensure data quality and accuracy by implementing data quality checks, data contracts and data governance processes. Contribute to the ongoing more »

Data Technical Lead Location: Hybrid - London Package: Negotiable + Benefits Brown & Brown, Inc. is one of the largest insurance brokers globally having in excess of $4bn revenue in 2023 by providing risk management solutions to help protect what our customers value most. We have experienced significant growth over … size of the business by improving customer experience, growing organically and acquiring, accelerating knowledge and capabilities of teammates and using technology with purpose. Data and Data services is a key strategic enabler in Brown and Brown Europe as we look to integrate acquired business, enable them … to provide great customer service and grow through the provision of superior data services As a Technical Lead for our Data Team, you will play a critical role in shaping our data strategy, driving technical excellence, and ensuring the successful delivery of data more »

Engineer to spearhead their Data Engineering team in London! Interesting and varied role where you can work across the whole pipeline on a breadth of impactful projects! Great scope for career progression & you’ll be leading and managing a small team of Data … insights from the Data Engineering work to contribute towards the Data Science/Analytics.Work across the whole Data pipeline (Data Engineering, Data Warehousing, Analytics)Help shape the strategy and roadmap for whole Data function and contribute towards … Engineering pipeline.Preferably someone who is capable of ‘wearing many hats’ and getting hands on in other areas across the Data Engineering pipeline if necessary!Experience leading team of Data Engineers/Scientists.Min. Bachelors’ degree in STEM subject (ideally from UK university).THE BENEFITSCompetitive SalaryDiscretionary more »

be a developer in the Client Reporting Product Team within Distribution Technology. The Client Reporting Product team partners with Marketing, Reporting Operations and Data Management teams in Technology to build data integration solutions for the automation of a wide variety and high volume of reports. This … software development skills and can pick up new technologies and assist in all aspects of the team’s work such as development using data warehouse and master data vendor platform tools. You will not only need excellent problem-solving skills, but you should also be able … to see the strategic impacts of any cross-team dependencies and escalate where necessary. You will require strong teamwork skills. The distribution data space is in a transition period. Significant data assets are located in on-premise hardware and software, such as Oracle, Informatica, Autosys, and more »

Senior Data Engineer About us HENI is a technology company pioneering art markets and information. We work with world-leading artists and estates across various sectors, including printmaking, book publishing, NFTs, digital content, and art research – all underpinned by cutting-edge research in analytics, AI, and machine learning. … Our team comprises of talented web developers, data engineers, data scientists, and infrastructure specialists who thrive on solving complex problems. We enjoy developing machine learning tools and embracing the latest in infrastructure and coding methodologies. We maintain high standards of technical quality via thorough testing and … best of open-source data engineering and scientific Python toolset. Our tech stack includes Airbyte for data ingestion, Prefect for pipeline orchestration, AWS Glue for managed ETL, along with Pandas and PySpark for pipeline logic implementation. We utilize Delta Lake and PostgreSQL for data more »

management firm that utilizes long-short equities, long-only, fixed income and commodities futures strategies across its funds seek to add a experienced Data Engineer Risk Developer. They have deep expertise in trading, technology and operations and attribute their success to rigorous scientific research. As a technology and … data - driven firm, they design and build their own cutting - edge systems, from high- performance trading platforms to large- scale data analysis and compute farms. The group manages the lifecycle of data used by investment for trading, backtesting and research. Working with quants and … role (strong understanding of data engineering principles and best practices)experience in database technologies (PostgreSQL, SQL), and query optimizationwith data pipeline development and automation.coding skills (mainly with Python)knowledge (understanding of risk concepts such as VaR calculations) and good understanding of statistics with version control more »

Location : London, Must have SC Clearance Certificate Mode: Hybrid Salary: Competitive Overview: We are seeking a talented Data Engineer with expertise in Azure to join our team. The ideal candidate will be responsible for developing data pipelines, building and maintaining data warehouses and data lakes, implementing data integration and transformation solutions, ensuring data quality and integrity, and optimizing data performance. Responsibilities: Develop data pipelines for ETL/ELT (Extract, Transform, Load) processes using Azure technologies. Build and maintain scalable and efficient data warehouses and data lakes on Azure platforms. Implement data integration and transformation solutions to enable seamless data flow across systems and applications. Ensure data quality and integrity by implementing data validation and cleansing processes. Perform performance tuning and more »

management firm that utilizes long-short equities, long-only, fixed income and commodities futures strategies across its funds seek to add a experienced Data Engineer Risk Developer. They have deep expertise in trading, technology and operations and attribute their success to rigorous scientific research. As a technology and … data - driven firm, they design and build their own cutting - edge systems, from high- performance trading platforms to large- scale data analysis and compute farms. The group manages the lifecycle of data used by investment for trading, backtesting and research. Working with quants and … strong understanding of data engineering principles and best practices) experience in database technologies (PostgreSQL, SQL), and query optimization with data pipeline development and automation. coding skills (mainly with Python) knowledge (understanding of risk concepts such as VaR calculations) and good understanding of statistics with version more »

Introduction Axtria is a global provider of cloud software and data analytics to the Life Sciences industry. We help Life Sciences companies transform the product commercialization journey to drive sales growth and improve healthcare outcomes for patients. We are acutely aware that our work impacts millions of patients … Artificial Intelligence and Machine Learning. Our cloud-based platforms - Axtria DataMax™, Axtria InsightsMax™, Axtria SalesIQ™, and Axtria MarketingIQ™ - enable customers to efficiently manage data, leverage data science to deliver insights for sales and marketing planning and manage end-to-end commercial operations. With customers in over … to be the leaders we produce for the industry. Will you be next? Job Description: We are seeking a highly skilled and experienced Data Engineer to join our team. The ideal candidate will have 5-7 years of recent experience in data engineering and hold a more »

Position: Data Engineer Reporting to: Lead Data Engineer Contract type: Full Time, Permanent, Remote (with Occasional Travel) This is an exciting opportunity to join one of Europe's fastest growing healthtech startups About Cera At Cera Care, our mission is to empower people to live longer … the global leader in revolutionising affordable care for the better, for our service users and society and the perfect coordination across care practice, data, science and technology is how we’ll get there. . About Data at Cera Data has a critical role to … operational workforce, from caregiver recruitment to care delivery to our client base, and provides insights for performance improvement and cost-efficiency. Health-related data offers a comprehensive financial overview that could lead to cost savings and crucially lead to health deterioration prevention. The primary mission of our Data more »