1 to 25 of 196 Data Pipeline Jobs in London

Azure Data Engineers wanted by a London based company at the forefront of revolutionising healthcare through cutting-edge technology and data-driven solutions. Our mission is to improve patient outcomes and streamline healthcare processes by leveraging the power of data. We are looking for a talented … Azure Data Engineer to join our dynamic team and help us drive our mission forward. As an Azure Data Engineer, you will play a crucial role in designing, developing, and maintaining the data infrastructure. You will work closely with our data science … analytics, and engineering teams to ensure that data systems are robust, scalable, and secure. Your expertise will help us harness the full potential of our data to deliver innovative healthtech solutions. The business is already a well established leader within their domain. However, they like to more »

Lead Data Engineer Salary: Up to £50,000 National/Up to £60,000 London Do you like working with the latest technology and are interested in enhancing your tech abilities? We have an exciting opportunity for a highly skilled data engineer with significant experience of … snowflake. As well as being an expert in the snowflake cloud platform, you’ll have a strong background in data ingestion and integration, designing and implementing data pipelines on various technologies, data modelling, and a rounded understanding of data warehousing. Aviva believes … strongly in experimentation leading to industrialisation and we are searching for passionate, energetic data engineers who are focussed on using their skills to drive out real business value for our customers. A bit about the job: An exciting opportunity within one of our data engineering teams more »

Job Role: Azure Data Engineer Location: Central London - Hybrid/3 Days in Office Salary: £60,000 - £70,000 + 12.5% Bonus Chapman Tate is collaborating with a leading property management company that is revolutionizing the industry through innovative data-driven solutions. As they continue to … expand their operations, they are seeking a talented Azure Data Engineer with strong SQL expertise to join their dynamic team. Are you passionate about leveraging data to drive business insights and enhance operational efficiency? Do you have experience working with Azure data technologies and … so, this is an exciting opportunity to contribute to the success of a forward-thinking property management company. Responsibilities: Design, develop, and maintain data pipelines and ETL processes using Azure Data Factory. Optimize data storage and retrieval processes to ensure scalability, reliability, and performance. more »

DATA ENGINEER LONDON BASED £75,000-95,000 PER ANNUM This growing e-commerce trading company are searching for a new Data Engineer. You will be responsible for building and maintaining a new AWS serverless platform. You will also be responsible for building and maintaining data pipelines, using Python. THE COMPANY This E-commerce trading company have been growing since their formation. They are expanding their Data and Analytics team as the CEO has realised how important Data and Technology are to the business and is investing further in Machine … Learning variables. THE ROLE You will be joining an expanding Data and Analytics team, which currently has more than 20 employees. You will be responsible for building and maintaining a new AWS serverless data platform. Improve best coding practices within the Business. Stream real-time Data more »

Our client are a renowned name in the Property Sector, they are on the lookout for 2 AWS Data Engineers to come in on a contract basis. Key Skills/Requirements: Must have Python & Spark experience Must have strong AWS experience Must have Terraform experience SQL & NoSQL experience … Have built out Data Warehouses & built Data Pipelines Strong Databricks & Snowflake experience Docker, ECS, Kubernetes & Orchestration tools like Airflow or Step Functions are nice to have Contracts are running for 6 months initially, paying up to £450p/day (Outside IR35) and will be mostly remote more »

Push the boundaries of BI, building complex data pipelines and beautiful visualisations for all kinds of business users at this $50bn Hedge Fund. Every business user here understands the power and importance of data for them to do their jobs and for the business to excel. … t need to understand what a Hedge Fund does, but you’ll need to be adaptable and eager to learn. You’ll take data requirements direct from the business users and you’ll be given freedom and support to come up with your own ideas to give the … with modern tech such as Snowflake, Airflow, DBT, Kubernetes/Docker and cubeJS whilst also using Tableau CRM and Jupyter notebooks for gaining data insights. If you think there’s a better or newer tool, you’re free to use that too. The key think is that you more »

do things differently. We’re focused on our core values; using these we’ve seen significant growth across our practices and our Digital, Data and Cloud team is preparing for the next phase of expansion. This creates new opportunities for driven and skilled individuals to join one of … the fastest-growing consultancies globally. Job Description This is an exciting opportunity for an experienced developer of large-scale data solutions. You will join a team delivering a transformative cloud hosted data platform for a key Version 1 customer. The ideal candidate will have a proven … of large data sets Data quality system and process design and implementation. Integrating the end to end data pipeline to take data from source systems to target data repositories ensuring the quality and consistency of data is more »

Are you passionate about leveraging cutting-edge data technologies to drive business success? Do you thrive in dynamic environments where innovation and collaboration are valued? If so, we have an exciting opportunity for you! Position: Lead Data Engineer Responsibilities: Lead Data Management and Architecture … Spearhead the design and implementation of robust data solutions across the organization, ensuring scalability and efficiency. Next Gen Data Platform Leadership: Act as the primary data engineer supporting the development of our next-generation data platform, collaborating closely with third-party vendors … to build a highly scalable and robust data hub. Develop and Optimize Data Pipelines: Design, construct, install, test, and maintain highly scalable data management systems, ensuring they meet business requirements and industry best practices. Data Integration and Storage Solutions: Implement data more »

Principal Data Engineer London - £90k - £100k + Package 2 days per week in Office/Client Site ***Unfortuantely due to the nature of this role we cannot accept candidates who hold a Visa*** My client is seeking a Principal Data Engineer to join their dynamic team. … As a Principal Data Engineer, you'll work in multidisciplinary teams to create, support, and maintain user-focused digital solutions that deliver real value to everyone. About the Role Our Principal Data Engineers are leaders in their field. In this role, you'll serve as a … principal consultant, offering technical expertise in data engineering to both our internal team and clients. You'll guide the technical deliverables of the team, nurture client relationships, and be passionate about developing and mentoring others. Key Responsibilities Serve as a technical subject matter expert within the engineering community more »

Principal Data Engineer Location: London based Salary: £90k - 100k A Principal Data Engineer has the opportunity to join one of the United Kingdom’s leading digital innovation and design hubs that are renowned for building digital products and services in the public sector. In this role … is also desirable. Solid experience of at least one Cloud provider such as AWS, Azure or GCP. Strong experience of working with different data formats, e.g., CSV, JSON and XML Building reliable Data Pipelines Working in an Agile Environment Test-Driven Development and/or Behaviour more »

crypto technology company that specialise in building trading and MEV-related infrastructure and services, based in London. They are looking to add a data engineer to their tech function. Background in or strong understanding of Crypto and/or the Miner Extractable Value (MEV) is needed - please only … apply if you have good knowledge of the space. The team are focusing on designing, building, and optimizing their data infrastructure. Previous experience in low latency systems, algorithmic trading, or MEV-related systems is a plus but not required. Responsibilities: Oversee the complete process of gathering and analyzing … telemetry data to derive insights regarding company performance Support the development team in resolving issues by using data gathered from various sources Designing, developing, and maintaining scalable and reliable data pipelines and ETL processes Building and managing data monitoring tools to ensure more »

London & Edinburgh Work Mode: Mostly WFH, candidates should be open to travel as required. Job Description: We are seeking a highly skilled Senior Data Engineer with expertise in SAS to join our team and contribute to the transformation of our data pipelines. The successful candidate will … play a crucial role in rewriting existing pipelines from SAS, Info, pearl, and shell scripts to Python and DBT-based solutions, ultimately loading data into Snowflake and Salesforce on Azure. Key Responsibilities: Lead the conversion of existing SAS-based modules to Python-based solutions. Design, develop, and implement … Python and DBT-based pipelines for efficient data transformation and loading. Collaborate with cross-functional teams to understand requirements and ensure successful pipeline migration. Utilize expertise in SAS, pearl, and shell scripts to seamlessly transition to Python and DBT technologies. Implement best practices for data more »

Leading ecommerce client are now searching for a Senior Data Engineer to contribute towards the delivery of their data strategy. This engineer will shape our client’s data function, delivering end to end solutions for an array of customer data projects. The … role: Designing data pipelines, managing data warehouses, and implementing complex ETL processes. Work closely with data scientists, analysts, and stakeholders to optimise our client’s infrastructure. Optimise data driven products, personalisation, reporting and overall business success. Manage the whole data lifecycle (End to End). Ensure the reliability and quality of data. Key Requirements: Strong experience designing data pipelines/warehouses using AWS and Snowflake. Exposure to big data technologies such as Kafka, Spark, or Hadoop. Solid experience with Snowflake, including performance optimisation and more »

Data Engineer On-Premise/Excel/SQL We have partnered with a leading client based in Central London who are looking to increase their data engineering team. Role: You will be working on a large volume of various financial reporting data sources. The … client here is using on-premise databases including Excel services & SQL. You will be working to gather data from these various sources for reporting, and creating the data pipelines. Experience: Data engineering experience within financial services (Asset Management/Harrington Starr) Experience working to … gather data from on-premise sources such as Excel, SQL and Oracle. If this is of interest please reach out. archie.odonovan@harringtonstarr.com more »

Position: Data Engineer Reporting to: Lead Data Engineer Contract type: Full Time, Permanent, Remote (with Occasional Travel) This is an exciting opportunity to join one of Europe's fastest growing healthtech startups About Cera At Cera Care, our mission is to empower people to live longer … the global leader in revolutionising affordable care for the better, for our service users and society and the perfect coordination across care practice, data, science and technology is how we’ll get there. . About Data at Cera Data has a critical role to … operational workforce, from caregiver recruitment to care delivery to our client base, and provides insights for performance improvement and cost-efficiency. Health-related data offers a comprehensive financial overview that could lead to cost savings and crucially lead to health deterioration prevention. The primary mission of our Data more »

knowledge on which to develop their careers and progress in the direction they choose. The Role: We are looking for an experienced Senior Data Engineer to join our team here at Solirius. You will be working as part of a team, developing and delivering exciting projects with a … fantastic team of technology experts. You will be responsible for designing, developing, and maintaining data pipelines and systems that enable data analysis and machine learning. You will also collaborate with data scientists, analysts, and other stakeholders to ensure data quality and reliability. … Self motivated, enthusiastic and committed The ideal candidates will share our values and be aligned with our ways of working Key Responsibilities: Develop Data Engineering solutions for our clients/projects Design and build data models, schemas to support business requirements Develop and maintain data more »

Senior Data Engineer (Inside IR35) - £700 to 825 per day Location: Remote/Hybrid (London) Rate: £825 per day Contract: Inside IR35 Are you an experienced Data Engineer with a strong background in banking? Our client are seeking Senior Data Engineers to contribute to … innovative data projects within a leading financial institution. Key Responsibilities: Data Engineering: Design and implement data models, ETL processes, and data warehousing solutions. Programming: Utilize Python, Java, Scala, or GoLang to build and optimize data pipelines. Distributed Processing: Work with … Hadoop, Spark, and other platforms for large-scale data processing. Real-Time Data Streaming: Develop and manage pipelines using CDC, Kafka, and Apache Spark. Database Management: Handle SQL databases like Oracle, MySQL, or PostgreSQL. Data Governance: Ensure data quality, security, and compliance more »

Senior Data Engineer. Java or Python and AWS. Investment Management. 12 month rolling Fixed Term Contract. £135,000 - £145,000 + 15% Guaranteed Bonus paid monthly + Benefits including health, holidays and 13% pension. Cannot Sponsor. CANNOT SPONSOR The Data Office team at my client is … view to the table—because that is how they help their clients succeed. You will be an individual contributor on the Analytics and Data Platform engineering team responsible for architecting, building, testing, and maintaining the data platform. You will be responsible for: Contributing to the … platform’s architectural design Development and testing of the integration, modelling, data persistence and querying tools and analytical systems Data pipeline implementation, maintenance and testing Metadata management processes and tools Monitoring the overall performance and stability of the pipelines Implementing data curation, metadata more »

Truth Builds Trust If our mission, values, and purpose align with your own, we would love to hear from you! Your opportunity The Data Engineer is a crucial member of the Investments Data, Technology team, tasked with developing, managing, and optimizing data pipelines within … our data ecosystem. Reporting to the Investments Data Lead, this role is instrumental in implementing the data strategy that supports front office stakeholders, systems, and clients. The Data Engineer will leverage cutting-edge technology in Snowflake, Python, SQL, and Azure to enhance … within our subsidized onsite canteen Must have skills Proven experience as a Data Engineer, with a strong background in data pipeline construction, data architecture, and data warehousing. Expertise in Snowflake, Python, SQL, and cloud-native ETL/ELT tools. Familiarity with more »

promises exciting, engaging and rewarding projects for those that are keen to develop and build a successful career. The Role The Data Pipeline Engineers ensure that our customer work with updated and correct data in our data analytics tool. You are the first … schedules so that pipelines run effectively. up and maintaining health checks on different data pipelines Triage, and debug the data pipeline when broken. code and writing code changes and/or modifying the monitoring set-up and understanding how to navigate the data … providers when data is incorrect or not received on time. outages with the end users of a data pipeline What We Value reading and writing code in Python, Pyspark and Java. understanding of Spark and interested in learning the basics of tuning Spark more »

consultancy client. The successful candidate will be responsible for designing, developing, and maintaining KDB databases, as well as creating and optimizing real-time data analytics solutions. You will work closely with the technology team to enhance our data management systems, ensure data accuracy and … integrity, and help us make data-driven decisions. Key Responsibilities: Design and develop KDB databases and data processing systems to support real-time data analysis and reporting. Optimize database performance, troubleshoot issues, and implement improvements as needed. Collaborate with data analysts and … other teams to understand data requirements and develop efficient data retrieval processes. Write and maintain KDB Q scripts, ensuring code is well-documented and follows best practices. Monitor data quality and accuracy, implementing data validation and cleansing processes. Develop and maintain data more »

trading strategies using modern C++ (17/20/23) Work closely with quantitative researchers to improve the profitability of strategies Build robust data pipelines to assist analysis for large datasets Develop infrastructure for trading services to convert research ideas into production Experience with large-scale distributed computing more »

Data Architect (Azure) Our client, a leading analytics and consulting firm, is seeking an experienced Data Architect to join their dynamic team. In this role, you will be responsible for designing and implementing robust data solutions leveraging the power of Microsoft Azure. Key Responsibilities … Analyse business requirements and translate them into scalable, secure, and efficient data architectures Design and implement data pipelines, data lakes, and data warehouses on the Microsoft Azure platform Ensure data integrity, reliability, and performance through the effective use of Azure … services such as Data Factory, Synapse Analytics, Cosmos DB, and more Collaborate with cross-functional teams to identify and address data-related challenges Stay up-to-date with the latest Azure technologies and best practices, and recommend innovative solutions to drive business value Qualifications: Bachelor's more »

Lead Data Engineer (Java/Python/AWS/Glue/Dremio). Investment Management. £160,000 - £170,000 + 25% Bonus Target and Benefits. Hybrid 2 Days a week in St Pauls Office. CANNOT SPONSOR CANNOT SPONSOR My client is a top tier multi asset investment management … house based in Central London. The Data Office team is playing a key role in helping build the future of financial services, working together with business partners to create client experiences that are changing the way people invest. You will work with smart, talented people across our business. … platform’s architectural design Development and testing of the integration, modelling, data persistence and querying tools and analytical systems Data pipeline implementation, maintenance and testing Metadata management processes and tools Monitoring the overall performance and stability of the pipelines Implementing data curation, metadata more »

Omni client solutions. These solutions are major differentiators that puts Omnicom ahead of the competition. Specific responsibilities of the team are listed below. Data Taxonomies design & maintenance Data Governance and QA Dashboard design, build and maintenance Data Pipelines design & delivery TW/Planit (& TrackIT … for maintaining and improving tables and business logic for a reporting platform. The Business Intelligence Manager will support an MI Director with all data operations and reporting workstreams for one of Omnicom’s largest global autotive clients and work closely with our engineering team; TSOC. This role will … ensure our AWS data infrastructure is supporting a best-in-class reporting platform for one our our largest accounts within Omnicom Media Group, and will be reporting into the Senior Marketing Intelligence and Reporting Director. Communication skills and the ability to prioritize tasks across projects to meet deadlines more »