101 to 125 of 346 Permanent Data Pipeline Jobs in the UK

and Oaktree Capital. Find out more about us by visiting our website at: http://www.hartreepartners.com/JOB OVERVIEW: As a Senior Data Scientist at Hartree, you will play a pivotal role in shaping our data strategy and driving the development of advanced machine learning … time series forecasting solutions, specifically focused on the European Power Markets. You will collaborate with cross-functional teams to design, implement, and optimize data pipelines and predictive models that inform trading decisions and enhance operational efficiency. JOB RESPONSIBILITIES: Market Data Analysis: Conduct comprehensive analysis of data … Python and its data science libraries (e.g., Pandas, NumPy, Scikit-Learn, TensorFlow, PyTorch). Solid understanding of ML and data pipeline architectures and best practices. Experience with big data technologies and distributed computing (e.g., Spark, Hadoop) is a plus. Proficient in SQL and more »

Palantir Data Engineer £85,000 - £95,000 UK Wide Must be able to be eligible for SC clearance! Travel to client site when required! A leading consultancy-based organisation is currently looking to grow their cloud platforms team and wanting to add a Palantir Data Engineer … to their practice. The successful Palantir data engineer will be passionate about cloud technology being involved in using Palantir data focussed reference architecture to assist in designing and building data engineering solutions supporting planning and implementations. Skills and Experience Experience of developing enterprise grade … and deliver the data solutions in Palantir Strong experience with Palantir Data Engineering features such as Code Repo, Code Workbook, Pipeline Builder, migration techniques, Data Connection and Security setup Developing data integration pipelines, transformations, pipeline scheduling, Ontology, and applications in more »

Data Engineer Location: London – Hybrid Employment: Full Time All applicants must hold a DV (Developed Vetting) clearance to apply. The Opportunity: For the Ministry of Defence (MOD), maintaining National Security is a 24-hour-a-day, 365-day-a-year mission. For this reason, it partners with this … Big 4 Firm to support its multidisciplinary AI & Cloud Data Engineering Capabilities. The Role: As an AI & Cloud Data Engineering practitioner, you’ll build large-scale data pipelines and infrastructure, utilising data integration tools to optimise complex data solutions. Collaborating … with high-performing Data Science and ML Engineering teams to drive innovation and insight through AI and Data across the Defence & Security sector. Required Skills/Experience: British Passport Holder with active DV (Developed Vetting Clearance) End-to-end development with data pipelines, ETL more »

consultancy client. The successful candidate will be responsible for designing, developing, and maintaining KDB databases, as well as creating and optimizing real-time data analytics solutions. You will work closely with the technology team to enhance our data management systems, ensure data accuracy and … integrity, and help us make data-driven decisions. Key Responsibilities: Design and develop KDB databases and data processing systems to support real-time data analysis and reporting. Optimize database performance, troubleshoot issues, and implement improvements as needed. Collaborate with data analysts and … other teams to understand data requirements and develop efficient data retrieval processes. Write and maintain KDB Q scripts, ensuring code is well-documented and follows best practices. Monitor data quality and accuracy, implementing data validation and cleansing processes. Develop and maintain data more »

Are you an experienced Data Engineer ? Do you have a passion for Data and experienced with the Microsoft stack ? Do you have SQL, Microsoft Fabric and Azure skills ? If so I have a fantastic new role with my client, with offices in Edinburgh and Glasgow, please … are planning to re-locate. You will be part of a team that works in an Agile environment, contributing to the companies wider data strategy. Responsibilities include: Modernising existing key business solutions by reverse engineering and rearchitecting for Azure. Be involved in designing and implementing systems to effectively … whilst maintaining data integrity and provenance. Experience designing and building robust, production ready, scalable data pipelines Familiarity with optimising strategies, pipeline architectures, data sets and ETL/ELT processes Have experience of selecting the appropriate tools to deliver the solution Coding skills, could more »

Runcorn, WA7 3DJ Salary: £46,000 - £49,000 and extensive benefits package Hours: 37.5 hours per week Are you a seasoned expert in data engineering looking to make a significant impact in the healthcare industry? Phoenix Medical Supplies is seeking a Senior DataOps Engineer to lead our data … services across the region. As a Senior DataOps Engineer, you will be instrumental in architecting and implementing solutions across the entire data pipeline, from ingestion to visualisation. The role will require you to showcase your skills in data ingestion, storage, modelling, transfer, hosting, visualisation, analysis … and consumption, all while mentoring and training team colleagues. Key Responsibilities: Design and develop robust data pipelines for ingestion, storage, processing, and interface creation. Produce both machine consumable interfaces and human consumable data products, including standard and ad hoc reports. Maintain and optimise data more »

Vertex Search is representing a fast-growing financial services firm that is now looking to make a Senior Data Engineering hire, the first role of its kind into the business with huge scope for impact and influence. The nascent data engineering team will collaborate with internal … teams around the business with a focus on data engineering, pipelines, tooling and services. Key Responsibilities: Data Engineering: Design and develop data services across various business functions Data Pipelines: Design and develop data pipelines for extracting, transforming, and loading data into appropriate storage systems. Data Tools and Services: Develop data tools and services to discover, understand, and utilize data effectively. Best Practice: Advise on and contribute to the development of data governance and management practices. Collaborate across teams to understand more »

on delivering quality services and solutions. Able to distill complex technical challenges to actionable and explainable decisions. Having in-depth knowledge on Azure data components with this feature to provide solutions and necessary recommendations on the architecture. Inspire teams by building consensus and mediating compromises when necessary. Demonstrate … technologies and developing a vision of their suitability and role as a new product within a platform & product landscape. Having knowledge on the data components such as Azure datafactory, data lake, Azure SQL DB, Synapse and visualization with expertise skills in terms of performance tuning and … advanced troubleshooting. 8+ years of experience in data architecture, analytics, design and development Good working knowledge in trouble shooting data warehouse/analytical systems using Azure Synapse. Strong, in-depth experience in data modelling and experience with business intelligence Design and develop batch and more »

Senior Data Engineer (GCP) My client a financial services business are looking for a Senior Data Engineer (GCP), to spearhead global initiatives to migrate complex data systems to GCP. You will design, implement, and optimise data pipelines, ensuring data integrity … during migration and utilising GCP services for scalable and efficient data transformation. Salary: Up to £65K Hybrid: 2 days a week Key Responsibilities: Lead data engineering projects in Europe, working with partners for GCP transition. Design and deploy data pipelines with automated data … GCP. Essential Qualifications: 5+ years of SQL. 3+ years of GCP and its different services. Experience with microservice architecture and data processing pipeline design. Desired Qualifications: Certification in GCP. Financial Services experience. Experience coaching and mentoring data engineers. If this role is of interest, please more »

Are you an experienced Data Engineer ? Do you have a passion for Data and experienced with the Microsoft stack ? Do you have SQL, Microsoft Fabric and Azure skills ? If so I have a fantastic new role with my client, with offices in Edinburgh and Glasgow and … its hybrid working. You will be part of a team that works in an Agile environment, contributing to the companies wider data strategy. Responsibilities include: * Modernising existing key business solutions by reverse engineering and rearchitecting for Azure. * Be involved in designing and implementing systems to effectively handle large … whilst maintaining data integrity and provenance. * Experience designing and building robust, production ready, scalable data pipelines * Familiarity with optimising strategies, pipeline architectures, data sets and ETL/ELT processes * Have experience of selecting the appropriate tools to deliver the solution * Coding skills, could more »

Senior Azure Data Engineer - 6 Month Contract - Inside IR35 This is an Azure Data Engineer 6 month contract position for a financial services end client helping them on delivering and implementing new data pipelines. You will need prior financial services experience as well as … having strong Azure Data Factory and Databricks experience. Work Involved: Implementing and delivering data pipelines Providing mentorship to junior members of the team Working with the Platform team defining capabilities for the lake … framework Developing business conceptual data models Producing new frameworks using design patterns and data ingestion Requirements: Strong Data Pipeline experience Databricks Azure Data Factory Spark Azure cloud Financial Services If you're interested get in touch ASAP more »

Data Engineer – Azure A fast growing and innovative company are looking for a Data Engineer to join their team. They are a people focused company, who since 2021 have won 3 best place to work awards, representing a fantastic working environment. Based in either Derby or … the Chief Operating Officer, the candidate will play a crucial role in driving the company's digital initiatives forward. Key Responsibilities As a Data Engineer, you will play a pivotal role in supporting the delivery and evolution of the company's digital portfolio. Collaborating with a talented team … the candidate will oversee the management and maintenance of existing data pipelines, monitor workloads, conduct root cause analysis, ensure data quality, and escalate technical issues as necessary. What you’ll be getting up to: Monitoring and recovery of Azure Data Factory and Databricks jobs more »

Senior Data Engineer (Remote) - Stealth SaaS Startup Location: Manchester (Remote Friendly) Salary: £65,000 - £75,000 per annum About Us: We are an innovative stealth start-up set to launch a cutting-edge SaaS product next year. We’re looking for a talented Senior Data Engineer to join our team and build robust data pipelines. Responsibilities: Build, and maintain scalable data pipelines. Maintain the pipeline through Databricks and Apache Beam Seamless data integration and quality. Data storage solutions using GCP (will still be considered with … to bring the AI/ML models into production Databricks for daily data processing Qualifications: Experience in data engineering and pipeline construction. GCP and familiarity with other cloud environments. Understanding of SDLC. Experience with Databricks. Programming skills (Python and SQL). What We Offer more »

Job Title: Cloud Data Engineer Location: Knutsford, UK Hybrid work: 2-3 days a week Capgemini is seeking a Cloud Data Engineer t o join Capgemini Financial Services I&D Practice. This is a permanent, full-time position in Knutsford, UK and this represents a unique … opportunity for someone to enhance their I&D Consulting career. What you’ll do and bring: Proficiency in AWS services relevant to data engineering such as S3, Glue, EMR, Athena, and Lambda. Hands-on experience with Snowflake, Redshift cloud data warehousing solutions. Familiarity with DBT (Data … Glue, Starburst, Snowflake, Redshift, Starburst, Hybrid(on-prem to cloud) for building data pipelines and ETL processes. Experience with data pipeline orchestration tools like Apache Airflow . Knowledge of SQL and ability to write complex queries for data transformation and analysis. Understanding of more »

Job Title: Cloud Data Engineer Location: Northampton,UK Hybrid work: 2-3 days a week Capgemini is seeking a Cloud Data Engineer t o join Capgemini Financial Services I&D Practice. This is a permanent, full-time position in Northampton, UK and this represents a unique … opportunity for someone to enhance their I&D Consulting career. What you’ll do and bring: Proficiency in AWS services relevant to data engineering such as S3, Glue, EMR, Athena, and Lambda. Hands-on experience with Snowflake, Redshift cloud data warehousing solutions. Familiarity with DBT (Data … Glue, Starburst, Snowflake, Redshift, Starburst, Hybrid(on-prem to cloud) for building data pipelines and ETL processes. Experience with data pipeline orchestration tools like Apache Airflow . Knowledge of SQL and ability to write complex queries for data transformation and analysis. Understanding of more »

Data Engineer Our client based near Manchester is looking for a skilled Data Engineer to join their agile team. In this role, you'll play a pivotal part in designing, building, and maintaining data pipelines, ensuring the integrity and availability of data for various stakeholders, including BI Teams, Analytics, and Data Scientists. Responsibilities: Design, build, and automate data flow processes and ELTs to provide comprehensive data from business systems. Collaborate with product development teams to ensure best practices and data integrity from … source to destination. Assist in integrating data between systems, including third-party platforms. Write specifications and documentation for data processes. Continuously explore new solutions, tools, and processes to enhance data operations. Requirements: Bachelor's or Master's degree in Computer Science, related field, or more »

A global boutique (re)insurance broker are looking for an experienced Azure Data Engineer to join their growing team, to act as the lead Engineer on the support and development of their brand-new next-generation data platform. This is a permanent role with hybrid working … incredibly digitally-driven and are fully operating on Azure, with no legacy systems. You'll be joining their small (but growing) Cloud and Data team as their sole Data Engineer, making this a role with huge growth potential. They're currently working with an external partner … to build a brand-new Snowflake data warehouse, which brings a huge scope of work for you to lead on internally, including the development of scalable data pipelines, overseeing the integration of new data management technologies, and implementing data storage solutions. This more »

Job Title: Data Operations Analyst Company: Global Asset Manager Location: City of London, 3 days per week in office Salary: £85,000 per annum + Benefits About the Company: Join a prestigious global asset manager with over $200 billion in assets under management. Our firm is renowned for … excellence. Based in the heart of London's financial district, you'll be part of a dynamic team shaping the future of financial data operations. Role Overview: As a Data Operations Analyst, you will play a pivotal role in implementing a cutting-edge global platform aimed … at centralizing data management processes. Leveraging your expertise in R, Python, SQL, and Enterprise Data Management (EDM), you'll contribute to streamlining data workflows and optimizing data quality across the organization. Key Responsibilities: Collaborate with cross-functional teams to understand data more »

Data Engineer £80,000-£95,000 London, Hybrid Permanent I am seeking an experienced Data Engineer who is passionate about designing, optimising, and maintaining AI-native data processing pipelines. Your role will involve tackling complex challenges with robust data pipelines across diverse … sources, enhancing the quality of data for our systems, and readiness to meet the challenges of scale. Responsibilities: Ensure the secure handling of data through end-to-end encryption, data isolation, and other best practices. Design robust data processing and indexing pipelines … handling large amounts of data across several modalities, ensuring robustness and timeliness. Curate and manage large-scale data ingestion and indexing pipelines, ensuring data quality and error handling. Manage cloud orchestration and data infrastructure, applying your expertise with various data more »

A global boutique (re)insurance broker are looking for an experienced Azure Data Engineer to join their growing team, to act as the lead Engineer on the support and development of their brand-new next-generation data platform. This is a permanent role with hybrid working … incredibly digitally-driven and are fully operating on Azure, with no legacy systems. You'll be joining their small (but growing) Cloud and Data team as their sole Data Engineer, making this a role with huge growth potential. They're currently working with an external partner … to build a brand-new Snowflake data warehouse, which brings a huge scope of work for you to lead on internally, including the development of scalable data pipelines, overseeing the integration of new data management technologies, and implementing data storage solutions. This more »

Data Engineer - London - £75,000 I am working with a hugely successful, global organisation who are looking to add a Data Engineer to their growing London based data team. This organisation have recently implemented a brand new data platform encompassing the full … Microsoft Azure Stack including leveraging Databricks in order to deliver on its goal of becoming truly data driven. In this role you will be responsible for the BI and data engineering lifecycle across the business supporting the global data needs of various business functions. … This role is a split between data engineering and data analytics and you will take a prominent role on a number of exciting projects. You will be joining a business that values its people at the centre of everything they do. They are looking to expand more »

Data Engineer - London/Hybrid - £70,000 FTC I am currently working with a well-established organisation within the education sector who are looking for a Data Engineer to join their established IT team. Their current project is based around the creation and implementation of a … brand new Azure Data Platform. You will take responsibility for the design, development and maintenance of the data platform utilising your experience with the Azure tech stack. This is an opportunity to join a team who are serious about the way that they use data and work within an organisation who are providing significant investment into all of their data teams. The teams itself are growing in size but the company are also investing in the technology that they use with an emphasis on Microsoft tech such as Azure Synapse, Databricks more »

Azure Data Engineer | Cheltenham | £50,000-£70,000 Locke & McCloud are working with a leading Cyber Security Consultancy that is home to over 4000 specialists who support clients across manufacturing, defence and security, energy and utilities, financial services and many other industries. They have offices globally and are … recognised for the impressive high standard of work they conduct. They are looking for an Azure Data Engineer, you will play a pivotal role in this journey, leveraging your expertise to architect, design, and implement scalable data solutions on the Azure platform. You will collaborate closely … with stakeholders to understand business requirements, design data solutions, and deliver value-added insights. You will be responsible for architecting data pipelines, optimizing data storage and retrieval, and ensuring the integrity and security of our data ecosystem. Additionally, you will have the more »

Job Title: Data Analyst - Bristol - £50K We are working with a growing digital agency client based in Bristol who are looking for a talented Data Analyst to join their team. This is a hybrid role with at least 2 days a week in their local office … hub (Bristol or London). As the Data Analyst, you will be responsible for providing structure and process around data collection and analysis as the company scales. You will have a passion for data and enjoy using it to support decision making. Your key … reports that improve performance, formulating and documenting requirements for outputs, performing data analysis on datasets, troubleshooting issues in the data pipeline and establishing standard reports on key business areas. You will also need to maintain and enhance datasets, review data collection processes and more »

Job Title: Technical Lead Data Engineer Location: Edinburgh, Scotland (Hybrid) - All employees must be eligible to work without sponsorship Salary: £70,000 - £85,000 The Role: Lead the Data team to deliver ETL processes, build and maintain data pipelines, enhance scalability, performance, maintainability, configurability … and usability Be involved in the entire life-cycle of the Data Strategy, including idea creation, deployment, and longer term updates and enhancements Support the product and Software Development Managers in prioritisation, scoping and leading delivery of Data Strategy features in the exciting world of Future … State Architecture Manage company data, monitor accuracy and completeness, and advise ways that data quality can be improved Able to zero-in on bugs without looking at the code Technical Lead Data Engineer Requirements: Good SQL and NoSQL experience Expert-level ability in TSQL more »