76 to 100 of 290 Data Pipeline Jobs in the UK excluding London

our award-winning team at Sparta Global. About the Job You'll become versatile in a wide array of tools across topics covering data focused coding, data visualisation, Cloud Services, Big Data and Machine learning. You will be equipped to work within a variety … of roles including but not limited to; Data Engineer, Data Analyst, Data Scientist, Information Engineer. You'll be designing, building, maintaining, and troubleshooting the data pipelines that enable organizations to store, process, and analyse their data, and ensuring the data is reliable through testing and debugging. Looking for efficiencies and optimising the data pipelines for scalability and performance will be a focus. You'll be handling and working with large sets of structured and unstructured data and will be responsible for ensuring that the more »

our award-winning team at Sparta Global. About the Job You'll become versatile in a wide array of tools across topics covering data focused coding, data visualisation, Cloud Services, Big Data and Machine learning. You will be equipped to work within a variety … of roles including but not limited to; Data Engineer, Data Analyst, Data Scientist, Information Engineer. You'll be designing, building, maintaining, and troubleshooting the data pipelines that enable organizations to store, process, and analyse their data, and ensuring the data is reliable through testing and debugging. Looking for efficiencies and optimising the data pipelines for scalability and performance will be a focus. You'll be handling and working with large sets of structured and unstructured data and will be responsible for ensuring that the more »

Data Engineer Paddington, London (Hybrid) Overview Data and Analytics is foundational to our Petcare OGSM and will drive our transformation to a business that is powered by data. To deliver on this ambition set by this OGSM we require the very highest level of technical/… engineering expertise within Global Petcare Data & Analytics. There is a need to expand this high performing team of creative, skilled individuals – to help build out new capabilities and bring fresh ideas to the table. Key Responsibilities Engineer and orchestrate data flows & pipelines in a cloud environment … in a cloud environment (Azure, AWS, GCP) Experience in at least one of: Python (or similar), SQL, PySpark Experience in building data pipeline/ETL/ELT solutions Ability and strong desire to research and learn new technologies and languages Interest in the contemporary data more »

Data Engineer/Scientist London (Hybrid) £50,000-£60,000 (+bens) 83DATA are partnered with a leading Internet provider seeking the skills of a Data Engineer/Scientist. The successful candidate can expect to be tasked to connect data sources to pre-existing Azure … data infrastructure. You will lead projects with a data centric approach, for this reason, a passion and aptitude for critical analysis of large and complex datasets is vital. Responsibilities: Build data engineering pipelines to ingest, transform, store, and serve high quality data for Data Scientists Writing and tuning complex data pipelines, workflows, and data structures. Identify opportunities for increasing performance, minimising costs and increase platform monitoring. Experience with components such as Azure Active Directory, Management Groups, Subscriptions, vNets, Route Tables, NSG’s and Azure more »

Senior Data Platform Engineer Up to £80,000 1 day a week in office (flexible) Xcede are delighted to be working with a well-renowned insurance company based in Hertfordshire. The company have an excellent Data Science and Machine Learning unit supported by a group of … Data Engineers who are responsible for Data Pipelines. As a bonus - Data Engineers are also able to help with ML model deployment and MLOps. Responsibilities Design and implement Data Pipelines for both structured and unstructured data. Support and improve ML infrastructure – particularly … Azure ML. Data Engineering: Guarantee data access, Data quality, ETL, Data pipelines Platform administration: Help set up accesses, infrastructure, create new storage, enforce best practices Production: Help the preparation of the data science pipelines, interact with data scientists more »

Job Title: Data Engineer Job Type: Full Time, Permanent Working location: London, Hybrid Role Purpose At Travelex we are developing modern data services, which will be at the heart of our relationship with our customers. Our data architecture is becoming more service-oriented and … for you to solve. Our work is fast-paced although our team remains collaborative and flexible even when delivering fast. Your role as data engineer has a wide remit: working with product owners, subject matter experts and managers across the company, to design and deliver new components of … to the traditional data engineering type! Significant hands-on experience with AWS, especially AWS services focused on data flow, pipelines, data transformation, storage and streaming. Excellent data engineering skills, for example with SQL, Python, DBT and Airflow. Good understanding of service-oriented more »

Job Title: Data Engineer Location: Edinburgh, Scotland (Hybrid) - All employees must be eligible to work without sponsorship Salary: £50,000 - £70,000 The Role: Design and develop ETL processes Design, build and maintain end-to-end data pipelines Identify, root cause analysis, and fix incidents related … to data management Manage company data, monitor accuracy and completeness, and advise ways that data quality can be improved Routinely provide status updates to colleagues and stakeholder groups Data Engineer Requirements: Hands-on experience with SQL database design Experience writing automated tests … for data pipelines Technical expertise with data models, data mining, and segmentation techniques Knowledge of programming languages (e.g. Python, R) Data Engineer Desirable's: Proven experience of Azure Services Knowledge of Data bricks and Jupyter notebooks Experience in data more »

our award-winning team at Sparta Global. About this role You'll become versatile in a wide array of tools across topics covering data focused coding, data visualisation, Cloud Services, Big Data and Machine learning. You will be equipped to work within a variety … of roles including but not limited to; Data Engineer, Data Analyst, Data Scientist, Information Engineer. You'll be designing, building, maintaining, and troubleshooting the data pipelines that enable organizations to store, process, and analyse their data, and ensuring the data is reliable through testing and debugging. Looking for efficiencies and optimising the data pipelines for scalability and performance will be a focus. You'll be handling and working with large sets of structured and unstructured data and will be responsible for ensuring that the more »

our award-winning team at Sparta Global. About this role You'll become versatile in a wide array of tools across topics covering data focused coding, data visualisation, Cloud Services, Big Data and Machine learning. You will be equipped to work within a variety … of roles including but not limited to; Data Engineer, Data Analyst, Data Scientist, Information Engineer. You'll be designing, building, maintaining, and troubleshooting the data pipelines that enable organizations to store, process, and analyse their data, and ensuring the data is reliable through testing and debugging. Looking for efficiencies and optimising the data pipelines for scalability and performance will be a focus. You'll be handling and working with large sets of structured and unstructured data and will be responsible for ensuring that the more »

Data Engineer Location: London – Hybrid Employment: Full Time All applicants must hold a DV (Developed Vetting) clearance to apply. The Opportunity: For the Ministry of Defence (MOD), maintaining National Security is a 24-hour-a-day, 365-day-a-year mission. For this reason, it partners with this … Big 4 Firm to support its multidisciplinary AI & Cloud Data Engineering Capabilities. The Role: As an AI & Cloud Data Engineering practitioner, you’ll build large-scale data pipelines and infrastructure, utilising data integration tools to optimise complex data solutions. Collaborating … with high-performing Data Science and ML Engineering teams to drive innovation and insight through AI and Data across the Defence & Security sector. Required Skills/Experience: British Passport Holder with active DV (Developed Vetting Clearance) End-to-end development with data pipelines, ETL more »

Runcorn, WA7 3DJ Salary: £46,000 - £49,000 and extensive benefits package Hours: 37.5 hours per week Are you a seasoned expert in data engineering looking to make a significant impact in the healthcare industry? Phoenix Medical Supplies is seeking a Senior DataOps Engineer to lead our data … services across the region. As a Senior DataOps Engineer, you will be instrumental in architecting and implementing solutions across the entire data pipeline, from ingestion to visualisation. The role will require you to showcase your skills in data ingestion, storage, modelling, transfer, hosting, visualisation, analysis … and consumption, all while mentoring and training team colleagues. Key Responsibilities: Design and develop robust data pipelines for ingestion, storage, processing, and interface creation. Produce both machine consumable interfaces and human consumable data products, including standard and ad hoc reports. Maintain and optimise data more »

Vertex Search is representing a fast-growing financial services firm that is now looking to make a Senior Data Engineering hire, the first role of its kind into the business with huge scope for impact and influence. The nascent data engineering team will collaborate with internal … teams around the business with a focus on data engineering, pipelines, tooling and services. Key Responsibilities: Data Engineering: Design and develop data services across various business functions Data Pipelines: Design and develop data pipelines for extracting, transforming, and loading data into appropriate storage systems. Data Tools and Services: Develop data tools and services to discover, understand, and utilize data effectively. Best Practice: Advise on and contribute to the development of data governance and management practices. Collaborate across teams to understand more »

consultancy client. The successful candidate will be responsible for designing, developing, and maintaining KDB databases, as well as creating and optimizing real-time data analytics solutions. You will work closely with the technology team to enhance our data management systems, ensure data accuracy and … integrity, and help us make data-driven decisions. Key Responsibilities: Design and develop KDB databases and data processing systems to support real-time data analysis and reporting. Optimize database performance, troubleshoot issues, and implement improvements as needed. Collaborate with data analysts and … other teams to understand data requirements and develop efficient data retrieval processes. Write and maintain KDB Q scripts, ensuring code is well-documented and follows best practices. Monitor data quality and accuracy, implementing data validation and cleansing processes. Develop and maintain data more »

two days) at our Leeds, Edinburgh, or Bristol locations. About this opportunity Join IP&I as we embark on a transformative journey in data engineering. We are seeking a dynamic and experienced Senior Manager within Advanced Analytics to lead our team in architecting and implementing cutting-edge data solutions. If you are passionate about driving innovation and possess a proven track record in cloud and legacy data engineering leadership, we want to hear from you! You will be at the forefront of our data engineering initiatives. You'll lead a talented team … in designing, developing, and optimizing scalable data solutions. Your role will involve providing technical leadership, driving innovation, and ensuring the successful delivery of complex data projects. Here's where you'll make the difference: Lead and mentor a team of data engineers Architect and more »

Job Role: Azure Data Engineer Location: Central London - Hybrid/3 Days in Office Salary: £60,000 - £70,000 + 12.5% Bonus Chapman Tate is collaborating with a leading property management company that is revolutionizing the industry through innovative data-driven solutions. As they continue to … expand their operations, they are seeking a talented Azure Data Engineer with strong SQL expertise to join their dynamic team. Are you passionate about leveraging data to drive business insights and enhance operational efficiency? Do you have experience working with Azure data technologies and … so, this is an exciting opportunity to contribute to the success of a forward-thinking property management company. Responsibilities: Design, develop, and maintain data pipelines and ETL processes using Azure Data Factory. Optimize data storage and retrieval processes to ensure scalability, reliability, and performance. more »

Senior Azure Data Engineer - 6 Month Contract - Inside IR35 This is an Azure Data Engineer 6 month contract position for a financial services end client helping them on delivering and implementing new data pipelines. You will need prior financial services experience as well as … having strong Azure Data Factory and Databricks experience. Work Involved: Implementing and delivering data pipelines Providing mentorship to junior members of the team Working with the Platform team defining capabilities for the lake … framework Developing business conceptual data models Producing new frameworks using design patterns and data ingestion Requirements: Strong Data Pipeline experience Databricks Azure Data Factory Spark Azure cloud Financial Services If you're interested get in touch ASAP more »

Job Title: Data Operations Analyst Company: Global Asset Manager Location: City of London, 3 days per week in office Salary: £85,000 per annum + Benefits About the Company: Join a prestigious global asset manager with over $200 billion in assets under management. Our firm is renowned for … excellence. Based in the heart of London's financial district, you'll be part of a dynamic team shaping the future of financial data operations. Role Overview: As a Data Operations Analyst, you will play a pivotal role in implementing a cutting-edge global platform aimed … at centralizing data management processes. Leveraging your expertise in R, Python, SQL, and Enterprise Data Management (EDM), you'll contribute to streamlining data workflows and optimizing data quality across the organization. Key Responsibilities: Collaborate with cross-functional teams to understand data more »

Data Engineer – Azure A fast growing and innovative company are looking for a Data Engineer to join their team. They are a people focused company, who since 2021 have won 3 best place to work awards, representing a fantastic working environment. Based in either Derby or … the Chief Operating Officer, the candidate will play a crucial role in driving the company's digital initiatives forward. Key Responsibilities As a Data Engineer, you will play a pivotal role in supporting the delivery and evolution of the company's digital portfolio. Collaborating with a talented team … the candidate will oversee the management and maintenance of existing data pipelines, monitor workloads, conduct root cause analysis, ensure data quality, and escalate technical issues as necessary. What you’ll be getting up to: Monitoring and recovery of Azure Data Factory and Databricks jobs more »

Senior Data Developer Salary – Circa £50,000 This is a great opportunity for someone looking for a new challenge as we expand our Solutions Development Team. We are looking for an enthusiastic and experienced data engineer to bring their technical and teamworking skills to rapidly build … data domain knowledge and capability to support and enhance our constantly expanding portfolio of data assets. A bit about the job: The role resides within the central Data Insights Team that predominantly support the Insurance Wealth & Retirement (IWR) business through the provision of data products, and the role will both support the management and change of the existing data estate alongside design and implementation of new data capability in-line with Aviva project objectives. You will be working collaboratively with colleagues across the wider DI function to ensure more »

Senior Data Engineer (Remote) - Stealth SaaS Startup Location: Manchester (Remote Friendly) Salary: £65,000 - £75,000 per annum About Us: We are an innovative stealth start-up set to launch a cutting-edge SaaS product next year. We’re looking for a talented Senior Data Engineer to join our team and build robust data pipelines. Responsibilities: Build, and maintain scalable data pipelines. Maintain the pipeline through Databricks and Apache Beam Seamless data integration and quality. Data storage solutions using GCP (will still be considered with … to bring the AI/ML models into production Databricks for daily data processing Qualifications: Experience in data engineering and pipeline construction. GCP and familiarity with other cloud environments. Understanding of SDLC. Experience with Databricks. Programming skills (Python and SQL). What We Offer more »

Data Engineer £80,000-£95,000 London, Hybrid Permanent I am seeking an experienced Data Engineer who is passionate about designing, optimising, and maintaining AI-native data processing pipelines. Your role will involve tackling complex challenges with robust data pipelines across diverse … sources, enhancing the quality of data for our systems, and readiness to meet the challenges of scale. Responsibilities: Ensure the secure handling of data through end-to-end encryption, data isolation, and other best practices. Design robust data processing and indexing pipelines … handling large amounts of data across several modalities, ensuring robustness and timeliness. Curate and manage large-scale data ingestion and indexing pipelines, ensuring data quality and error handling. Manage cloud orchestration and data infrastructure, applying your expertise with various data more »

About the Role: I am looking for a talented Data Engineer to join our clients dynamic team on a 6-month contract basis. This is an exciting opportunity for a mid-level professional with 3-5 years of experience to make a significant impact on our data …/hybrid working options and a competitive day rate of £250-£400, falling inside IR35 regulations. Key Responsibilities: Design, develop, and maintain scalable data pipelines and ETL processes using AWS, Databricks, Python, Spark, and SQL. Collaborate with data scientists, analysts, and other stakeholders to understand data requirements and deliver high-quality data solutions. Optimize and troubleshoot data workflows to ensure efficiency and reliability. Implement data quality and validation checks to ensure data integrity. Contribute to the development and maintenance of our data lake and more »

Azure Data Engineer | Cheltenham | £50,000-£70,000 Locke & McCloud are working with a leading Cyber Security Consultancy that is home to over 4000 specialists who support clients across manufacturing, defence and security, energy and utilities, financial services and many other industries. They have offices globally and are … recognised for the impressive high standard of work they conduct. They are looking for an Azure Data Engineer, you will play a pivotal role in this journey, leveraging your expertise to architect, design, and implement scalable data solutions on the Azure platform. You will collaborate closely … with stakeholders to understand business requirements, design data solutions, and deliver value-added insights. You will be responsible for architecting data pipelines, optimizing data storage and retrieval, and ensuring the integrity and security of our data ecosystem. Additionally, you will have the more »

Job Title: Data Analyst - Bristol - £50K We are working with a growing digital agency client based in Bristol who are looking for a talented Data Analyst to join their team. This is a hybrid role with at least 2 days a week in their local office … hub (Bristol or London). As the Data Analyst, you will be responsible for providing structure and process around data collection and analysis as the company scales. You will have a passion for data and enjoy using it to support decision making. Your key … reports that improve performance, formulating and documenting requirements for outputs, performing data analysis on datasets, troubleshooting issues in the data pipeline and establishing standard reports on key business areas. You will also need to maintain and enhance datasets, review data collection processes and more »

Job Title: Technical Lead Data Engineer Location: Edinburgh, Scotland (Hybrid) - All employees must be eligible to work without sponsorship Salary: £70,000 - £85,000 The Role: Lead the Data team to deliver ETL processes, build and maintain data pipelines, enhance scalability, performance, maintainability, configurability … and usability Be involved in the entire life-cycle of the Data Strategy, including idea creation, deployment, and longer term updates and enhancements Support the product and Software Development Managers in prioritisation, scoping and leading delivery of Data Strategy features in the exciting world of Future … State Architecture Manage company data, monitor accuracy and completeness, and advise ways that data quality can be improved Able to zero-in on bugs without looking at the code Technical Lead Data Engineer Requirements: Good SQL and NoSQL experience Expert-level ability in TSQL more »