1 to 25 of 131 Data Pipeline Jobs in the UK excluding London

financial and tax planning, offering home finance and annuities propositions, and providing collective fund solutions to third party customers. Job Purpose The Lead Data Engineer is a hands-on technical role responsible for designing, developing, and maintaining data pipelines within the IT department. The pipelines will … be realised in a modern lake environment and the engineer will collaborate in cross-functional teams to gather requirements and develop the conceptual data models. This role plays a crucial part in driving data-driven decision-making across the organisation, ensuring data availability, quality … in guiding and upskilling more junior data engineers and setting data standards and guidelines. Duties/Responsibilities Data Pipeline Development Design, model, develop and maintain data pipelines to ingest, store, process, and present data. Ensure data quality, accuracy, and more »

We are searching for an experienced and talented Head of Data Engineering for a critical role working for a successful and expanding business, a company that is a market leader in its field, producing award-winning software solutions and services for a diverse portfolio of customers. The role … UK without company sponsorship. In this role you will be responsible for the following: - Creating and managing a complex service model that delivers data timely and accurately to a wide variety of stakeholders and downstream consuming applications and services. Defining and managing the cost model for the Data Platform and the staff who work on the platform. Working hand-in-hand with Data Operations and Data Insight teams to build key data pipelines and platform processes, deploying models and implementing orchestrations. Implementing governance models for data access, data more »

Data Engineer – Hybrid Working £45,585 - £54,395 30 Days Holidays + 4 Days Christmas + Bank Holidays Great Pension Career Development Flexible Working - (Hybrid) A Great Working Environment Excellent Sport & Leisure Facilities Training and Personnel Development Offerings Office Location is close to A45, A46, M1, M6, M40 … implement data security measures Design and implement data quality checks and validation processes within the pipelines using expectations with the pipeline, or dedicated testing tools (SODA) Collaborate with data governance, architects, system/product owners and end users to ensure quality issues are … addressed as needed. Create and maintain technical documentation, including data pipeline specifications, data models, and transformation logic Contribute to data modelling and schema design standards and related documentation. Qualifications & Requirements Degree in Computer Science, Data Engineering, or a related field or more »

data - driven solutions, we eagerly await your application. Your main responsibilities will be to: · Design, create and maintain optimal data pipeline architecture. · Assemble large, complex data sets that meet the project and/or client requirements from a wide variety of structured and … cleaning, wrangling, and processing on a variety of data sets. · Design, build and integrate analytics tools that utilise the data pipeline to provide actionable insights for the customers using metrics, statistics and ML techniques. · Identify, design , implement and ensure adherence to internal process improvements: automating … analytics, and other data wrangling tools and methodologies. · Experience of designing, building and optimising ‘big data’ data pipeline, architectures and data sets. · The ability to plan, design and work in a way that creates standardisation so that other team members more »

the future of datadriven solutions, we eagerly await your application. Your main responsibilities will be to: Design, create and maintain optimal data pipeline architecture. Assemble large, complex data sets that meet the project and/or client requirements from a wide variety of structured and … cleaning, wrangling, and processing on a variety of data sets. Design, build and integrate analytics tools that utilise the data pipeline to provide actionable insights for the customers using metrics, statistics and ML techniques. Identify, design, implement and ensure adherence to internal process improvements: automating … analytics, and other data wrangling tools and methodologies. Experience of designing, building and optimising big data data pipeline, architectures and data sets. The ability to plan, design and work in a way that creates standardisation so that other team members more »

Data Analyst, SQL Azure V12, SQL, Relational databases, ETL, SSIS, SSMS, Analytical and problem solving softer skills. Permanent role. 2 days a week in the office (Huntingdon area between Peterborough and Cambridge), 3 days remote/home working. £40-£45k salary, 7.5% pension, 25 days holiday. Lithium3 are … working with our client exclusively to recruit a Data Analyst to join their established team of 3 who sit between the main Enterprise Data … Warehouse Build function and the visualisation team for reporting. This individual will be responsible for managing, maintaining expanding and optimising the data pipeline and architecture, and supporting end users through data and report delivery. This team is very much doing Projects , working with technical and more »

Are you passionate about leveraging data to drive positive impact in the realm of public service? My client is an award-winning, government-backed not-for-profit organisation embarking on a major digital transformation journey with a primary focus on becoming more data-led. As they … pipelines to ensure optimal throughput and minimal latency, contributing to the overall efficiency of their data ecosystem. Address data pipeline failures and performance bottlenecks with precision, utilising your problem-solving skills to maintain the integrity of their data processes. Keep abreast of … ETL best practices. Proficiency in Python, C#, or PowerShell to enhance and optimize data processes. Ability to troubleshoot complex data pipeline issues with precision. Effectively collaborate with both technical and non-technical stakeholders. Azure certifications, such as Microsoft Certified: Azure Data Engineer Associate. more »

Data Engineer to join our growing team. You will be responsible for expanding and optimizing our data, data pipeline architecture and optimizing data flow across functional teams. The Data Engineer will work with our software developers, database architects, data analysts and data scientists on data initiatives. Responsibilities for Data Engineer: Configuring Azure Event Hubs Writing Azure Function Apps in C# Creating data pipelines in Azure Data Factory Creating PowerBI reports and visualisations Analysing analytics business requirements to … determine the data models and data flows required to deliver new dashboards and reports Creating Solution Design documents for analytics solutions Documenting Architectural Decisions that are relevant for analytics solutions Designing Extract-Transform-Load processes Required Technical and Professional Expertise Development of complex PowerBI reports from more »

Data Engineer Role purpose: The Data Engineer position is an integral part of the Data team within the Digital and Information function. This role entails overseeing the availability, integrity, quality, and accuracy of data. The primary responsibilities include designing, developing, and maintaining data pipelines and workflows to support the organisation's data processing needs and facilitate BI reporting and analytics delivery. The Data Engineer will contribute to enhancing our Azure-based data platform using technologies such as Azure Data Factory and Azure Databricks … functional teams including data scientists, analysts, and business stakeholders to understand data requirements, perform UAT and deliver solutions accordingly. Monitor pipeline performance and troubleshoot issues promptly to ensure uninterrupted data processing. Document data pipelines, workflows, and processes to ensure knowledge sharing more »

An experienced Cloud Data Engineering Manager with at least 2-3 years of managing data engineering experience is needed to join the team at this rapidly growing Data Analytics Consultancy based in Oxford. You will be excited to play a pivotal role in leading … career with a well-established company! About the Company Founded in 2014 and based in Oxford, UK, this dynamic analytical consultancy specialises in data analysis, statistical modelling, and various analytical techniques like BI and Risk Management. They provide businesses with custom, secure solutions, primarily working in the cloud … data services (e.g., S3, Athena, Lambda, Glue) Advanced working knowledge in programming languages - specifically Python Hands-on experience in data pipeline development using tools such as AWS Glue Excellent communication and presentation skills, with the ability to articulate complex technical concepts to all stakeholders (non more »

Senior Data Engineer Role purpose: The Senior Data Engineer plays a pivotal role within the Data team of the Digital and Information function, assuming a leadership position in ensuring the availability, integrity, quality, and accuracy of data. With a wealth of experience, the Senior … Data Engineer leads the design, development, and maintenance of data pipelines and workflows, driving the organisation's data processing capabilities and enabling the seamless delivery of BI reporting and analytics. In addition to core responsibilities, the Senior Data Engineer takes a proactive … stakeholders, to comprehensively understand data requirements, conduct User Acceptance Testing (UAT), and deliver tailored solutions that align with business objectives. Proactively monitor pipeline performance, swiftly identifying and resolving any potential issues to uphold seamless data processing operations and minimize downtime. Drive comprehensive documentation of data more »

last decade, IGS has built its reputation as a leading global provider of vertical farming technology, and is looking to recruit an experienced Data Engineer to join the team. The purpose of this role is to create and productionise data-centric applications and features, data pipelines and data analysis tools in an Azure cloud environment. You will also provide real time control and information for our market leading vertical farming solution. Responsibilities: Build new data-centric applications that are tested at multiple levels and ready for our Azure/… kubernetes production environment Update and add new features to existing applications Work closely with data scientists to automate and productionise machine learning and other complex solutions to business problems Collaborate and communicate with other team members including product managers, crop scientists, hardware, software and IoT engineers Use, update more »

Data Engineer – Hybrid (3 days in Office) Stratford-Upon-Avon £56k + 5% bonus & benefits Do you want to grow your expertise and experience and use your skills in a vibrant environment where teamwork, creativity, diversity, inclusivity, and technical excellence are expected and enjoyed? This SaaS company are … associates to principals, applying your technical and problem-solving capabilities to scale their applications. You’ll contribute to the organisation’s high-quality data engineering practices, participating in technical design, implementation, testing, deployment, and ongoing support and maintenance of the data platform. By going above and … beyond implementing new features, the company focus on customer experience, building a high-quality, secure, and scalable data platform. You'll apply your skills effectively and continue to develop them, while benefiting from the expertise of your more experienced colleagues, including: Pair and mob-programming. Peer code reviews. more »

Data Engineer … Dynamics 365 CE/CRM and Power Platform experience required A well established MS partner who are experiencing growth through a very strong project pipeline are looking to add a Data Engineer and a Data Analytics Consultant to their team. They pride themselves on their … integration workflows, and manage data load processes to support both regular and ad-hoc activities. * Create and maintain optimal data pipeline architecture, which might include integrating with Dataverse via cloud flows or external data sources. * Design, develop, and maintain data warehouse more »

Please note: you need to have the right to work in the UK – no sponsorship, unfortunately About the job Reporting to the Senior Data & Analytics Lead, the BI Developer will be responsible for driving the direction and maintaining the business data model ensuring it aligns with … business activities and processes. The role will work closely with key internal and external stakeholders in organising and creating a data model that makes data and information accessible with appropriate channels of access controls. Fostering a relationship of trust and playing an advisory role in the … integrity, and dependability of the data. Improve the scalability, efficiency, and cost-effectiveness of data pipelines. Monitoring and resolving data pipeline problems will guarantee consistency and availability of the data. Key Requirements Good understanding and competency in technologies currently used (and available in the marketplace more »

Introduction A Data Asset Manager is needed to be the driving force in maximising the value and impact of data assets within a data strategy. This is an opportunity to play a key role in shaping the future of data management and … analytics and you will join a highly functioning team driven by a common goal. The Data Asset Manager will develop investment cases for data assets, implement innovative strategies to enhance return on investment, and ensure alignment with organisational goals. This role will require you to understand … business objectives, translate them into data strategies, and then sell them back to internal stakeholders to gain buy-in from the wider business. As such, you will need excellent people skills and the ability to communicate technical matters to stakeholders at all levels. Key Responsibilities Manage and maintain more »

do things differently. We’re focused on our core values; using these we’ve seen significant growth across our practices and our Digital, Data and Cloud team is preparing for the next phase of expansion. This creates new opportunities for driven and skilled individuals to join one of … the fastest-growing consultancies globally. Job Description This is an exciting opportunity for an experienced developer of large-scale data solutions. You will join a team delivering a transformative cloud hosted data platform for a key Version 1 customer. The ideal candidate will have a proven … of large data sets Data quality system and process design and implementation. Integrating the end to end data pipeline to take data from source systems to target data repositories ensuring the quality and consistency of data is more »

Head of Data 4 days a week/Gloucestershire Up to 100k Austin Fraser is working with a leading hospitality group, keen to bring on a Head of Data to make the chains within the groups more data driven. This is a new role … through growth and well suited to an individual with a mix of data, analytics and business acumen. They're looking for someone who is keen to remain hands-on and has strong experience working with Python, Azure and a strong understanding of data security and privacy … package which includes 33 days holiday, private medical insurance and stakeholder pension scheme. The role: Work alongside systems engineers to create and deploy data pipelines, data warehouses, and data lakes. Create and implement a robust data architecture strategy that aligns with business more »

Head of Data 4 days a week/Gloucestershire Up to 100k Austin Fraser is working with a leading hospitality group, keen to bring on a Head of Data to make the chains within the groups more data driven. This is a new role … through growth and well suited to an individual with a mix of data, analytics and business acumen. They're looking for someone who is keen to remain hands-on and has strong experience working with Python, Azure and a strong understanding of data security and privacy … package which includes 33 days holiday, private medical insurance and stakeholder pension scheme. The role: Work alongside systems engineers to create and deploy data pipelines, data warehouses, and data lakes. Create and implement a robust data architecture strategy that aligns with business more »

Description Due to continued growth we are currently looking for Senior Data Engineer to join our Professional Services division - you will be part of a cross functional Data consulting team spanning data engineering, visualisation, analytics and science. You will be working with our clients … across multiple sectors, helping them to explore next-generation data techniques and tools to drive incremental business value from their data assets. A day in the life of an Aiimi Senior Data Engineer: You will be responsible for collaborating with business subject matter experts … to discover the information hidden in various sources of content and data, helping our clients make smarter decisions to reduce service failure and deliver better outcomes to their customer base Your primary focus will be in connecting to and extracting data from source systems, applying business more »

business critical reporting systems across the Protein Works including marketing, trading, operations and finance. You will have a strong understanding and appreciation of data engineering concepts and how to apply them using the Google Cloud Platform. You will possess an unwavering passion for data and how … Engineer Immediate Responsibilities Managing and delivering end to end BI reporting projects from initial brief/requirement gathering to data pipeline development (APIs/ETL/ELT/SQL) to visualization Undertaking ad hoc BI tasks based on business requirements and priorities As part of … Coding experience with Python/Javascript/C# Familiarity with statistical/machine learning/AI concepts and techniques Understanding of data pipeline/orchestration tools e.g. dbt, dataform Appreciation of GCP’s serverless technologies e.g. Cloud Run/Workflows Understanding of Google’s marketing stack, Google more »

Data & Analytics Permanent - Full Time Newcastle-upon-Tyne, GB, NE991RN Salford, GB, M50 2HQ London, GB, W12 7TP BBC PUBLIC SERVICE B'CAST Job Closing Date: 30/04/2024 THE ROLE Job Band Package description Band: D Contract Permanent Salary : up to £97625.00 depending on level … that the BBC online becomes a valued daily habit for all audiences just as television and radio have been over the last century. Data is fundamental to our future: both in helping us prioritise and shape our work and in creating richer, more personalised experiences for our audiences. … we’ve got one of the widest, most diverse, and most exciting datasets to work within the UK. The role responsibilities The Principal Data Engineer is a new role that will support the newly created Product Data Domain teams. You will help to build ETL pipelines more »

nineDots are excited to be helping Cloudsmith ramp up their data engineering function. They are on a truly amazing upward trajectory with fantastic seasoned pros at the helm. It's gonna be a fun journey. Are you ready to lead the design and implementation of a best-in … class data platform on AWS?Doyou have a deep understanding of data engineering principles and a passion for creating scalable, efficient, and secure data solutions? If you're seeking a challenging and rewarding role, join Cloudsmith as our Principal Data Engineer! We … engineering (e.g., Redshift, S3, Glue, Lambda, Kinesis). ● Strong Programming Skills: Proficiency in languages like Python, SQL, and experience with data pipeline tools. ● DataModeling and Warehousing: Expertise in data modeling techniques and experience with data warehousing solutions. ● Excellent Problem-Solving Skills: Ability more »

Are you a Data Engineering professional with a passion for leading-edge technology and a knack for turning data into strategic assets? Look no further! We're on the hunt for a skilled Data Engineering Manager to drive the delivery of our cutting-edge … data strategy. As a key player in our team, you will wield significant influence in shaping and implementing our data strategy. Leading a dynamic team, you'll spearhead the construction, implementation, and maintenance of our data lake and data solutions. If you … armed with an in-depth understanding of data modelling, data warehousing, ETL (Extract, Transform, Load) processes, and data pipeline design. If you're fuelled by a passion for transforming data into strategic assets and ready to tackle the challenge of steering more »

My Prop-Fintech client is looking for a Data Engineer to join the Head of Data and 2 data analysts in the building, extraction, and refining of new and innovative datasets. This Geospatial Dataset contains billions of datapoints and has infinite possibilities of growth. … bonus, private healthcare for you and the family, Gym Benefits, and home office equipment. KEY RESPONSIBILITY Design and uphold an efficient data pipeline architecture. Enhance data storage platforms for optimal speed, scalability, reliability, and performance. Oversee critical functions such as data acquisition, extraction … preparation, and processing of extensive datasets for predictive modelling purposes. Conduct web scraping and proficiently manipulate data as required. Take the lead in pioneering modelling techniques utilising emerging technologies. Collaborate closely with data subject specialists to enhance data functionality and accessibility. Provide technical support more »