1 to 25 of 463 Data Pipeline Jobs in the UK

About the Job you are considering: We are looking for an experienced Data Architect with strong expertise in Azure Databricks, Azure Data Factory, Azure Data Lake, Delta Lake, data modelling, data governance, and cloud data architecture, along with … domain experience in London Market Insurance. The candidate will be responsible for defining and delivering scalable data architecture solutions for insurance transformation programmes covering underwriting, claims, policy administration, bordereaux, delegated authority, premium processing, regulatory reporting, reinsurance, analytics, and data migration.The ideal candidate should have strong hands ...

Engineer, Lead Data Engineer, Data Engineer, ETL, AWS, AWS Glue, AWS Lambda, AWS S3, AWS Redshift, Amazon EMR, Python, SQL, Pipeline Orchestration, PySpark, data warehousing, data lakehouse, data architecture, Agile, Senior Data Engineer, Lead Data … proficiency, including experience with AWS services such as AWS Glue, AWS Lambda, AWS S3, AWS Redshift, and Amazon EMR. Proven expertise in Python, SQL, Pipeline Orchestration and PySpark is also expected, as is experience with data lakehouse architectures and data warehousing solutions. Agile experience ...

Principal Data Architect Location: Remote Duration: 6 months £700 - £800 per day - Umbrella only Our client is looking for Principal Data Architect for Data Science family under DSF as part of Professional services sell. This request for NGET under the department of Customer Network … GDPR and Regulatory/CNI control environments In-depth experience in providing resolution to an extensive range of complicated data pipeline related problems, proactively and as issues surface. Demonstrable ability to troubleshoot problems across infrastructure, platform, and application domains. Controlling access to the data ...

summary Are you a strong Data Engineer who builds reliable pipelines, improves data quality and delivers trusted data products? BPAS is recruiting a Data Engineer to join the growing Business Intelligence team at an important stage of development. You will work with … Software Engineering, IT and business stakeholders to deliver secure, practical data solutions. Reporting to the BI Lead, you will strengthen the data foundations that support reporting, analytics, service improvement and decision-making. You will design and maintain robust data pipelines and analytical data ...

Manpower are currently seeking an interim Data & Analytics Analyst to work with our global FMCG client The Magnum Ice Cream Company, renowned for brands such as Magnum, Ben & Jerry's, Cornetto and Wall's, and become an integral part of their fast-paced FMCG environment. The position … wholesale partners and data feeds Ensure consistent, accurate mapping of SKU, customer, and channel data Data Ingestion & Pipeline Collaboration Support and oversee data ingestion processes for wholesaler and partner data Work closely with data engineers to bring ...

Senior Data Engineer Remote UK | Occasional Travel Senior Data Engineer | Azure | Microsoft Fabric | Data Platform | Python | SQL | Power BI | AI & Analytics 167 Solutions are partnering with a global organisation embarking on a significant data transformation programme. As investment in analytics … business intelligence continues to grow, we are seeking a Senior Data Engineer to play a key role in shaping and delivering a modern enterprise data platform. This opportunity offers the chance to work on large-scale data initiatives, modern cloud technologies and next-generation ...

Staff-level Data & Agentic AI engineer to join the programme - driving the agentic AI based conversion framework & Data pipeline modernisation, partnering across a Senior team of hands on Data Engineers, and driving the programme to completion with the quality and rigour that … engineering organisation can own and easily maintain into the future. What you’ll do Shape the technical direction for the Data pipeline modernisation and re-engineering programme: Define architecture, standards, engineering approach, and testing framework Ensure coherence across all workstreams from discovery through to validated delivery ...

Title: Senior Data Engineer Location: Warwick (Hybrid Working) Contract: Initial 6 Months Pay Rate: £550 - £650 (Umbrella) Role Overview Our client is seeking a hands-on Senior Data Engineer with deep experience in enterprise data warehousing and data integration. This role … develop, and maintain ETL/ELT processes utilising tools such as Matillion. Create and maintain optimal data flows and data pipeline architectures. Create and implement semantic data models, including fact and dimension tables. Own the end-to-end data lifecycle: extraction ...

contribute to the shaping, validation, and monitoring of data solutions. Your work ensures accuracy, clarity, and integrity across the entire delivery pipeline RESPONSIBILITIES Data Analysis & Insight Development Exploratory Analytics: Conduct detailed exploratory data analysis to identify trends, root causes, and opportunities for improvement … Insight Packaging: Translate complex findings into clear, concise insights, visualisations, and recommendations for both technical and non-technical stakeholders. 2. Data Preparation & Pipeline Support Data Wrangling: Prepare, clean, and transform datasets to support modelling, prototyping, and production-ready pipelines. Requirements Translation: Work with the Data ...

Write efficient SQL queries and manage large datasets Ensure data quality, integrity, and security across systems Monitor and troubleshoot data pipeline performance issues Collaborate with data analysts, data scientists, and software engineers Implement data governance and best practices Support … Strong proficiency in SQL and database technologies Experience with programming languages such as Python, Java, or Scala Hands-on experience with data pipeline tools (e.g., Apache Airflow, Kafka) Familiarity with cloud platforms (AWS, Azure, Google Cloud) Understanding of data warehousing solutions (e.g., Snowflake, Redshift, BigQuery ...

Senior Data Engineer 6 Month contract with extension Full-time Inside IR35 Hybrid 1 day every two weeks in the office Nottingham - NG1 4BX Join our client in embarking on an ambitious data transformation journey using Databricks, guided by best practice data governance … warehousing. Able to write repeatable, production-quality code for data pipelines, utilizing templating and parameterization where needed. Can make data pipeline design recommendations based on business requirements. Experience with data migration is a plus. Open to new ways of working and new technologies. ...

Oracle, SQL Server, MySQL, or cloud-native systems) combined with deep expertise in AWS data engineering services and modern data pipeline development. Key Responsibilities: Lead end to end database migrations to AWS from diverse sources, including on-premises systems (e.g., Oracle, SQL Server, MySQL … cloud to cloud). Experience designing and implementing data validation and reconciliation frameworks. Strong experience in ETL/data pipeline development (Glue, PySpark, EMR, or equivalent). Deep proficiency in SQL and data modeling concepts. Experience working with S3-based data ...

Department: IT Type: 3-month rolling contract Outside IR35 Position: Analytics Engineer The organisation is seeking an experienced Analytics Engineer to join its Data & Analytics team. The role focuses on building scalable data pipelines, transforming raw data into clean, reusable datasets, and enabling data-driven decision-making. Key Responsibilities Design and build data products, with proficiency throughout the data lifecycle. Develop robust data models through close collaboration with business users and the engineering team. Partner with senior management, operational leads, and other stakeholders, coaching and supporting ...

Lead Data Engineer Location: London, City (Hybrid) £ Highly competitive base with bonus and benefits Lead Data Engineer is sought by expanding central data platform team within a prestigious global wealth management organisation. This role offers the opportunity to help shape enterprise-wide data engineering standards, contribute to platform strategy, and build modern, scalable data solutions that support business-critical decision-making. Essential to this role are strong python skills, expertise in MS Fabric, SQL, data warehousing, Data Lake and Azure technologies. The Role ...

with engineering best practices and the ability to translate customer expectations into applied technical functionality are key to success in this role. Key responsibilitiesData pipeline development Build and maintain scalable data pipelines using Apache Spark and PySpark, processing and transforming large datasets across distributed cloud infrastructure. Workflow … monitoring, and execution of data processing workflows. Root cause analysis Perform data analysis to identify and resolve root causes of pipeline failures and data quality issues - including reviewing EMR output logs and CloudWatch metrics. Data modelling Apply understanding of dimensional data ...

Data Architect UK-Wide | Permanent | Remote-First Our client is a leading global technology and consulting organisation , recognised as one of Fortune’s World’s Most Admired Companies , delivering large‐scale transformation across government and public sector environments. Due to the nature of these programmes, active SC clearance … required for this role . This opportunity sits within a growing Data & AI Architecture capability , focused on delivering complex, high‐impact programmes at the heart of government. The role combines deep technical architecture expertise with leadership, consulting, and delivery oversight . You’ll have the opportunity to: Lead ...

Data Architect £Up to £120,000 GBP Competitive Bonus + Car Allowance Hybrid WORKING Location: London; Bristol; Manchester, Central London, Greater London - United Kingdom Type: Permanent Data Architect London | Manchester | Bristol | Hybrid | Permanent Our client is a leading global technology and consulting organisation , recognised … environments. Due to the nature of these programmes, active SC clearance is required for this role . This opportunity sits within a growing Data & AI Architecture capability , focused on delivering complex, high-impact programmes at the heart of government. The role combines deep technical architecture expertise with leadership ...

Data Architect £Up to £120,000 GBP Competitive Bonus + Car Allowance Hybrid WORKING Location: London; Bristol; Manchester, Central London, Greater London - United Kingdom Type: Permanent Data Architect London | Manchester | Bristol | Hybrid | Permanent Our client is a leading global technology and consulting organisation , recognised … environments. Due to the nature of these programmes, active SC clearance is required for this role . This opportunity sits within a growing Data & AI Architecture capability , focused on delivering complex, high-impact programmes at the heart of government. The role combines deep technical architecture expertise with leadership ...

Data Engineer (Databricks) – AI/Data Consulting Firm About Us We are an ambitious consulting firm focused on delivering cutting-edge solutions in data and AI. Our mission is to empower organisations to unlock the full potential of their data by leveraging … Optimise data warehouse and lakehouse architectures for performance and cost efficiency. Maintain and improve CI/CD processes for data pipeline deployment and monitoring. What We Are Looking For: 5+ years of experience in data engineering or related roles. Strong expertise in Databricks ...

seeking an experienced Data Architect to join our Group Technology team on a 6-month contract. You will help realise the value of the immense Connells Group data landscape for the benefit of customers, colleagues and the overall business. You will use your expertise to define … evolve and communicate the existing Estate Agency enterprise data architecture, to democratise safe access to consolidated and unified data, modelled to help engineering teams develop single customer and property views. The data architecture will also support system migration and integration, to establish effective master ...

Data Engineer £45,000 to 55,000 GBP Bonus Hybrid WORKING Location: Central London, Greater London - United Kingdom Type: Permanent Data Engineer - London Salary: Up to £53,000 + bonus + benefits Location: London (Hybrid) Security Clearance: SC eligible (must be sole UK citizen … seeking a Data Engineer to join a high-performing digital and data team working on secure, complex data environments across government and regulated sectors. This is an opportunity to work on meaningful data platforms that support large-scale transformation programmes, helping organisations ...

Media, Retail and CPG, and Public Services. Consolidated revenues as of $13+ billion. For more information on how we process your personal data, please refer to HCLTech’ s Candidate Data Privacy Notice. Experience - 8+ years Hybrid - 3 days/week onsite in London Investment Banking … prerequisite. Must Have- FIC/Front Office Exposure Experience with Fixed Income/Currencies/Sales/Trading data Worked closely with front-office stakeholders Data Architecture/Engineering Awareness Understanding of data pipelines, ETL, reporting layers Exposure to data platforms ...

xperience and Education Required Approximately 10 years of industry experience implementing data integration tools such as Microsoft SSIS, Azure Data Factory, Databricks, Glue, Step Functions, Airflow, Apache Flume/Sqoop/Pig, and similar technologies. 3–5 years of team management experience. 3–5 years … Warehouse), Snowflake, Redshift, and others. Strong experience with orchestration tools such as Azure Data Factory, HDInsight, Cloud Composer, Data Pipeline, or equivalent. Understanding of key data architecture methodologies (Dimensional, ODS, Data Vault, etc.). Familiarity with modern data ...

Lead Data Engineer/Data Tech Lead Permanent, Leeds/Hybrid Who are we? Enablis is a technology delivery consultancy who partner with clients to accelerate digital roadmaps. By combining our deep knowledge and best practice experience with the client’s domain expertise, we are able … open the door to incredible opportunities and introduce you to a network of equally driven, like-minded professionals. Job Brief The Lead Data Engineer role is responsible for owning the technical direction, design and delivery of data platforms and data engineering solutions ...

Data Platform Manager UK/Remote Circa £70k per annum + 15% bonus & benefits Are you ready to take ownership of a data platform and shape how data drives business decisions? This is a rare opportunity to step into a leadership role where … combine hands-on data engineering with strategic product management, all while building and mentoring a high-performing team. You'll play a pivotal role in evolving a modern data platform (Microsoft Fabric/Azure Synapse), ensuring it delivers real business value, scalability, and innovation. ...