51 to 75 of 428 Data Pipeline Jobs in the UK

Data Engineer Consultant - Data & AI (DV Cleared/Eligible)65,000 – 75,000 + packageLocation: United Kingdom (London – London) Type: PermanentData Engineer Consultant - Data & AI (DV Cleared/Eligible)Location: LondonSalary: 65k - 75kAbout the Role:Our client, a leading consultancy in the Defence & Security … sector, is seeking a skilled Data Engineer to join their AI & Data Engineering team. This pivotal role offers the opportunity to be at the forefront of advanced technology projects aimed at enhancing national security. The position is perfect for individuals who are passionate about leveraging their … data security and quality across projects.Qualifications and Skills:Proven experience in data engineering, particularly with ETL processes, data pipeline construction, and workflow orchestration.Familiarity with distributed computing techniques, such as parallel processing and batch data management.Strong foundation in data system more »

Data Engineer Consultant - Data & AI (DV Cleared/Eligible)65,000 – 75,000 + packageLocation: United Kingdom (Bristol – Bristol) Type: PermanentData Engineer Consultant - Data & AI (DV Cleared/Eligible)Location: BristolSalary: 65k - 75kAbout the Role:Our client, a leading consultancy in the Defence & Security … sector, is seeking a skilled Data Engineer to join their AI & Data Engineering team. This pivotal role offers the opportunity to be at the forefront of advanced technology projects aimed at enhancing national security. The position is perfect for individuals who are passionate about leveraging their … data security and quality across projects.Qualifications and Skills:Proven experience in data engineering, particularly with ETL processes, data pipeline construction, and workflow orchestration.Familiarity with distributed computing techniques, such as parallel processing and batch data management.Strong foundation in data system more »

Data Engineer Consultant - Data & AI (DV Cleared/Eligible)65,000 – 75,000 + packageLocation: United Kingdom (Greater Manchester – Manchester, Greater Manchester) Type: PermanentData Engineer Consultant - Data & AI (DV Cleared/Eligible)Location: ManchesterSalary: 65k - 75kAbout the Role:Our client, a leading consultancy in … the Defence & Security sector, is seeking a skilled Data Engineer to join their AI & Data Engineering team. This pivotal role offers the opportunity to be at the forefront of advanced technology projects aimed at enhancing national security. The position is perfect for individuals who are passionate … data security and quality across projects.Qualifications and Skills:Proven experience in data engineering, particularly with ETL processes, data pipeline construction, and workflow orchestration.Familiarity with distributed computing techniques, such as parallel processing and batch data management.Strong foundation in data system more »

html Are you passionate about leveraging cutting-edge data technologies to drive business success? Do you thrive in dynamic environments where innovation and collaboration are valued? Position: Lead Data Engineer Lead Data Management and Architecture: Spearhead the design and implementation of robust data solutions across the organization, ensuring scalability and efficiency. Next Gen Data Platform Leadership: Act as the primary data engineer supporting the development of our next-generation data platform, collaborating closely with third-party vendors to build a highly scalable and robust data hub. Develop and Optimize Data Pipelines: Design, construct, install, test, and maintain highly scalable data management systems, ensuring they meet business requirements and industry best practices. Data Integration and Storage Solutions: Implement data storage solutions and manage large-scale processing more »

Senior Databricks Migration Consultant Remote This role demands in-depth knowledge of data engineering, cloud technologies (preferably AWS), and a successful record in enterprise level data migrations into Databricks. You will ensure the efficient and secure transition of our data assets to the Databricks … Lakehouse Platform, working closely with our external vendors. You will - Analyze data requirements, migration impacts, and project dependencies. - Develop detailed data migration strategies, including architectural plans, data quality evaluations, testing, and creating robust data pipelines and workflows for Databricks best practices. - Maintain … data integrity, security, and compliance with regulations like GDPR. - Manage data migration from legacy analytics data warehouses, data lakes, and ETL tools to Databricks within set deadlines. - Comfortable operating in all project phases, adjusting requirements, and supporting the transition from build and more »

Are you passionate about leveraging cutting-edge data technologies to drive business success? Do you thrive in dynamic environments where innovation and collaboration are valued? If so, we have an exciting opportunity for you! Position: Lead Data Engineer Responsibilities: Lead Data Management and Architecture … Spearhead the design and implementation of robust data solutions across the organization, ensuring scalability and efficiency. Next Gen Data Platform Leadership: Act as the primary data engineer supporting the development of our next-generation data platform, collaborating closely with third-party vendors … to build a highly scalable and robust data hub. Develop and Optimize Data Pipelines: Design, construct, install, test, and maintain highly scalable data management systems, ensuring they meet business requirements and industry best practices. Data Integration and Storage Solutions: Implement data more »

operational excellence of SEKO Logistics through the development of its core reporting infrastructure. The BI Analyst will be expected to design and build data a scalable data warehouse and data mart models using a combination of SQL, Power BI Modelling, and DAX. Experience of … Azure DevOps frameworks (or similar task management and source control systems) would be beneficial, and there would be opportunity to design and maintain data pipelines using Azure Data Factory, Spark, and other technologies. Key Responsibilities: Elicit requirements from end users Reverse engineer existing reports (BI platform … Work with users and other members of the global BI team to fully understand new requests Planning changes with thought of how the data will be reported and consumed by end users Design changes to the data model as required, following the agreed process and checks more »

Data Engineer We are seeking a Data Engineer to join our friendly team. Our client is building a new data platform centred around a Microsoft SQL server-based data warehouse that runs on AWS RDS. This role is crucial in expanding our … client's data capabilities and supporting the launch of our new platform and mobile app. Key Responsibilities: Support and maintain the central Data Platform, ensuring its availability and performance to meet business needs. Build data pipelines to connect APIs, files, databases, and data streams to our client's Data Platforms. Shape data from various sources into a transactional ODS for detailed reporting and a Kimball-modelled Data Warehouse for analytics and dashboards. Define and embed data quality measures within data pipelines. more »

Data Engineer We are seeking a Data Engineer to join our friendly team. Our client is building a new data platform centred around a Microsoft SQL server-based data warehouse that runs on AWS RDS. This role is crucial in expanding our … client's data capabilities and supporting the launch of our new platform and mobile app. Key Responsibilities: Support and maintain the central Data Platform, ensuring its availability and performance to meet business needs. Build data pipelines to connect APIs, files, databases, and data streams to our client's Data Platforms. Shape data from various sources into a transactional ODS for detailed reporting and a Kimball-modelled Data Warehouse for analytics and dashboards. Define and embed data quality measures within data pipelines. more »

Rate: £400-£450 per day Duration: 6 months Number of Positions: 1 Role Description: As a database expert, you will design solutions for data ingestion, transform data, optimize data storage, and ensure data integrity. You will also structure data for … easy consumption by BI tools. This role significantly impacts the organization by leveraging cutting-edge technology to enhance data management and analysis. Key Responsibilities: Develop ETL processes to extract data from various sources, transform it into the desired format, and load it into Azure Data Lake. Transform raw data into structured datasets, create models, and manage data pipelines using DBT Lab. Create scalable data transformation processes using dbt models for analytics-ready data. Ensure data exposed in business tools is populated by version-controlled, quality more »

Job Description: Data Engineer (Specialist/Professional) Work Location: London/Birmingham, UK Platform: GCP Position Overview: We are seeking a skilled and experienced GCP Data Engineer to join our team. As a GCP Data Engineer, you will be responsible for designing, implementing, and … infrastructure and pipelines on the Google Cloud Platform (GCP). Key Responsibilities: Design & Development: Design and implement data ingestion pipelines, data processing workflows, and data storage solutions using GCP services such as Cloud Storage, Big Query, Dataflow, Composer etc. Work with other … DEs to integrate data solutions into existing or new applications. Develop and manage data integration processes to extract data from various sources, transform and cleanse it, and load it into appropriate data storage systems. Implement robust ETL processes to ensure data more »

Data Engineer - AI & Cloud Data Engineering Industry : Defence & Security Location: London, Bristol or Manchester Clearance: All applicants must hold UK security clearance to Developed Vetting level (DV) Salary: £55,000 - £75,000 (DOE) (+£15k Benefits Package) As a practitioner in AI & Cloud Data Engineering, you are responsible for bridging the gap between our customers' needs and technical solutions. Your primary responsibilities are building large-scale data pipelines and infrastructure to ingest, transform, and organise raw data into accessible, trusted information assets. In your role, you will be responsible … engagement and your skillset, but could include and may require you to: • Utilize data integration tools and frameworks independently, adhering to standard pipeline authoring procedures. • Implement optimised solutions for complex data integration problems. • Design high-quality software applications in collaboration with other developers. • Demonstrate advanced more »

Data Engineer – London - £100,000 Description My client, a prominent figure in the insurance industry are on the hunt for a Data Engineer to join there ever growing Data team. The right individual will have a passion for Data as well as … being able to communicate to non-technical stakeholders. The right Data Engineer will ideally have experience working with Azure however my client are open to seeing people equally AWS/GCP. Responsibilities Designing and implementing scalable and efficient data storage solutions on Azure. Building and maintaining … with data scientists and analysts to understand their requirements and develop optimized data models. Monitoring and troubleshooting data pipeline performance issues. Implementing security measures to protect our data assets. Conducting regular data quality checks and applying necessary fixes. Keeping more »

We are seeking an experienced Python Data Engineer Consultant to join a leading hedge fund. The ideal candidate should have a proven ability to design and implement data solutions, with expertise in extracting, transforming, and loading (ETL) data from various sources, including websites, PDFs … and Real Time data streams. Role: Python Data Engineer Client: Leading Hedge Fund Type of role: contract (6 months) Location: London office Rate: up to £1500/day Key Responsibilities: Architect, develop, and optimize data pipelines for extracting, transforming, and loading data from diverse alternative data sources, such as web scrapes, PDF files, and Real Time data infrastructures. Implement and manage database solutions in SQL Server and various cloud-based environments, ensuring data integrity, efficiency, and accessibility. Build and maintain Real Time data more »

Senior Data Engineer 💰 £70,000-£80.000 + Bonus 📍Manchester city centre location – flexible hybrid working options ⭐ Benefits including 25 days holiday + bank holidays, private medical, 5% pension, life assurance We are currently partnered with a highly successful Manchester based SaaS business who have recently received over … million in funding to help them continue their upward trajectory. They are looking for a Senior Data Engineer who will get the chance to work on brand new projects using Version 3 of Microsoft Fabric. As a Senior Data Engineer, you will work closely with the … development team, legacy data administrator and the support engineers to help the business get the most out of their data. You will be expected to be the subject matter expert of all things to do with modern Azure tooling’s. 🚀 What you'll be doing... Developing high quality more »

Dive into a pivotal role leading data management and analytics transformation. We're on the hunt for an SC Cleared Azure Data Lead with a knack for innovation and a comprehensive skill set: MUST HAVE ACTIVE SC CLEARANCE Key Responsibilities Oversee the evolution of the data platform, encompassing data pipelines, storage, and processing. Establish and nurture robust relationships with stakeholders to guarantee a positive user experience and to identify any potential opportunities for additional services or products. Play a key role in the formulation and execution of the data management … record in the development and management of an Azure cloud-based data & analytics platform. Comprehensive experience in the complete data pipeline development process, encompassing data warehousing, data analytics, and ETL processes. Profound understanding of best practices in data governance more »

FT 35 hours per week Hybrid Home and Hub; London or Ashford Saga is recruiting a Lead Data Engineer for our central Data & Analytics team. You will be managing a team of Data Engineers and report directly to the Director of Data Engineering. Working as our Lead Data Engineer, youll be fully responsible for the implementation of data solutions and products within our Azure environments and technology stack. You will lead a team of Data Engineers, optimise data architecture, ensure data integrity and availability, and collaborate with stakeholders and partners to meet business objectives. We currently need our Lead Data Engineer to be a hands-on technical lead, able to work closely with our Data Solutions Architect, Technical BA, Scrum Master and Analytics team. The more »

of how people think, feel and act; globally and locally in over 90 markets. By combining the deep expertise of our people, our data resources and benchmarks and our innovative analytics and technology, we help our clients understand people and encourage growth. Backed by industry-leading anti-fraud … than 100 markets worldwide. Our clients include many of leading brands, who trust us to collect reliable insights. Join Kantar Profiles as a Data Engineer, where your technical expertise and collaborative spirit will contribute to the success of our data-driven initiatives. Apply now and be … to-day operations, involving ad hoc issue resolution, collaborative project-based work, and continuous improvement of data systems. Responsibilities Data Pipeline Development Collaborate with multi-functional teams to craft, develop, and maintain data pipelines. Ensure data integrity, reliability, and efficiency in more »

30+ years, Magnit offers the industry’s first holistic platform for the modern workforce. Magnit’s integrated workforce management (IWM) platform supported by data, software, intelligence, and best-in-class services team is key to our clients’ success. It can adapt quickly to regional or industry economic shifts … who collaborate and deliver meaningful results that positively transform the largest companies around the globe. About the Role We are seeking a Sr. Data Engineer with a minimum of 3 plus years of relevant work experience, preferably 5 plus years, to join our team. The ideal candidate has … a strong background in database management and would have the necessary skills to explore, analyze, and manipulate data from diverse sources (SQL, Oracle, etc.). What You Will Do • Explore and analyze data from a wide range of sources to extract actionable insights and drive data more »

Data Engineer (one day a week in the central London office) Sports Analytics This role is a unique opportunity to combine technical challenges with creativity in a collaborative, high-standard work environment. By joining this team, you’ll not only be part of a creative and open work … a detail-oriented innovator and team player, we’re excited to hear from you. Responsibilities Design, construct, install, test, and maintain highly scalable data management systems. Ensure systems meet requirements and industry practices for data quality and integrity. Integrate data management technologies and software … databases, as well as NoSQL databases Familiarity with cloud services (preferably GCP ) and understanding of its ETL frameworks. Desired : experience with data pipeline and workflow management (Airflow, Composer). Desired : Familiarity with machine learning algorithms and data science principles is a plus. Qualifications Degree in more »

understand client requirements and provide technical expertise during pre-sales engagements. Design and implement Proof of Concepts (POCs) to showcase the potential of data integration solutions and demonstrate value to clients. Prepare technical proposals and cost estimates for data integration projects. Conduct client presentations and answer … technical questions effectively. Design, develop, and deploy data pipelines for ingesting, transforming, and loading data from various sources into target systems (cloud and on-premises). Implement data cleansing, quality checks, and transformation logic to ensure data integrity and … accuracy. Configure and manage data integration tools, including cloud-based platforms and on-premises solutions. Automate workflows and schedule data pipeline execution for efficient data processing. Monitor data pipelines for performance and troubleshoot any issues. Collaborate with data analysts more »

business critical reporting systems across the Protein Works including marketing, trading, operations and finance. You will have a strong understanding and appreciation of data engineering concepts and how to apply them using the Google Cloud Platform. You will possess an unwavering passion for data and how … Engineer Immediate Responsibilities Managing and delivering end to end BI reporting projects from initial brief/requirement gathering to data pipeline development (APIs/ETL/ELT/SQL) to visualization Undertaking ad hoc BI tasks based on business requirements and priorities As part of … Coding experience with Python/Javascript/C# Familiarity with statistical/machine learning/AI concepts and techniques Understanding of data pipeline/orchestration tools e.g. dbt, dataform Appreciation of GCP’s serverless technologies e.g. Cloud Run/Workflows Understanding of Google’s marketing stack, Google more »

An exciting opportunity for an experienced Data Engineer to join the Engineering team working within the Digital & Product Department. Location: Chiswick (West London) Work Type: 3 days in office, 2 day from home Salary: £80k - £90k dependent on experience The successful candidate will lead a team of … data engineers to design and create scalable data pipelines. This hands-on role involves discussing requirements, designing solutions, and coding to automate data ingestion, cleansing, and transformation. You'll collaborate with Products, Engineering, Data, and Business Analytics teams to understand data requirements and develop integration solutions for necessary migrations. The Data Engineer will support software developers, PowerBI developers, data analysts, and data scientists, ensuring a consistent and optimal data delivery architecture across projects. Self-direction and comfort in supporting multiple teams more »

Data Engineer - Python - SQL - London - Up to £65k I'm currently working with an organisation who are globally recognised due to the nature of their work, despite only having a presence within the UK. My client are directly involved and help to organise annual events which attract every … provide career progression plans as well as extensive opportunities to gain industry recognised certifications. Experience required Python & SQL experience is essential Extensive experience in pipeline building, ideally using Azure Data Factory Background in Data Engineering Additional experience of Snowflake is advantageous This is a great … removed). Keywords: Azure, Azure Data Factory, MS Business Intelligence, MS Power BI, MS SQL Server, Snowflake, Python, SQL, Data Pipeline, Data Modelling, MDM, T-SQL, Data Lake, Logic Apps, Function App, London more »

Lead Data Engineer/Lead Data Developer - Azure, SQL What's it all about? Information, Systems and Change and Data and Digital Solutions. You will be responsible for leading the technical design and implementation of data integration projects using Microsoft Azure and … Integration Services alongside other cloud components. The Lead Data Engineer will also oversee the quality, performance, and security of the data integration solutions. How you'll make a difference By delivering data integration solutions on their data platform, you will contribute to … the continuous improvement of data-driven decisions that improve the services the client provides for their customers. How you'll do it Lead the Engineering Team in the development of data and integration solutions leveraging the latest technologies, tools, processes, and best practice. Create and provide more »