1 to 25 of 34 Data Pipeline Jobs in the West Midlands

Job Title: Senior Data Engineer Location: Warwick, Hybrid - 2 days per week onsite Remuneration: Up to £650 per day Contract Details: Fixed Term Contract (6 months) Working Pattern: Full Time Join a leading organisation dedicated to connecting people to the energy they use safely, reliably, and efficiently! We're on the lookout for a talented Senior Data Engineer to elevate our data capabilities and drive our projects forward. If you're passionate about data engineering and thrive in a dynamic environment, we want to hear from you! Responsibilities: Lead the design, development, and maintenance of large-scale data systems, including data warehouses, data lakes, and … data pipelines. Create and manage ETL processes using tools like Matillion for seamless data integration. Develop and maintain data warehouse objects such as tables, views, and security roles in Snowflake. Extract data from various sources, ensuring accuracy and consistency throughout the process. Transform and cleanse data, implementing data More ❯

Data Engineer Permanent role Birmingham Based – Hybrid working Competitive Salary Based on Experience + Bonus scheme Amtis is proud to be recruiting for a Data Engineer to join a forward-thinking business building modern, scalable data solutions. This is a hands-on role focused on building robust data infrastructure that supports high … performance analytics. Responsibilities: Build scalable, reusable data pipelines using modern cloud technologies Design secure, flexible data models for internal and external users Optimise performance across large, complex datasets Collaborate with analytics and visualisation teams to align data flows Integrate external data sources using bespoke and automated pipelines Support CI/CD and … Factory, Blob Storage Python/PySpark SQL Server, Parquet, Delta Lake Deep understanding of: ETL/ELT, CDC, stream processing Lakehouse architecture and data warehousing Scalable pipeline design and database optimisation A proactive mindset, strong problem-solving skills, and clear communication If you're an experienced data engineer and interested in the opportunity, please More ❯

Senior Data/Power BI Engineer West Midlands/HybridPermanent - Full Time Are you passionate about turning data into powerful insights? TXP is looking for a Senior Data/Power BI Engineer to help our clients unlock the full potential of their data. In … this role, you'll: Design and deploy Power BI models and dashboards Collaborate with stakeholders to shape intelligent analytics solutions Support data warehousing and data pipeline development Drive analytics maturity and best practices across client organisations Mentor junior team members and lead by example What you'll bring: Expertise in Power BI, DAX, and SQL … Experience with Microsoft Fabric or similar platforms Strong understanding of manufacturing processes and ERP systems A passion for data storytelling and business impact Join a consultancy that values innovation, diversity, and continuous learning. Be part of a team that's shaping the future of data-driven decision-making. Apply now and help us build smarter, data More ❯

Lead Data Architect £67,000 - 76,000 GBP Hybrid WORKING Location: Birmingham, West Midlands - United Kingdom Type: Permanent Lead Data Architect Locations: Leeds, Manchester, Birmingham, Bristol, Glasgow, or London (Hybrid - up to 2 days on-site per week, remainder remote) Salary: £67,000 - £76,000 (non-London)/Up to £83,000 (London) + Benefits A … respected digital consultancy is hiring a Lead Data Architect to join its growing Data Engineering practice. Known for delivering modern, human-centred services across public and private sectors, this organisation supports clients through meaningful transformation by combining data, design, and engineering. This role is ideal for experienced architects who enjoy shaping cloud-native data platforms, working in multidisciplinary teams, and embedding data governance and architectural rigour across complex service ecosystems. The Role: As a Lead Data Architect, you'll play a key role in designing scalable, secure, and accessible data platforms that enable insight-driven decision-making. You'll work closely with engineering teams and client More ❯

Data Engineer//1 Day/Week Onsite Worcestershire//Initial 3 Months//Up to £400/day Outside IR35 REED Technology are working with a client who are seeking a contract Data Engineer to join their dynamic Data Engineering team. You'll play a key role in designing and … implementing complex data flows that power analytics and business intelligence (BI) systems across their growing financial services organisation. Key responsibilities Designing, building, and maintaining robust data pipelines and platforms Collaborating with BI, tech, and business teams to deliver scalable solutions Leading on the implementation of new data infrastructure Analysing data for integrity … and correlation across systems Translating stakeholder requirements into technical deliverables Producing and maintaining high-quality technical documentation Championing data engineering best practices and standards across the business Technical skills Cloud data platforms - Azure, AWS, or GCP (Azure preferred) Snowflake - Deep knowledge and hands-on experience Matillion - Expertise in ETL orchestration Data warehousing and advanced More ❯

you're inspired to think big and bring your ambition to work every day, which is why, at British Airways the sky is never the limit. The role: Data Scientist/Senior Data Scientist Join British Airways as a Data Scientist in our cutting-edge Operations Decision Support (ODS) product team. This full-stack … role sits at the heart of our transformation, combining data science, machine learning, and optimisation to build industrialised decision-support software. If you're passionate about solving real-world problems through intelligent, data-driven solutions and thrive in a collaborative, agile environment-this is your opportunity to make a real impact. What you'll do: Design … with software engineering teams, product & change teams to turn their models into industrialized products that are used by the airline every day to make better decisions. Build robust data pipelines, automate data cleaning, and deliver scalable, production-grade Python code using best practices such as typing, testing, and modular design. Integrate models into orchestration frameworks (e.g. More ❯

Contribute to security engineering projects, transitions, and transformations. Work closely with security operations and associated security incident response systems Stay informed about emerging threats and security best practices. Data Ingestion and Enrichment: Configure Elasticsearch pipelines for data ingestion from various sources, primarily from Kafka Enhance data enrichment by integrating threat intelligence feeds and contextual … towards achieving these. Skills Required for the Role Security and Compliance with Elastic Security: Set up access controls, authentication, and encryption using Elastic Security features. Ensure compliance with data protection regulations. Detection Rule Development: Ability to create, test, and optimise detection rules to identify suspicious activities and potential threats based on the MITRE ATT&CK Framework Performance Tuning … with Elasticsearch and Logstash: Fine-tune query performance using Elasticsearch indices and mappings. Monitor Logstash pipelines and optimize resource utilization. Kibana Visualization and Monitoring: Leverage Kibana for data visualization, dashboards, and real-time monitoring. Create custom visualizations to track data quality metrics and system performance. Elastic integration Integration of the SIEM with critical systems to provide More ❯

Contribute to security engineering projects, transitions, and transformations. Work closely with security operations and associated security incident response systems Stay informed about emerging threats and security best practices. Data Ingestion and Enrichment: Configure Elasticsearch pipelines for data ingestion from various sources, primarily from Kafka Enhance data enrichment by integrating threat intelligence feeds and contextual … achieving these. Skills Required for the Role Essential: Security and Compliance with Elastic Security: Set up access controls, authentication, and encryption using Elastic Security features. Ensure compliance with data protection regulations. Detection Rule Development: Ability to create, test, and optimise detection rules to identify suspicious activities and potential threats based on the MITRE ATT&CK Framework Performance Tuning … with Elasticsearch and Logstash: Fine-tune query performance using Elasticsearch indices and mappings. Monitor Logstash pipelines and optimize resource utilization. Kibana Visualization and Monitoring: Leverage Kibana for data visualization, dashboards, and real-time monitoring. Create custom visualizations to track data quality metrics and system performance. Elastic integration Integration of the SIEM with critical systems to provide More ❯

Data Engineer (Minerva MMTAR Functional Core Build) Max Pay: £537 SC is required, Duration: 6 months initially Location: Telford: If local to Telford, requirement will be 2 days per week on site. If not local, expectation will be to attend once/twice a month. Via Umbrella Key Responsibilities: Design, development, and deployment of data integration … and transformation solutions using Pentaho, Denodo, Talend, and SAS. Architect and implement scalable data pipelines and services that support business intelligence and analytics platforms. Collaborate with cross-functional teams to gather requirements, define technical specifications, and deliver robust data solutions. Champion Agile and Scrum methodologies, ensuring timely delivery of sprints and continuous improvement. Drive DevOps practices … for CI/CD, automated testing, and deployment of data services. Mentor and guide junior engineers, fostering a culture of technical excellence and innovation. Ensure data quality, governance, and security standards are upheld across all solutions. Troubleshoot and resolve complex data issues and performance bottlenecks. Key Skills: SAS 9.4 (DI), SAS Viya 3.x (SAS More ❯

Viya 4. Support the migration of legacy SAS platforms to cloud-native SAS Viya 4 environments, including namespace separation and Kubernetes orchestration. Implement and manage configuration objects, service data, and translation objects within SAS environments. Collaborate with data engineers and DevOps teams to integrate SAS with Oracle, GitLab CI/CD pipelines, and other enterprise platforms. … Monitor and optimise platform performance, ensuring high availability and compliance with security standards. Provide technical support and troubleshooting for SAS users and developers. Automate environment provisioning and data transfers using approved EA integration patterns. Participate in community knowledge sharing and contribute to best practice documentation. Essential Skills and Experience: SAS Enterprise Guide (EG) – workflow orchestration and user support. … Viya 4 – platform configuration, migration, and containerisation (Kubernetes). SAS Visual Analytics (VA) and SAS Visual Investigator (VI) – dashboard and investigation tool support. Oracle – integration and data pipeline support. Configuration Management – experience managing service objects, deployment artefacts, and environment baselines. Testing Lifecycle Knowledge – including test strategy design, environment comparison utilities, and automated testing integration. Desirable Attributes: Experience More ❯

Viya 4. Support the migration of legacy SAS platforms to cloud-native SAS Viya 4 environments, including namespace separation and Kubernetes orchestration. Implement and manage configuration objects, service data, and translation objects within SAS environments. Collaborate with data engineers and DevOps teams to integrate SAS with Oracle, GitLab CI/CD pipelines, and other enterprise platforms. … Monitor and optimise platform performance, ensuring high availability and compliance with security standards. Provide technical support and troubleshooting for SAS users and developers. Automate environment provisioning and data transfers using approved EA integration patterns. Participate in community knowledge sharing and contribute to best practice documentation. Essential Skills and Experience: SAS Enterprise Guide (EG) – workflow orchestration and user support. … Viya 4 – platform configuration, migration, and containerisation (Kubernetes). SAS Visual Analytics (VA) and SAS Visual Investigator (VI) – dashboard and investigation tool support. Oracle – integration and data pipeline support. Configuration Management – experience managing service objects, deployment artefacts, and environment baselines. Testing Lifecycle Knowledge – including test strategy design, environment comparison utilities, and automated testing integration. Desirable Attributes: Experience More ❯

Viya 4. Support the migration of legacy SAS platforms to cloud-native SAS Viya 4 environments, including namespace separation and Kubernetes orchestration. Implement and manage configuration objects, service data, and translation objects within SAS environments. Collaborate with data engineers and DevOps teams to integrate SAS with Oracle, GitLab CI/CD pipelines, and other enterprise platforms. … Monitor and optimise platform performance, ensuring high availability and compliance with security standards. Provide technical support and troubleshooting for SAS users and developers. Automate environment provisioning and data transfers using approved EA integration patterns. Participate in community knowledge sharing and contribute to best practice documentation. Essential Skills and Experience: SAS Enterprise Guide (EG) - workflow orchestration and user support. … Viya 4 - platform configuration, migration, and containerisation (Kubernetes). SAS Visual Analytics (VA) and SAS Visual Investigator (VI) - dashboard and investigation tool support. Oracle - integration and data pipeline support. Configuration Management - experience managing service objects, deployment artefacts, and environment baselines. Testing Lifecycle Knowledge - including test strategy design, environment comparison utilities, and automated testing integration. Desirable Attributes: Experience More ❯

Excel VBA. Experience with CI/CD pipelines and Azure DevOps is also valued. Key Responsibilities: - Develop serverless applications using Cloud Functions (Python or Node.js) - Build and optimise data pipelines with Cloud Dataflow - Manage relational databases via Cloud SQL (PostgreSQL/MySQL) - Implement workflows using GCS triggers and IAM permissions - Integrate APIs using Apigee - Create Excel-based automation More ❯

ranks 189 on the Fortune 500. Further information about Lear is available at or on Position overview: As an Application Solution Architect, you will design and implement scalable, data-driven solutions using Palantir Foundry. You'll work closely with business stakeholders, data engineers, and developers to translate business needs into robust technical architectures that leverage Foundry … technical constraints. Design scalable and efficient data models and ontologies within Foundry. Define and oversee the development of data pipelines and workflows using Foundry's Pipeline Builder and Code Repositories. Lead integration of external systems and APIs into Foundry for seamless data flow. Work with cross-functional teams to gather requirements, define architecture … TECHNICAL/PERSONNEL SKILLS : Experience designing scalable data models and ontologies. Understanding of semantic data modelling and entity relationships. Familiarity with Foundry's Ontology SDK, Pipeline Builder, and Code Workspaces. Experience integrating external systems and APIs into Foundry. Knowledge of data transformation, cleansing, and enrichment techniques. Awareness of data privacy, access More ❯

days onsite per week End Date: 28/11/2025 Role Overview: We are seeking a highly skilled Hadoop Engineer to support and enhance our Operational Data Platform (ODP) deployed in an on-premises environment. The ideal candidate will have extensive experience in the Hadoop ecosystem, strong programming skills, and a solid understanding of infrastructure-level data analytics. This role focuses on building and maintaining scalable, secure, and high-performance data pipelines within enterprise-grade on-prem systems. Key Responsibilities: Design, develop, and maintain data pipelines using Hadoop technologies in an on-premises infrastructure. Build and optimise workflows using Apache Airflow and Spark Streaming for real-time data processing. … Develop robust data engineering solutions using Python for automation and transformation. Collaborate with infrastructure and analytics teams to support operational data use cases. Monitor and troubleshoot data jobs, ensuring reliability and performance across the platform. Ensure compliance with enterprise security and data governance standards. Required Skills & Experience: Minimum 5 years of experience More ❯

version control systems like Azure DevOps is a strong plus. Main responsibilities: Design, develop, and deploy applications using Google Cloud Functions (Python or Node.js). Implement and optimize data pipelines using Google Cloud Dataflow. Manage and query relational databases using Cloud SQL (PostgreSQL/MySQL). Handle file operations and trigger-based workflows using Google Cloud Storage. Configure More ❯

you the chance to help make the UK a safer place through innovation, working in partnership with government and law enforcement organisations on some of the most challenging data and technology problems out there. We strongly encourage candidates of all different backgrounds and identities to apply. We are committed to building an inclusive, safe and supportive environm ent … Here are some things we've worked on recently that might give you a better sense of the kind of work you could be leading on: Building a data analytics tool using a Neo4J graph database. A DevOps platform to support 3000 users. Processing digital forensic data to enable search and insight generation. A search and … reporting system backed by an ElasticSearch cluster. A custom workflow tool using Camunda, with a React/TS front-end An AI assurance capability Orchestrating a data pipeline and designing specialist metrics for the evaluation of the performance of LLMs in different contexts. Accountabilities: Understands the technical aspects of the project and the wider customer business model. More ❯

Data Engineer - Award-Winning AI SaaS platform We're partnered with a pioneering SaaS company that's transforming their industry through cutting-edge AI and data technology. With a multi-award-winning platform and a client base that includes global household names, this is a rare opportunity to join a high-growth tech business that blends … startup agility with enterprise stability. We're looking for a passionate Data Engineer who is experienced with AI tooling - including building Gen AI agents in-product - to join their growing Data Engineering ream. Why You'll Love This Role: Tech at Scale : Work with hundreds of millions of data points daily, using distributed systems … office if you prefer Exceptional Benefits : From unlimited holiday and private healthcare to stock options and paid parental leave. What You'll Be Doing: Build and maintain scalable data pipelines using Spark with Scala and Java, and support tooling in Python Design low-latency APIs and asynchronous processes for high-volume data. Collaborate with Data Science More ❯

We are a Global Recruitment specialist that provides support to the clients across EMEA, APAC, US and Canada. We have an excellent job opportunity for you. Title: Data Visualisation Engineer Clearance required: Must hold an Active SC Clearance Location: Telford/Hybrid 6 months contract Job description: Purpose: This project seeks to streamline that experience by delivering a … a new scrum team within Minerva Platform to develop and deliver the Ingestion and Risking within the SAS Platform including IDP. Key Responsibilities: Design, development, and deployment of data integration and transformation solutions using Pentaho, Denodo, Talend, and SAS. Architect and implement scalable data pipelines and services that support business intelligence and analytics platforms. Collaborate with … cross-functional teams to gather requirements, define technical specifications, and deliver robust data solutions. Champion Agile and Scrum methodologies, ensuring timely delivery of sprints and continuous improvement. Drive DevOps practices for CI/CD, automated testing, and deployment of data services. Mentor and guide junior engineers, fostering a culture of technical excellence and innovation. Ensure data More ❯

Cambridge, Gatwick, Milton Keynes, Reading Business Line Enabling Functions Job Type Permanent/FTC Date published 10-Jun-2025 19366 Connect to your Industry Are you passionate about data science and AI? Do you want to apply your skills and knowledge to help shape the future of Deloitte? If so, we have an exciting opportunity for you to … join our Enabling Functions team as an AI Architect & Data Scientist. You be part of an exciting innovative team that delivers cutting edge GenAI solutions. As an AI Architect & Data Scientist, you will work on innovative projects that leverage data and AI to enhance our internal capabilities and deliver value to our employees. You … values lead every decision wemake and action we take, guiding us to deliver impact how and where it mattersmost . Connect to your opportunity As an AI Architect & Data Scientist, you will be expected to: Design and build Agentic AI and GenAI solutions, from PoC to production, using agile methodologies and best practices. Apply advanced analytical techniques, such More ❯

impactful projects that leverage the latest in cloud-native technologies. Key Responsibilities: Design, develop, and deploy applications using Google Cloud Functions (Python or Node.js). Implement and optimise data pipelines using Google Cloud Dataflow. Manage and query relational databases using Cloud SQL (PostgreSQL/MySQL). Handle file operations and create trigger-based workflows using Google Cloud Storage. More ❯

a new scrum team within Minerva Platform to develop and deliver the Ingestion and Risking within the SAS Platform including IDP. Key Responsibilities: Design, development, and deployment of data integration and transformation solutions using Pentaho, Denodo, Talend, and SAS. Architect and implement scalable data pipelines and services that support business intelligence and analytics platforms. Collaborate with … cross-functional teams to gather requirements, define technical specifications, and deliver robust data solutions. Champion Agile and Scrum methodologies, ensuring timely delivery of sprints and continuous improvement. Drive DevOps practices for CI/CD, automated testing, and deployment of data services. Mentor and guide junior engineers, fostering a culture of technical excellence and innovation. Ensure data quality, governance, and security standards are upheld across all solutions. Troubleshoot and resolve complex data issues and performance bottlenecks. Key Skills: SAS 9.4 (DI), SAS Viya 3.x (SAS Studio, VA, VI). Platform LSF, Jira, Platform Support. GIT. Strong expertise in ETL tools: Pentaho, Talend. Experience with data virtualisation using Denodo. Proficiency in SAS More ❯

a new scrum team within Minerva Platform to develop and deliver the Ingestion and Risking within the SAS Platform including IDP. Key Responsibilities: Design, development, and deployment of data integration and transformation solutions using Pentaho, Denodo, Talend, and SAS. Architect and implement scalable data pipelines and services that support business intelligence and analytics platforms. Collaborate with … cross-functional teams to gather requirements, define technical specifications, and deliver robust data solutions. Champion Agile and Scrum methodologies, ensuring timely delivery of sprints and continuous improvement. Drive DevOps practices for CI/CD, automated testing, and deployment of data services. Mentor and guide junior engineers, fostering a culture of technical excellence and innovation. Ensure data quality, governance, and security standards are upheld across all solutions. Troubleshoot and resolve complex data issues and performance bottlenecks. Key Skills: SAS 9.4 (DI), SAS Viya 3.x (SAS Studio, VA, VI). Platform LSF, Jira, Platform Support. GIT. Strong expertise in ETL tools: Pentaho, Talend. Experience with data virtualisation using Denodo. Proficiency in SAS More ❯

a new scrum team within Minerva Platform to develop and deliver the Ingestion and Risking within the SAS Platform including IDP. Key Responsibilities: Design, development, and deployment of data integration and transformation solutions using Pentaho, Denodo, Talend, and SAS. Architect and implement scalable data pipelines and services that support business intelligence and analytics platforms. Collaborate with … cross-functional teams to gather requirements, define technical specifications, and deliver robust data solutions. Champion Agile and Scrum methodologies, ensuring timely delivery of sprints and continuous improvement. Drive DevOps practices for CI/CD, automated testing, and deployment of data services. Mentor and guide junior engineers, fostering a culture of technical excellence and innovation. Ensure data quality, governance, and security standards are upheld across all solutions. Troubleshoot and resolve complex data issues and performance bottlenecks. Key Skills: SAS 9.4 (DI), SAS Viya 3.x (SAS Studio, VA, VI). Platform LSF, Jira, Platform Support. GIT. Strong expertise in ETL tools: Pentaho, Talend. Experience with data virtualisation using Denodo. Proficiency in SAS More ❯

a new scrum team within Minerva Platform to develop and deliver the Ingestion and Risking within the SAS Platform including IDP. Key Responsibilities: Design, development, and deployment of data integration and transformation solutions using Pentaho, Denodo, Talend, and SAS. Architect and implement scalable data pipelines and services that support business intelligence and analytics platforms. Collaborate with … cross-functional teams to gather requirements, define technical specifications, and deliver robust data solutions. Champion Agile and Scrum methodologies, ensuring timely delivery of sprints and continuous improvement. Drive DevOps practices for CI/CD, automated testing, and deployment of data services. Mentor and guide junior engineers, fostering a culture of technical excellence and innovation. Ensure data quality, governance, and security standards are upheld across all solutions. Troubleshoot and resolve complex data issues and performance bottlenecks. Key Skills & Experience: SAS 9.4 (DI), SAS Viya 3.x (SAS Studio, VA, VI). Platform LSF, Jira, Platform Support. GIT. Strong expertise in ETL tools: Pentaho, Talend. Experience with data virtualization using Denodo. Proficiency in More ❯