22 Data Pipeline Jobs in the West Midlands

Data Engineer – Hybrid Working £50,000 - £55.000 30 Days Holidays + 4 Days Christmas + Bank Holidays Great Pension Career Development Flexible Working - (Hybrid) A Great Working Environment Excellent Sport & Leisure Facilities Training and Personnel Development Offerings Office Location is close to A45, A46, M1, M6, M40, M42 … implement data security measures Design and implement data quality checks and validation processes within the pipelines using expectations with the pipeline, or dedicated testing tools (SODA) Collaborate with data governance, architects, system/product owners and end users to ensure quality issues are … addressed as needed. Create and maintain technical documentation, including data pipeline specifications, data models, and transformation logic Contribute to data modelling and schema design standards and related documentation. Qualifications & Requirements Degree in Computer Science, Data Engineering, or a related field or more »

Senior Data Engineer Role purpose: The Senior Data Engineer plays a pivotal role within the Data team of the Digital and Information function, assuming a leadership position in ensuring the availability, integrity, quality, and accuracy of data. With a wealth of experience, the Senior … Data Engineer leads the design, development, and maintenance of data pipelines and workflows, driving the organisation's data processing capabilities and enabling the seamless delivery of BI reporting and analytics. In addition to core responsibilities, the Senior Data Engineer takes a proactive … stakeholders, to comprehensively understand data requirements, conduct User Acceptance Testing (UAT), and deliver tailored solutions that align with business objectives. Proactively monitor pipeline performance, swiftly identifying and resolving any potential issues to uphold seamless data processing operations and minimize downtime. Drive comprehensive documentation of data more »

Data Engineer – Hybrid (3 days in Office) Stratford-Upon-Avon £56k + 5% bonus & benefits Do you want to grow your expertise and experience and use your skills in a vibrant environment where teamwork, creativity, diversity, inclusivity, and technical excellence are expected and enjoyed? This SaaS company are … associates to principals, applying your technical and problem-solving capabilities to scale their applications. You’ll contribute to the organisation’s high-quality data engineering practices, participating in technical design, implementation, testing, deployment, and ongoing support and maintenance of the data platform. By going above and … beyond implementing new features, the company focus on customer experience, building a high-quality, secure, and scalable data platform. You'll apply your skills effectively and continue to develop them, while benefiting from the expertise of your more experienced colleagues, including: Pair and mob-programming. Peer code reviews. more »

do things differently. We’re focused on our core values; using these we’ve seen significant growth across our practices and our Digital, Data and Cloud team is preparing for the next phase of expansion. This creates new opportunities for driven and skilled individuals to join one of … the fastest-growing consultancies globally. Job Description This is an exciting opportunity for an experienced developer of large-scale data solutions. You will join a team delivering a transformative cloud hosted data platform for a key Version 1 customer. The ideal candidate will have a proven … of large data sets Data quality system and process design and implementation. Integrating the end to end data pipeline to take data from source systems to target data repositories ensuring the quality and consistency of data is more »

A fantastic opportunity has a risen with our Financial Services client, for a Head of Data Engineering to join the team. The Head of Data Engineering will be responsible leading a team of data engineers to architect, build, and maintain a new data lakehouse solution and pipelines that drive the digital transformation initiatives and support business reporting requirements. The Head of Data Engineering will be involved from the very beginning of the project and will have experience of Azure Datalake and Databricks. Key responsibilities for the Head of Data Engineering Define and drive the data engineering strategy and technical roadmap in alignment with the company's digital transformation goals. Lead and mentor a team of data engineers to design, develop, and deploy data pipelines and solutions that meet business needs. Lead more »

Our client, a well-known and market-leading Financial Services company, is currently recruiting for a Senior Data Engineer to join their Data engineering team. The Senior Data Engineer will be responsible for designing, developing, and maintaining data pipelines and workflows to … support the organisation's data processing needs and facilitate BI reporting and analytics delivery. Responsibilities for the Senior Data Engineer Design, develop, and maintain scalable data pipelines and workflows using Azure … Data Factory and Databricks Implement data quality checks and validation processes to ensure accuracy, completeness, and consistency of data. Monitor pipeline performance and troubleshoot issues promptly to ensure uninterrupted data processing. Key Skills & Experience for the Senior Data Engineer Essential Hands more »

Data Engineer Hybrid, West- Midlands £56k + bonus & great benefits Our client are a fast growing SaaS scale-up, revolutionising supply chain ecosystems and increasing revenue for their customers. You'll contribute to the organisation's high-quality data engineering practices, participating in technical design, implementation … testing, deployment and ongoing support and maintenance of the data platform. By going above and beyond implementing new features, the company focus on customer experience, building a high-quality data platform. You'll apply your skills effectively and continue to develop them, while benefiting from the … expertise of your more experienced colleagues, including: Skill set & Experience: • Data Modelling and Querying - Proficiency in designing and implementing data models, including star schemas and snowflake schemas. Expertise in SQL. • Data Storage Technologies - Experience in SQL, NoSQL, Blob, Delta Lake , and/or other more »

Senior Data Engineer- Fast growing SaaS Scale up 🚀 West Midlands- hybrid working Up to £86K base with bonus and benefits The client SaaS Software business platform which is revolutionising financial management for their customers. Responsibility: Working to ensure a high-quality data platform. Design, develop, test … and deploy data pipelines and integrations using IPaaS technology Collaborate on engineering platform ingestion, orchestration, data warehouse/data lake and API strategies for the data management ecosystem Willingness and an enthusiastic attitude to work within existing processes/methodologies. Collaborate with … the DevOps team to provide high availability data solutions that scale geographically. Big data processing - Expertise in processing enterprise-scale volumes of data, ideally with proficiency in Snowflake. Experience Data Stores - Technical excellence in SQL, NoSQL, Blob,Delta Lake, and other enterprise more »

Job title – Analyst Developer (Data engineering, Databricks, python) Location – Birmingham We will consider new graduates with some knowledge or classwork in SQL and python who are passionate about data engineering and learning Azure Databricks, but may not have a full complement of relevant work experience. About … FederatedHermes.com. Main Function of Role: Working as part of a team responsible for the development and maintenance of the Federated Hermes Global Enterprise Data Platform, the candidate will be expected to contribute to all aspects of the development lifecycle including support. The primary focus for this candidate will … be on data engineering, using Azure Data Lake, Azure Databricks with python and other related Azure Data services to support the strategic and tactical goals of Federated Hermes’ data and analytics projects. The successful candidate will have the initiative and incentive to more »

people just like you! About the Role We are DataOps advocates and use software engineering best practices to build scalable and re-usable data solutions to help clients use their data to gain insights, drive decisions and deliver business value. Clients don’t engage BJSS to … which means we get to work with a wide range of tools and technologies and there are always new things to learn. BJSS data engineers are specialist software engineers that build, optimise and maintain data applications, systems and services. This role combines the discipline of software … engineering with the knowledge and experience of building data solutions in order to deliver business value. As a BJSS data engineer you’ll help our clients deploy data pipelines and processes in a production-safe manner, using the latest technologies and with a DataOps more »

UI Developer (Data) Location: Covntry (3 Days a Week) Salary: up to £55,000 + Benefits Investigo are working with a global technology business based in the Midlands in search of a UI Developer to come and join their new team. Our client are leaders in their industry … in the Automobile sector. We are looking to assist them in growing out a new data focused team to ultimate service the pipeline of clients in need of their support and expertise. This responsibilities will include: Designing seem-less UI utilising of tools in a proprietary system … HTML, CSS, Typescript) Participate in projects from inception, creating standardised tools for data analysis Contextualise information to showcase trends and patterns from data sources Transform technical data into understandable insights for stakeholders Develop interactive dashboards combining visuals with real-time data Contribute more »

troubleshooting of the SOC's technology stack (hardware and software). The engineer will also assist with the continued development and maintenance of data pipelines and signature updates and the professional development of the system engineering team. Tasks: Perform system administration on specific cyber defence applications and systems … network tools in response to new or observed threats within the network environment or enclave. Manage the compilation, cataloguing, distribution, and retrieval of data from a range of enterprise networks and data sources. Implement data management standards, requirements, and specifications. Develop data standards, policies, and procedures. Analyse data sources to provide actionable recommendations and facilitate data-gathering methods. To share knowledge, skills and experience, create and improve documentation, and train new members of the data engineering team. Knowledge: Knowledge of big data more »

troubleshooting of the SOC's technology stack (hardware and software). The engineer will also assist with the continued development and maintenance of data pipelines and signature updates and the professional development of the system engineering team. Tasks: * Perform system administration on specific cyber defence applications and systems … network tools in response to new or observed threats within the network environment or enclave. * Manage the compilation, cataloguing, distribution, and retrieval of data from a range of enterprise networks and data sources. * Implement data management standards, requirements, and specifications. * Develop data standards, policies, and procedures. * Analyse data sources to provide actionable recommendations and facilitate data-gathering methods. * To share knowledge, skills and experience, create and improve documentation, and train new members of the data engineering team. Knowledge: * Knowledge of big data more »

Senior Data Optimisation Engineer Location: Coventry (3 Days a Week) Salary: up to £80,000 + Benefits Investigo are working with a leading global technology business based in the Midlands in search of a Senior Data Optimisation Engineer to come and join their new team. They … in the Automotive sector. We are looking to assist them in growing out a new data focused team to ultimate service the pipeline of clients in need of their support and expertise. This responsibilities will include: Manage and execute data-focused projects using internal analytic … Provide expertise in AWS monitoring and optimisation, optimising databases and ETL pipelines. Utilise programming languages such as Python and PySpark to transform big data into manageable datasets. Contribute to the development of interactive dashboards and provide expert analysis across program lifecycles. Transform technical data into actionable more »

Xpertise is seeking a series of Data Engineers with cloud experience to join a fledgling team in Birmingham. As part of our client's growing engineering division, you will play a pivotal role in leading the data engineering capabilities, working closely with Platform Engineers, Developers, and … and even venture into the world of MLOps/AIOps. Key experience desired/what you will learn: Design, develop, and maintain scalable data pipelines and ETL processes leveraging GCP services such as BigQuery, Dataflow, and Cloud Storage. Architect data models and schemas in BigQuery to … practices to ensure compliance and data integrity. Monitor and troubleshoot data pipelines, ensuring high availability and reliability. CI/CD pipeline automation utilising GitLab DevOps: GKE (Kubernetes), Terraform, Ansible Leadership capabilities: mentorship, management Role overview: If you're eager to collaborate with a driven team more »

About The RoleHippo Digital is recruiting for a Principal Data Engineer to join our Hippo Herd. Principal Data Engineers work in multi-disciplinary teams that build, support & maintain User-Centred digital solutions that offer real value and work for everyone. Hippo’s Principal Data Engineers are experts in their field. You will act as a principal consultant providing technical expertise in data engineering practices both internally and to our clients. You will drive the team's technical deliverables, maintain client relationships and be passionate about developing and upskilling others.Your role … identifying and ensuring that challenges are overcomeLead in the recruitment of other engineers, and support other consultants in their professional developmentEnsure Hippo’s Data and Engineering communities contribute to Hippo’s business at a strategic levelLead on new business opportunities, writing technical bids and proposalsPromote Hippo’s Data more »

our medium blog which received more than 300 claps here) API's & Gateway - Focuses on how evolve to Microservice architecture using .NET Core Data Engineering - Focuses on how to build the best backend data pipelines to manage/move our data using Python, SQL … our event driven architecture over RabbitMQ. Data Science - Focuses on how we can improve our AI assistant (Aimie) and maximise the huge amount of data that we sit on SRE - Focuses on how we best deploy/manage all the infrastructure every single day using tools more »

AI solutions that enhance, organise, and enrich their data, cost-effectively, accurately, and scalably Plan and manage their ML/AI delivery pipeline Mentor and coach team members on ML engineering best practices Advise senior management on their ML/AI strategy Key skills: Python for Machine … Learning and Data Science Frameworks used in Machine Learning and Data Science such as TensorFlow, PyTorch, and scikit-learn Data Science and Visualisation libraries including Pandas, NumPy, scikit-learn, matplotlib, Seaborn Cloud services used in machine learning and data science, such as … Azure, OpenAI, Hugging Face, AWS ML/AI Machine Learning Applications development life cycle processes and tools - MLOps, including building ETL data pipelines. Cutting-edge approaches to Machine Learning, including transformer models, diffusion models, and adversarial machine learning Experience in fine-tuning off-the-shelf Large Language Models more »

troubleshooting of the SOC's technology stack (hardware and software). The engineer will also assist with the continued development and maintenance of data pipelines and signature updates and the professional development of the system engineering team. Skills/Experience: Previous experience of Enterprise ICS/network architectures … and technologies. * Working with frameworks and technologies that support data-intensive distributed applications. * Experience maintaining and administrating data analytical and SIEM platforms. * Experience using host and network-based IDS/IPS * Experience using packet capture solutions. * Skill in developing and deploying signatures. * Skill to apply cybersecurity … network tools in response to new or observed threats within the network environment or enclave. * Manage the compilation, cataloguing, distribution, and retrieval of data from a range of enterprise networks and data sources. * Implement data management standards, requirements, and specifications. * Develop data more »

part of our client's growing engineering division, you will play a pivotal role in designing, implementing, and optimising machine learning models and data pipelines. With a strong emphasis on AWS technologies and MLOps practices, you'll have the opportunity to contribute to the development of scalable, production … what you will learn: Experience developing, deploying, and maintaining machine learning models in production environments. AWS cloud services, especially in building and managing data pipelines and machine learning workflows: S3, Redshift, Lambda, Glue, EMR, EKS (Kubernetes) Familiarity with MLOps/DevOps concepts and practices, including version control, CI …/CD, and model monitoring. Proficiency in Python and relevant data manipulation and analysis libraries (e.g., pandas, NumPy). Experience with distributed computing frameworks like Apache Spark is a plus. Apache Spark and Airflow would be a bonus. Role overview: If you're looking to work with a more »

contribute to exciting projects. As a Senior CGI Artist within the Automotive Visualisation Team, you will play a crucial role in transforming raw data into high-quality assets for use in visualisation projects. Working closely with a team of artists, designers, and engineers, you will be responsible for … and Tasks: Collaborate with cross-functional teams to understand project requirements and timeline including design, CAS, and CMF teams. Prepare and clean raw data, including 3D models, textures, and CAS files, for use in visualisation projects. Optimise data for performance and visual fidelity, ensuring assets meet … quality standards and specifications. Implement efficient data organisation and management strategies to streamline workflows and maximise productivity. Conduct quality assurance checks on prepared assets to identify and resolve any issues or discrepancies. Stay updated on industry trends and best practices in data preparation and visualisation techniques. more »

troubleshooting of the SOC’s technology stack (hardware and software). The engineer will also assist with the continued development and maintenance of data pipelines and signature updates and the professional development of the system engineering team. Requirements • DV Clearance • Coordinate with SOC and CTI Analysts to assist … administration concepts for operating systems such as but not limited to Unix/Linux, IOS, Android, and Windows operating systems. • Knowledge of big data technologies and ecosystems (e.g., NiFi). • Knowledge of current market and emerging leaders in data analytical and SIEM platforms. • Working with frameworks … and technologies that support data-intensive distributed applications. • Experience using packet capture solutions. • Skill in developing and deploying signatures. • Skill to apply cybersecurity and privacy principles to organizational requirements (relevant to confidentiality, integrity, availability, authentication, non-repudiation). • Ability to provide technical and service leadership to junior SOC more »