1 to 25 of 67 Permanent Data Pipeline Jobs in Shrewsbury

Key Responsibilities: Data Pipeline Development: o Develop and maintain data pipelines for ingesting, transforming, and loading data from various sources (e.g., APIs, databases, files) into the Azure Databricks platform. o Implement data extraction, transformation, and loading (ETL) processes. o Ensure data pipelines are efficient, reliable, and scalable. Data Transformation & Processing: o Implement data transformations using Spark (PySpark or Scala) and other relevant tools. o Perform data cleaning, validation, and enrichment. o Ensure data quality and consistency. Azure Databricks Implementation: o Work … in team meetings and code reviews. Automation & DevOps: o Build CI/CD pipelines, for environmental deployments. o Contribute to automating data pipeline deployments and other data engineering tasks. o Learn and apply DevOps principles. More ❯

Job Title: Senior Data Engineer Key Requirements: 4-8 years of experience from tier 1 or 2 big tech companies Job Location: Abu Dhabi, UAE Benefits: Work with cutting-edge technology through modern infrastructure and automation projects Thrive in a growth-focused environment that prioritizes learning, innovation, and … role requires a deep understanding of modern data engineering practices, real-time processing, and cloud-native solutions. Key Responsibilities: Data Pipeline Development & Management: Design, implement, and maintain scalable and reliable data pipelines to ingest, transform, and load structured, unstructured, and real-time data … environments. Machine Learning & Advanced Analytics Enablement: Collaborate with data scientists to prepare and serve features for ML models. Maintain awareness of ML pipeline integration and ensure data readiness for experimentation and deployment. Documentation & Continuous Improvement: Maintain thorough documentation including technical specifications , data flow More ❯

Data Management Specialist (Azure) - Newcastle/Hybrid You will provide expert-level troubleshooting, root cause analysis, and performance optimisation for data platforms,, such as Azure SQL databases, Fabric Warehouse and Onelake, Databricks, and Azure Data Factory. Purview or another MDM tool is highly desirable. … Skills/experience Bachelor’s degree in Information Technology, Computer Science, Data Management, or a related field. 4+ years of experience in data management, data operations, or IT support roles, with at least 2 years in Level 3/4 support or equivalent. Hands … on experience with database technologies ( Azure landscape , Oracle, PostgreSQL, etc.). Desirable: Industry certifications in professional data management, cloud platforms (AWS, Azure, GCP), metadata management, or database administration (e.g., Microsoft Certified: Azure Data Engineer Associate, DAMA CDMP) are desirable. Proven experience with ETL tools, data More ❯

Key aspects of the role include: Data Pipeline Development: Architect, develop, and optimise data extraction, transformation, and loading (ETL) processes for high-volume healthcare data. Integration of NHS Systems Data: Work directly with NHS data sources and ensure seamless integration of … systems like UDAL, NCDR, SUS+, HES, and ODS into our analytics platform. System Performance: Monitor and enhance the performance of our data systems to ensure reliability and responsiveness. Collaboration: Partner with data scientists, analysts, and stakeholders to ensure that data solutions align with business … needs and analytic strategies. Scalability & Security: Implement and enforce best practices for data security, quality, and scalability across our data infrastructure. Essential Skills: Healthcare Data Expertise: Proven experience in designing and managing data solutions within healthcare environments. Demonstrated ability to work with More ❯

the ease of use - they have introduced several industry-first products and maintain a strong focus on research and development. As a Junior Data Engineer, you will play a crucial role in developing the data infrastructure that supports the analytics and operational functions of our business … enterprise. Role & Responsibilities: Data Infrastructure & Support: Provide first-line support for data engineering tasks, such as managing and monitoring data pipelines, resolving issues, and ensuring data integrity. Work with both structured and unstructured datasets to design and implement data models … -related errors. Collaborate with business stakeholders to identify data requirements and deliver sustainable solutions. Monitor and troubleshoot data pipeline issues to maintain data integrity and accuracy. Assist in the development, maintenance, and optimization of ETL (Extract, Transform, Load) processes for efficiency More ❯

join us at FPSM and be part of something bigger... What we are looking for: We are seeking a highly motivated and experienced Data Product Manager to own the strategy, roadmap, and execution of our Customer Data Platform (CDP) and Single Customer View (SCV). This … is a critical role within our rapidly growing data organization, and you will be instrumental in enabling data-driven decision-making across the business. You'll work closely with engineering, marketing, product, and other internal teams to ensure our CDP is a powerful tool for understanding … with overall business objectives and stakeholder needs. Requirements Gathering: Proactively gather, analyze, and prioritize requirements from internal stakeholders (marketing, product, analytics, etc.) for data integration, access, and utilization. Data Source Integration: Oversee the integration of various data sources into the CDP, ensuring data More ❯

Who are Inference Group? Are you ready to be a part of a company that’s redefining the future of data and AI? Welcome to Inference Group, where we’re not just a tech startup – we’re a team of passionate innovators dedicated to empowering businesses through the … transformative power of data, machine learning, and AI. At Inference Group, we’re on a mission to help businesses unlock their true potential and drive sustainable growth. As we rapidly expand, we’re looking for like-minded, creative technical experts who want to make a real impact. Here … of MLOps practices and tools – e.g., version control for data/models, CI/CD for machine learning, containerization (Docker, Kubernetes), and pipeline orchestration. Understanding of data engineering principles (ETL/ELT processes, data warehousing) to oversee data pipeline development. More ❯

Azure Data Engineer/Consultant – Up to £75K + Bonus Location: Remote (UK-based, with very occasional travel) Salary: Up to £75,000 + bonus Company & role Join one of the UK’s most advanced Microsoft partners, specialising in Azure-led Data & AI solutions. This consultancy … is known for transforming data into tangible value through innovative platforms and strategic data services. With a collaborative team of experts, they deliver impactful solutions across enterprise clients, leveraging the very latest in Microsoft technology. Why This Role Stands Out Play a key role in shaping … across the Azure Data Platform (Fabric, Synapse, Databricks, Data Lake, Data Factory, Purview, etc.) Strong data pipeline and ETL skills, including data modelling and transformation Experience implementing Medallion Architecture (Bronze, Silver, Gold layers) in lakehouse or Databricks environments Familiarity More ❯

Azure Data Engineer Remote (UK-based) Up to £60,000 per annum About the Company We are a leader in the insurance sector, committed to data-driven decision making and modernising our technology landscape. As part of our continued growth, we’re seeking a skilled Azure … Data Engineer to support our data platform and engineering initiatives. Role Overview As an Azure Data Engineer, you will be instrumental in designing, building, and optimising data pipelines and architecture in our Azure environment. This role is ideal for someone with strong … SQL skills and a background in Azure data services, looking to play a key role in transforming and scaling our data operations. Key Responsibilities Develop and maintain scalable data pipelines in Azure. Build and optimise data integration workflows using Azure Data More ❯

A high performing, multi-billion dollar AUM hedge fund are looking to make a newly created Data Analyst hire into their rapidly growing centralised Data Management team. Reporting into the Global Head of Data and taking on a highly visible role within the team … the hire will be working very closely with Operations, Product Control, Risk, Trading and Research teams to ensure the smooth transmission of data, in … a role that has a highly technical/engineer angle. You will optimize the operation and quality management of the internal data pipeline with oversight of raw data initially being onboarded through to the delivery of datapoints to front office/investment teams. You will More ❯

advertised date, so please apply as soon as you can. What you’ll be doing Contribute to the design and execution of the data migration and integration strategy within the Mortgages area, supporting the broader integration programme objectives. Collaborate with internal stakeholders, business analysts, and architects to translate … functional and non-functional requirements into technical data solutions. Design, build and maintain robust, scalable and high-performance ETL/ELT pipelines using modern data engineering tools and practices, ensuring quality, lineage and auditability. Support data discovery activities and analysis to inform data mapping, profiling, and migration planning, especially in the context of mergers or acquisitions. Develop and maintain detailed technical design documents and data flow diagrams aligned with architectural principles and data governance standards. Assist in the implementation and enforcement of best practices in data More ❯

collaboration, recognition and inclusivity and offer a wide range of benefits to support staff wellbeing. Your Future Starts Here PURPOSE OF JOB: The Data & AI Solutions Deployment Architect will be instrumental in architecting and deploying Azure Data & AI Solutions. A solid, technical, ‘hands-on’ (not just … grasp of Azure Fabric is essential as we navigate clients through all of the infrastructure, coding, and product development lifecycle challenges around successful Data and AI implementation and deployment. This customer-centric position involves delivering projects, conducting workshops and physically working on client’s environments via Python, and … C#. This role would suit an experienced Data platform engineer or Data Solutions Architect who thrives in working across multiple projects with a diverse range of clients, and who places the importance of bridging the gap between complex data problems and real life product More ❯

Job Title: Senior Data Engineer Salary: up to £65,000 Location: Hybrid – 1 day per fortnight in the Wilmslow offices Working hours 0900 to 1730 The Role Following recent investment, The Citation Group are looking to build out their data capabilities to be Best in Class. … As such, a fantastic opportunity has arisen for a strong data engineer to work in the data engineering team, to expand and maintain the data infrastructure that provides the foundations from which our BI, Data Science and Operational Data teams … insights and revenue to the functions across the wider group. We’re looking for an individual who has experience working with enterprise grade data pipelines and infrastructure, who can implement processes on our modern tech stack with robust, pragmatic solutions. Responsibilities develop, and maintain ETL/ELT data More ❯

Senior Data Engineer Bristol 6-month contract Paying up to £80p/h (Outside IR35) *** Please note that due to the nature of the work you must hold or be eligible to obtain DV level security clearance *** Responsibilities : Design, develop, and maintain secure and scalable data pipelines using the Elastic Stack (Elasticsearch, Logstash, Kibana) and Apache NiFi. Implement data ingestion, transformation, and integration processes, ensuring data quality and security. Collaborate with data architects and security teams to ensure compliance with security policies and data governance standards. … Manage and monitor large-scale data flows in real-time, ensuring system performance, reliability, and data integrity. Develop robust data models to support analytics and reporting within secure environments. Perform troubleshooting, debugging, and performance tuning of data pipelines and the Elastic Stack. More ❯

Data Engineer £50,000-£68,000 per annum Hybrid role with 3 days per week in Burton-on-Trent office Seeking a skilled Data Engineer to deliver the data engineering roadmap and help drive data maturity across the organisation. You'll be … responsible for the end-to-end data engineering lifecycle from ingestion and transformation to delivery while ensuring data solutions are scalable, secure and aligned with business goals. Key Responsibilities: · Design, build and maintain robust data pipelines and transformation logic · Own data engineering … batch, streaming, and APIs · Collaborate with cross-functional teams to understand and deliver on business needs · Ensure regulatory compliance, data quality, and pipeline documentation · Optimise the central data platform in partnership with architecture and analytics teams · Champion best practices in data engineering, MLOps More ❯

Data Engineer We’re working with a forward-thinking Microsoft partnered consultancy who are looking for a Data Engineer to join their growing team. You will have the opportunity to work with some of the latest Microsoft technologies with a focus on projects on Databricks implementations. … You will join a team at the centre of a number of data-driven projects where you will be responsible for the design, development and creation of data solutions. You will work on the full end-to-end product lifecycle from platform design to insights creation. … This is a fantastic opportunity for someone who is passionate about building scalable, high-performance data solutions utilising the latest cutting-edge cloud technologies. As part of this role, you will be responsible for some of the following areas • Design and develop scalable data pipelines to More ❯

Head of Data Engineering & Governance Investment & Wealth Management Hybrid (3 days at home with 2 days traveling in the UK) £120k - £140k basic (DOE) + bonus (10-25%) and benefits (pension, life insurance, healthcare, holidays, & more). “Money makes the world go round,” they say. Maybe it did … But now? It’s data. This company – a trusted name in the UK financial services sector – fully recognise this. They understand that embracing data and AI is what will continue to position them at the forefront of the UK’s investment management sector. This isn’t a dalliance … for investment or dragging people with you. Now, following a recent acquisition, they’re ready to embark on the next phase – transforming the data and harnessing its power. And that’s where you come in. Where you fit in As one of three new leaders in the newly More ❯

a wide range of benefits to support staff wellbeing. Your Future Starts Here PURPOSE OF JOB: We’re looking for an experienced Azure Data & AI Engineer with a strong focus on advanced analytics, machine learning, and applied AI - particularly Generative AI. This role will suit a technically capable … Insights or Log Analytics. KEY RESPONSIBILITIES: Have 5+ years of technical consulting or a similar chronology of demonstrable success in coding and deploying Data and AI models onto Azure environments. At least one year using Azure Fabric Create scalable and reliable data pipelines for data processing, transformation, and storage. Establish and enforce data governance policies and be ‘hands-on’ in exploratory data analysis to identify and measure quality gaps that are a block to Data and Ai implementation. Implement robust security measures to protect sensitive data More ❯

Job Title: Data Analyst Location: LS12, On-site Reports To: Marketing Director/Operations Manager Job Type: Full-Time Salary: £40- £45k DOE About Yorkshire Dental Suite Yorkshire Dental Suite is a leading private dental practice, known for its innovative approach to patient care, advanced cosmetic treatments, and … a commitment to excellence. As we continue to grow, we’re looking for a data-driven professional to help us make smarter business decisions through insightful analysis and strategic guidance. Role Overview We're looking for a proactive Data Scientist/Analyst to join our growing … performance across all areas of the business, with a strong focus on marketing campaigns, lead generation, and conversion rates. Using HubSpot and other data sources, you’ll be responsible for turning data into actionable insights that drive growth and optimise performance. Key Responsibilities Analyse data More ❯

Senior Data Engineer – 6-Month Contract | London | Insurance Sector We're exclusively recruiting for a Senior Data Engineer to join a leading organisation within the Property & Casualty Insurance sector. 📍 Location : London (Hybrid) ⏳ Contract : 6 months (potential extension) 📁 Sector : Insurance – Property & Casualty 💼 Work Type : Contract (Inside IR35 …/Outside – subject to discussion) You’ll be embedded within the Actuarial team , building scalable data platforms and pipelines to support pricing analysis. You’ll work closely with stakeholders, lead third-party collaborations, and play a critical role in optimising and managing robust data infrastructure. 🔧 Key … as a Data Engineer (ideally in the insurance or financial sector). Expertise in SQL Server , ETL frameworks , and data pipeline development . Solid knowledge of Python , Azure , AWS , and tools like Databricks , Glue , S3 , etc. Ability to identify and resolve performance bottlenecks and data More ❯

leading-edge finance solutions through a unique combination of teamwork and technology. From this continued success we are now looking to recruit a Data Analyst to allow us to continue our exciting growth plans. Our aim is to take the service we offer to the next level, to … d like to be part of? As a key member of our team, you’ll be responsible for translating business requirements into practical data models, reports, and dashboards that deliver real insights Role responsibilities; Stakeholder Engagement: Collaborate with key stakeholders across the business to understand their data needs, gather requirements, and provide clear insights that drive decision-making. Data Modelling & Analysis : Design, develop, and maintain scalable data models to ensure data quality and accuracy, utilising SQL and Azure Synapse where necessary. Dashboard Development: Build and maintain visually compelling and More ❯

Recruitment Company is working with an Environmental and Sustainability Business that supports and empower their customers journey to net zero. To join their Data-Technology team, we have an opportunity for a Data Engineer to help support the wider business (cross-functional) to meet their reporting … and data requirements. About the role: The Data Engineer will be working with the latest innovative technologies, to design, build and maintain data solutions, constructing process to surface data both internally for reporting purposes and externally through the customer portal. The Data Engineer will be responsible for developing scalable data pipelines to integrate diverse data sources whilst ensuring data quality under the framework of a new Data Platform for real time application integration and reporting, and will work closely with business stakeholders More ❯

About Apexon: Apexon brings together distinct core competencies – in AI, analytics, app development, cloud, commerce, CX, data, DevOps, IoT, mobile, quality engineering and UX, and our deep expertise in BFSI, healthcare, and life sciences – to help businesses capitalize on the unlimited opportunities digital offers. Our reputation is built … Capital, Apexon now has a global presence. About the Role We are seeking a highly motivated and experienced AI Engineer to join our Data and AI team . This role is ideal for someone who has a strong track record of designing, developing, and deploying AI/ML … client. Key Responsibilities End-to-End AI Solution Delivery: Lead AI/ML initiatives from conceptual design to production deployment, including problem scoping, data acquisition, model development, validation, deployment, and monitoring. AI-Driven Marketing Insights: Develop predictive and generative models to support audience segmentation, personalization, channel optimization, lead More ❯

Lead Backend Engineers (Platform Experience & Data Warehousing teams) 👩🏽‍💻 Location: UK, Ireland, Poland, Lithuania, Spain, Portugal, Finland 🌍 numi are proud to have partnered with a front runner in data integration technology providing fantastic experience for their clients (who are some of the biggest names in the space … We are looking for experienced Lead Backend Engineers (PHP) to join either Platform Experience or Data Warehousing teams. The Platform Experience team focuses on optimising end-to-end platform journeys and building impactful customer experiences. They are currently developing a data visualisation tool (dashboards) and build … features primarily from scratch. The Data Warehousing team is responsible for data storage and transfer solutions, helping users streamline data transfers to data warehouses/datalakes and improving internal storage. This involves scaling, refactoring, and enhancing a complex data transfer More ❯

Job Title: Senior Data Engineer - 12 month FTC Location: London/Hybrid (2 days per week onsite) Salary: Competative market rate + bonus + benefits Why Apply? This is an exciting opportunity for an experienced Azure Data Engineer to play a key role in delivering the … next phase of a business-critical data programme. You'll apply your technical expertise across Azure Databricks, Data Factory, Azure SQL, and SQL Server, contributing directly to the integration of financial data into a new Azure Data Platform. Responsibilities: Design, build and … optimise scalable data solutions using Azure cloud technologies. Lead the development of data pipelines in Azure Databricks, Data Factory, and T-SQL. Provide senior-level technical direction to onshore and offshore development teams. Translate data requirements into effective data engineering More ❯