1 to 25 of 28 Data Pipeline Jobs in Cardiff

technologies to improve safety and the ease of use - they have introduced several industry-first products and maintain a strong focus on research and development. As a Junior Data Engineer, you will play a crucial role in developing the data infrastructure that supports the analytics and operational functions of our business enterprise. Role & Responsibilities: Data Infrastructure & Support: Provide first-line support for data engineering tasks, such as managing and monitoring data pipelines, resolving issues, and ensuring data integrity. Work with both structured and unstructured datasets to design and implement data models, perform data cleansing, transformation, and validation. Maintain accurate documentation of data … resources and troubleshooting data-related errors. Collaborate with business stakeholders to identify data requirements and deliver sustainable solutions. Monitor and troubleshoot data pipeline issues to maintain data integrity and accuracy. Assist in the development, maintenance, and optimization of ETL (Extract, Transform, Load) processes for efficiency and reliability. Project & Improvement: Assist More ❯

EU work permit required: Yes col-narrow-right Job Views: 3 Posted: 10.06.2025 Expiry Date: 25.07.2025 col-wide Job Description: Key Responsibilities Lead the technical delivery of complex data engineering projects, ensuring solutions are scalable, secure, and aligned with our delivery framework, and client goals. Design and build high-quality data pipelines and integration workflows, setting … throughout the development lifecycle. Collaborate with multidisciplinary teams, including a wide range of other roles, to shape solutions that meet both technical and business requirements. Mentor and support data engineering teams, fostering a culture of continuous improvement, knowledge sharing, and technical excellence. Support testing activities by ensuring pipelines are testable, observable, and reliable; work with QA and analysts … to define test strategies, implement automated tests, and validate data quality and integrity. Contribute to technical planning, including estimation, risk assessment, and defining delivery approaches for client engagements and new opportunities. Engage with clients and stakeholders, translating data requirements into technical solutions and communicating complex ideas clearly and effectively. Champion engineering standards, contributing to the development More ❯

Lead Data Scientist (Equity Only) - Cardiff Job Details Location: Cardiff, United Kingdom Job Category: Other EU Work Permit Required: Yes Job Views: 4 Posted: 16.06.2025 Expiry Date: 31.07.2025 About Luupli Luupli is a social media app 'in development' aimed at providing opportunities for diverse creators from marginalized communities, highlighting their work and offering access to social media opportunities. … Job Description As a Lead Data Scientist at Luupli, you will leverage AWS analytics services to analyze data, collaborate with cross-functional teams, and develop data-driven solutions to support business growth. Responsibilities Understand business objectives and define analytics goals. Develop data analysis strategies using AWS services like Amazon Redshift, Athena, EMR … and QuickSight. Design and build data pipelines and ETL processes. Apply statistical and machine learning techniques for predictive and pattern analyses. Develop KPIs, dashboards, and visualizations to enable decision-making. Perform exploratory data analysis to uncover insights. Work with data engineers to optimize data architecture and governance. Requirements Bachelor's or master More ❯

About Our Client Leading Transportation organisation Job Description Leading Transportation organisation are seeking to hire a Head of Data & Data Engineering in this newly created role. You will lead a team of developers and analysts to provide a robust data platform on their journey to self-serve information, providing insight and analysis with the … sole purpose of providing excellent service to customers. Outside of Data Engineering you will fulfil a broad remit across Data Architecture, Data Security and Data Quality Management. Dimensions of the role: Deliver a wide range of projects with internal and external suppliers and have autonomy over an annual budget of £1M - £2M. … of data platform technologies, including Azure data warehouses, data lakes, and data governance tools. Knowledge of optimising data pipelines, pipeline architectures and integrated datasets. Demonstrable knowledge of working with and understanding data architecture principles and best practice. Experience of procuring and implementing cloud-based data More ❯

Social network you want to login/join with: We have partnered with a company that empowers underwriters to serve their insureds more effectively. They are using advanced data intelligence tools to rebuild the way that underwriters share and exchange risk. With a current focus on the small and medium-sized businesses that power our global economy and … policy to deliver unprecedented insight into insurance pools, and their speciality portfolio is fully diversified with very low catastrophe, aggregation or systemic risk. The Role: Designing and implementing data pipelines and models, ensuring data quality and integrity. Solving challenging data integration problems, utilising optimal patterns, frameworks, query techniques, sourcing from vast and varying data sources. Building, maintaining, and optimising our Data Warehouse to support reporting and analytics needs. Collaborating with product managers, business stakeholders and engineers to understand the data needs, representing key data insights in a meaningful way. Staying up-to-date with industry trends and best practices in data modelling, database development More ❯

Data Engineer – Data Engineering Team We’re looking for a Data Engineer to join our growing Data Engineering team at Creditsafe. You'll play a key role in building data pipelines and supporting faster, more informed decision-making for both our internal teams and customers. About Creditsafe Over the past … a hybrid working model to give you the flexibility you need to do your best work. Your Team You’ll join one of four delivery teams within our Data Engineering department. Together, we support over 1,000 data pipelines and manage data services across our centralised data warehouse. These services provide the … business-ready" data that powers our products. We work in an Agile environment using modern tools and technologies including AWS, Glue, Step Functions, Athena, PySpark, SQL, and Python. Our processes are metadata-driven to ensure scalable, consistent, and reliable data delivery. Your Role You’ll work closely with the analytics team to design and build data More ❯

Social network you want to login/join with: Head of Data Engineering & Governance, cardiff col-narrow-left Client: Data Science Talent Location: cardiff, United Kingdom Job Category: Other - EU work permit required: Yes col-narrow-right Job Views: 3 Posted: 04.06.2025 Expiry Date: 19.07.2025 col-wide Job Description: Head of Data Engineering & Governance … client began to focus solely on wealth management. But now? This company – a trusted name in the UK financial services sector – fully recognise this. They understand that embracing data and AI is what will continue to position them at the forefront of the UK’s investment management sector. This isn’t a dalliance with data. It’s a … debate about the direction, fighting for investment or dragging people with you. Now, following a recent acquisition, they’re ready to embark on the next phase – transforming the data and harnessing its power. And that’s where you come in. Where you fit in As one of three new leaders in the newly established Group Data More ❯

pay them on their own terms. Role Summary You would be working closely with a wide array of people in our team and business – this will be from data engineers, governance and risk analysts through to our CFO to deliver actionable reporting. This would primarily service the finance team working with tools including DBT, Snowflake, PowerBI and no … an analytics champion - working with the wider analyst community to enable and upskill them to deliver insights to the business themselves. You will work with the head of data to disrupt and challenge existing processes across the operational side of the business, promote product analytics and deliver complex analysis that the wider analyst community is not able to … serve itself (such as predictive modelling and forecasting). As we are a small team there is an ask to support across the data and product landscape which includes: dabbling with AI, securing external data feeds and investigating innovative data focused product initiatives such as complex financial products for our customers. Skills: We need More ❯

Social network you want to login/join with: We are currently looking a Data Engineer, to join the Data Engineering team in Cardiff. You will be expected to attend the office 50% of the working week, to align with our hybrid working policy. WHO ARE WE? At Creditsafe, we weave our core values of customer … part of a culture and a company where careers are made and where talent meet its true potential. JOB PROFILE You will be working closely with the product, data analysts and data engineering, building specific systems facilitating the extraction and transformation of Creditsafe data. The role will define and build data pipelines that will … and high throughput. Execute practices such as continuous integration and test driven development to enable the rapid delivery of working code. Design and build metadata driven data pipeline using Python and SQL in accordance with guidelines set by the Data Architect Ship medium to large features independently using industry standard processing patterns The responsibilities detailed More ❯

Staff Software Engineer - Data Department: Engineering Employment Type: Full Time Location: Cardiff Description is the UK's first comparison platform for car insurance. We've been helping customers since 2002 by empowering them to make better decisions around insurance and financial services. Our mission is simple: take away the confusion when comparing financial products and services to help … the organisation alongside your peers. Autonomy: Authority over technical strategy, decisions and implementation approach, so you can deliver using practices that align with your preferred ways of working. Data Driven: Utilise rich logs, metrics, and data to monitor and improve system performance & reliability. Culture: Enhancing a diverse engineering culture by taking part in various technical catch … and org level contributions. Rotate around the business to build relationships and act as a multiplier. What we look for in you Understand, assess and effectively apply modern data architectures (dimensional model, data mesh, data lake). Experience in applying and using data observability methods effectively. Experience in modern software development practices More ❯

within a small, collaborative team, this role will support cross-functional initiatives across Finance, HR, and Operations, directly impacting business processes and decision-making through technology, automation, and data integration. This position offers hands-on experience in ERP administration, BI reporting, ETL workflows, and custom application development. Duties & Responsibilities: Serve as the primary administrator for NetSuite ERP, overseeing … system configurations, workflows, permissions, and enhancements. Support and administer HRIS platforms, ensuring functionality and data integrity for HR and payroll processes. Build and maintain Power BI dashboards and reports to support business insights across departments. Develop and optimize DAX formulas and Power Query transformations for efficient data visualization and reporting. Write and refine SQL queries and … Power BI development 2-4 years of experience with HRIS administration 2-4 years of experience with NetSuite ERP administration 1-2 years of ETL/data pipeline experience 4-6 years of SQL and SSRS reporting Experience in a small IT or business systems team, managing diverse responsibilities Preferred Tools and Technologies: HRIS systems (UKG preferred More ❯

experienced and highly motivated Lead Machine Learning Engineer to drive the development, deployment, and optimization of machine learning solutions. As a technical leader, you will collaborate closely with data scientists, software engineers, and product managers to bring cutting-edge ML models into production at scale. You'll play a key role in shaping the AI strategy and mentoring … machine learning team. Responsibilities: Lead the end-to-end development of machine learning models, from prototyping to production deployment. Architect scalable ML pipelines and infrastructure. Work closely with data scientists to transition research models into robust production systems. Collaborate with engineering teams to integrate ML models into applications and services. Manage and mentor a team of machine learning … and data engineers. Establish best practices for model development, evaluation, monitoring, and retraining. Design experiments, analyze results, and iterate rapidly to improve model performance. Stay current with the latest research and developments in machine learning and AI. Define and enforce ML model governance, versioning, and documentation standards. Required Skills & Qualifications: Bachelor's or Master’s degree in Computer More ❯

started in Oslo and have grown to serve over 110,000 customers across 23 offices worldwide. Our tools support everything from risk assessment to customer engagement, powered by data, AI, and a commitment to innovation. We’re proud of our inclusive culture and global community, where people can be themselves and develop both professionally and personally. The Team … test automation. Understanding of the defect lifecycle and how to manage quality feedback loops. Hands-on experience with AWS (including serverless or cloud-native architecture). Experience with data pipelines and test automation within CI/CD pipelines. Familiarity with Azure DevOps and performance testing tools (e.g., JMeter). A good understanding of the software development lifecycle (SDLC More ❯

started in Oslo and have grown to serve over 110,000 customers across 23 offices worldwide. Our tools support everything from risk assessment to customer engagement, powered by data, AI, and a commitment to innovation. We're proud of our inclusive culture and global community, where people can be themselves and develop both professionally and personally. The Team … test automation. Understanding of the defect lifecycle and how to manage quality feedback loops. Hands-on experience with AWS (including serverless or cloud-native architecture). Experience with data pipelines and test automation within CI/CD pipelines. Familiarity with Azure DevOps and performance testing tools (e.g., JMeter). A good understanding of the software development lifecycle (SDLC More ❯

Social network you want to login/join with: Location: Remote (UK-based) with occasional client site visits This position sits within the Solution Consulting function of a data-focused consultancy, supporting the delivery of data projects across a range of strategic clients. The role involves working both remotely and on-site to lead analysis efforts … and engage with business stakeholders to define and document data-driven requirements. Key Responsibilities: - Lead business and data analysis activities across multiple clients. - Gather, analyse, and document requirements for data pipelines, system migrations, and reporting needs. - Work closely with both client stakeholders and internal engineering teams to ensure accurate and actionable specifications. - Facilitate workshops … and discovery sessions to identify business and data requirements. - Support the development and curation of business glossaries and manage associated processes. - Maintain up-to-date specifications and advocate for solutions that drive client value. Key Experience & Skills: - 5+ years in a business/data analyst role, ideally in a consultancy or commercial setting. - Strong analytical, problem More ❯

Data Engineer My client is in search of a Data Engineer who will concentrate on data warehousing and business intelligence for the organisation. The role involves utilising tools such as Azure Data Factory, Power BI, Fabric, SSRS Reporting, and data cubes. Key Responsibilities Oversee Azure Data Factory, Data Lake, and Data Warehouse SQL environments Work with business and technical teams to gather and document data requirements Design, implement, update, and support data pipelines in Azure Data Factory (ADF) Maintain data warehouse dimensional model and develop data marts Develop standards and processes to optimize cost … and service delivery Provide ongoing support, debug issues, and perform root cause analysis Enforce data management governance processes and standards Experience : Hands-on experience with Azure and Microsoft data stack, data pipelines, data cubes, SSRS to PowerBI migration Business and technical requirements analysis, business process modeling, data architecture More ❯

Data Engineer - Hybrid working My client is in search of a Data Engineer who will concentrate on data warehousing and business intelligence for the organisation. The role involves utilising tools such as Azure Data Factory, Power BI, Fabric, SSRS Reporting, and data cubes. Key Responsibilities Oversee Azure Data Factory, Data Lake, and Data Warehouse SQL environments Work with business and technical teams to gather and document data requirements Design, implement, update, and support data pipelines in Azure Data Factory (ADF) Maintain data warehouse dimensional model and develop data marts Develop standards and … processes to optimize cost and service delivery Provide ongoing support, debug issues, and perform root cause analysis Enforce data management governance processes and standards Experience: Hands-on experience with Azure and Microsoft data stack, data pipelines, data cubes, SSRS to PowerBI migration Business and technical requirements analysis, business process modeling, data More ❯

Want to drive a top brand's Data team with 1m+ users? If you love building software in Python, implementing robust data pipelines & driving best practices, you may be interested in a Senior Engineer role I have with a highly successful Tech driven business, where you'd drive the Data function playing a key … holiday (raising to 32), L&D budget, Health insurance, excellent maternity/paternity/adoption leave. What you'll like: You'd scale & improve a top brand's Data function, optimising retention of millions of users & having a real life impact You'd have autonomy over Tech to pick the latest, best languages/frameworks for the job … You'll have broad career progression opportunities across Group, which includes several high profile household name What you'll bring: Strong Python & TDD skills Expertise in modern data architecture (dimensional modelling, data mesh, data lake) & best practices (agile, CI/CD, IaC, observability). Experience with Cloud and big data technologies (e.g. More ❯

Social network you want to login/join with: Data & AI Solutions Deployment Architect, cardiff col-narrow-left Client: Bytes Software Services Location: cardiff, United Kingdom Job Category: Other - EU work permit required: Yes col-narrow-right Job Views: 8 Posted: 06.06.2025 Expiry Date: 21.07.2025 col-wide Job Description: Bytes is a top provider of premium IT solutions … foster a culture of innovation, collaboration, recognition and inclusivity and offer a wide range of benefits to support staff wellbeing. Your Future Starts Here PURPOSE OF JOB: The Data & AI Solutions Deployment Architect will be instrumental in architecting and deploying Azure Data & AI Solutions. A solid, technical, ‘hands-on’ (not just the theory) grasp of Azure … Fabric is essential as we navigate clients through all of the infrastructure, coding, and product development lifecycle challenges around successful Data and AI implementation and deployment. This customer-centric position involves delivering projects, conducting workshops and physically working on client’s environments via Python, and C#. This role would suit an experienced Data platform engineer or More ❯

We are seeking a talented and experienced Data Platform Product Lead to join our team. This is an exciting role focusing data warehousing and business intelligence for the firm and external clients. You will be joining our data and architecture team whose mission is to re-imagine how we manage and extract value from … our data. You will be a hands-on data engineer with experience in Azure Data Factory, PowerBI, Fabric, legacy SSRS reporting and data cubes. Your expertise and ability to implement best practices in a newly established team and fast changing data environment will be essential in driving the success of the team. … Key Responsibilities Overall management of Azure Data Factory, Data Lake, and Data Warehouse SQL environments. Work with business and technical teams to elicit and document requirements for data coming into the data warehouse from source systems to support changing and evolving business intelligence use cases. Plan, design, implement, update and More ❯

Social network you want to login/join with: A leading Financial Services client in the City of London is now seeking an experienced Data Engineering Manager to join on a permanent basis. This role is offering a base of £85,000 + a strong benefits package and flexible working. The ideal Data Engineering Manager will … come from a data engineering background and have strong knowledge in SQL, Snowflake, Microsoft Azure, Azure Data Factory and Azure DevOps. The Engineering Manager will design, improve and maintain robust data pipelines within data architecture. To be considered for this role you will need the following: Experience designing, improving and maintaining robust … data pipelines Strong SQL programming skills. Knowledge of other programming languages such as Python, C++ and Java beneficial Possesses a strong understanding of Snowflake - beneficial Experience managing small teams of Data Engineers Strong experience working in a cloud environment and knowledge in the following very beneficial: Microsoft Azure, Azure Data Factory and Azure DevOps More ❯

Location: Birmingham, Bristol, Cambridge, Cardiff, Edinburgh, Gatwick, Glasgow, Leeds, Liverpool, Manchester, Milton Keynes, Newcastle, Reading, Southampton Business Line: Enabling Functions Date published: 28-May-2025 Job Title: Data Analyst/Technical Business Analyst Connect to your Industry: Do you enjoy being the “voice of the business" ensuring that our technology teams deliver solutions that meet business needs? If … so, this opportunity is for you! Deloitte is looking for a mid-level Data Analyst/Technical Business Analyst to join its talented Business Change & Analysis team. Connect to your career at Deloitte: Deloitte drives progress. Using our expertise, we help clients become leaders. We invest in outstanding people, build diverse teams, and empower them to achieve more. … firm, using traditional and innovative tools to enable positive transformation. Industry experience is welcomed but not required. Your role involves ensuring solutions meet business and technical needs, transforming data into insights for strategic decisions, understanding business processes, eliciting requirements, and improving user experience and technical quality. You will also support the Head of Business Analysis in practice development More ❯

Social network you want to login/join with: ZONTAL is revolutionizing data management and digital transformation for the life sciences industry. Our cutting-edge platform enables seamless data integration, standardization, and accessibility—empowering scientists and organizations to drive innovation, enhance compliance, and accelerate breakthroughs in healthcare and biotechnology. ZONTAL has offices in the US, Europe … the job. Your Responsibilities: Be a subject matter expert in the ZONTAL Platform and its customer use-cases. Analyze business requirements for customer-facing projects, with emphasis on data flows and data formats as they feed into & out of the ZONTAL Platform. Create functional and design specifications for the proposed solution, focusing on overall Platform architecture … sciences data acquisition software, such as Scientific Data Management Systems (SDMS) or Laboratory Information Management Systems (LIMS). Hands-on experience with data pipeline orchestration tools (e.g., Apache Airflow) and data parsing. Familiarity with cloud service models, SaaS infrastructure, and related SDLC. Familiarity with containerization and container orchestration tools (e.g., Docker More ❯

Job Opportunity: Data Engineer Oliver Bernard has partnered with a retail bank undergoing a data transformation. Work Arrangement: Hybrid working - 2 days per week in their Cardiff office (CF10 1EP). Role Overview: Join their core Data team to deploy tooling through cloud platforms, utilize new LLM models, and develop new data More ❯

now to be part of our award-winning team at Sparta Global. About this role You'll become versatile in a wide array of tools across topics covering data focused coding, data visualisation, Cloud Services, Big Data and Machine learning. You will be equipped to work within a variety of roles including but not … limited to; Data Engineer, Data Analyst, Data Scientist, Information Engineer. You'll be designing, building, maintaining, and troubleshooting the data pipelines that enable organizations to store, process, and analyse their data, and ensuring the data is reliable through testing and debugging. Looking for efficiencies and optimising the data pipelines for scalability and performance will be a focus. You'll be handling and working with large sets of structured and unstructured data and will be responsible for ensuring that the data is organized and available for further analysis and investigation. Working with others is key, you could be working with other engineers, developers More ❯