1 to 25 of 85 Data Pipeline Jobs in the South East

Engineer, you will operate within an Agile delivery environment, working closely with the Data Product Manager and Data Architect to ensure your team maintains a pipeline of delivery against the Backlog; providing vital insight from our wide-ranging dataset to support executive and operational decision making that will underpin sustained growth of BFS business units … solutions. You will also have management responsibilities for a small team of Data Engineers, who you will coach, support and organise to ensure we sustain a predictable pipeline of delivery, whilst ensuring all appropriate governance and best practice is adhered to. Your responsibilities as our Senior Databricks Data Engineer will include: Understand the business/… Azure DevOps, collaborating on Sprint cycles, code deployment, version control, and development practices Develop and maintain data models, schemas, and documentation Monitor and troubleshoot data pipeline issues, ensuring timely resolution Stay up-to-date with the latest industry trends and technologies in data engineering and recommend improvements to existing systems or processes as More ❯

Role: Head of Data Location: Milton Keynes (Hybrid 2 Days On-site Weekly) Salary: £50,000 - £60,000 Benefits: Bonus, Company Cars, Flexible Benefit Package Network IT is recruiting for a Head of Data to join a leading automotive brand's UK IT function. In this senior role, youll lead a high-performing data management team, drive technical delivery across enterprise data platforms, and ensure compliance, scalability, and performance within a global standards-based ecosystem. Reporting to the Head of IT, youll take ownership of operational excellence, stakeholder engagement, and vendor performance across a broad data estate including Azure Data Factory and Data Lake … Strong experience in IT service management, strategy planning, and business process alignment. Hands-on knowledge of Azure Data Factory, Data Lake , and data pipeline architecture. Proven experience managing data security, access, and regulatory compliance. Familiarity with automation tools (AI, RPA) and Agile project delivery (e.g. Scrum, Kanban). Demonstrated ability to More ❯

City & Capital are currently representing a highly respected & fast-growing property finance lender who are achieving exciting growth. They have engaged our services to appoint a technical & experienced Data Architect who will take oversight of the company’s overall data architecture. The Company Our client is known for supporting property developers & investors with access to funds … in their sector & ahead of the pack in terms of innovation and engagement with tech and systems that drive business performance and lending efficiency. The Department & Team The Data Architect is a senior member of the IT department and will work closely with the Head of IT and CTO on critical projects that drive the growing technological landscape … of the firm. The IT team is set to grow considerably in light of the planned projects, making the appointment of a capable and forward thinking data architect absolutely critical to success. The department works closely with senior leaders & users in operational departments, so the capacity to communicate across the business is a firm requirement for this role. More ❯

We are looking to hire a additional 2 Data Engineers to join the Satalia Data Engineering Team. We are investing massively in developing next-generation AI tools for multimodal datasets and a wide range of applications. We are building large scale, enterprise grade solutions and serving these innovations to our clients and WPP agency partners. As … Your contributions will directly shape cutting-edge AI products and services that make a tangible impact for FTSE 100 clients. What you’ll be doing: Collaborate closely with data scientists, architects, and other stakeholders to understand and implement business requirements. Provide data engineering support for AI model development and deployment, ensuring data scientists have … access to the data they need in the format they need it. Implement and optimize data transformations and ETL/ELT processes, using appropriate data engineering tools. Work with a variety of databases and data warehousing solutions to store and retrieve data efficiently. Implement monitoring, troubleshooting, and maintenance procedures for More ❯

Data Engineer (Python SQL ETL) London/WFH to £80k Are you a tech savvy Data Engineer with experience of working with sensitive patient/healthcare data? You could be progressing your career as a Data Engineer working on Tech for Good application for the healthcare sector that utilises AI and Machine … Learning techniques to help prevent patients from experiencing health crisis. As a Data Engineer you will take ownership of the healthcare data pipelines and platforms, focussing on implementing robust ETL processes and scalable data architectures, collaborating with various teams to deliver high quality data solutions. You'll develop and maintain data infrastructure on Azure Data Lakes and Data Warehouses; create robust data transformation processes that ensure data quality, consistency and security and implement automation workflows. You will help to maintain compliance with ISO27001 and ISO13485 requirements for healthcare data management and ensure that security first practices are employed in More ❯

Join us as a Data Platform Engineer for our client. At Peregrine, were always seeking Specialist Talent that have the ideal mix of skills, experience, and attitude, to place with our vast array of clients. From Business Analysts in large government organisations to Software Developers in the private sector we are always in search of the best talent … you will be deployed across our portfolio of clients as a specialist consultant, working on a wide array of complex projects across multiple industries. The Role: As a Data Platform Engineer in a highly regulated environment, you will be responsible for designing, building, and maintaining secure and scalable data infrastructure that supports both cloud and on … with over 3 years of relevant experience in data engineering, platform engineering, or a related field, with demonstrated hands-on expertise in NiFi and data pipeline design in regulated environments. Responsibilities: Design, develop, and maintain robust and secure data pipelines using NiFi and related big data technologies. Troubleshoot and optimize NiFi More ❯

The Role: As a Data Platform Engineer in a highly regulated environment, you will be responsible for designing, building, and maintaining secure and scalable data infrastructure that supports both cloud and on premises platforms. You will play a key role in ensuring that all data systems comply with industry regulations and security standards while … enabling efficient access for analytics and operational teams. A strong command of Apache NiFi is essential for this role. You will be expected to design, implement, and maintain data flows using NiFi, ensuring accurate, efficient, and secure data ingestion, transformation, and delivery. You should be adept at identifying and resolving issues within NiFi flows, managing performance … with over 3 years of relevant experience in data engineering, platform engineering, or a related field, with demonstrated hands-on expertise in NiFi and data pipeline design in regulated environments. Responsibilities: Design, develop, and maintain robust and secure data pipelines using NiFi and related big data technologies. Troubleshoot and optimize NiFi More ❯

We do energy differently - we do it all. We make it, store it, move it, sell it, and mend it. About your team: Youll be joining Centricas Group Data function, working in close partnership with our Data Science and Data Governance teams. Together, were building a data ecosystem that powers strategic decision … making and enables the creation of reusable, high-quality data assets across the business. About your role: As Head of Data Engineering , youll lead the design, development, and optimisation of Centricas data infrastructure. Youll define and execute a forward-thinking data engineering strategy, ensuring our pipelines and platforms are scalable, secure, and … aligned with business goals. Youll mentor a high-performing team, collaborate across departments, and champion best practices in data management, governance, and technology adoption. Youll also work closely with the Head of Data Science to ensure our infrastructure supports advanced analytics and modelling, enabling the organisation to harness data-driven insights effectively. Regular travel More ❯

Job Description: We are seeking a highly skilled Data Architect for Gen AI Engineering to lead the design and implementation of robust data architectures that support our generative AI initiatives across various business functions, including Marketing, Product Development, and Customer Insights. This role is transformation-funded and therefore has an expected end date of 31st December … 2026. What are we looking for? • A seasoned data architect with 8+ years of experience in data architecture and engineering, specifically in the context of generative AI and machine learning applications. • Strong expertise in designing scalable data models and architectures, with proficiency in … cloud platforms such as AWS, GCP, or Azure, and familiarity with data warehousing solutions. • Experience with data integration tools, ETL processes, and data pipeline development, along with a solid understanding of data governance and security best practices. • Proven leadership skills in managing cross-functional teams, driving data strategy, and More ❯

Data Engineer - Azure/Microsoft Fabric City of London - Hybrid Working Circa £55-65k + Bonus & Benefits Our client, a forward-thinking and globally connected staffing & consulting organization is seeking a skilled and driven Data Engineer to join their growing Data & Analytics team. Based in the City of London, this role presents a … fantastic opportunity for someone with 2-3 years of data engineering experience who is passionate about building robust data solutions using Microsoft Azure and Microsoft Fabric technologies. This is a hands-on technical role where you'll work closely with the Lead Data Architect and collaborate across the business to design, build, and support … scalable data infrastructure that powers business intelligence, reporting, and analytics. The ideal candidate will be eager to grow their skills and take ownership of impactful data projects within a fast-paced, modern tech environment. Key Responsibilities: Design, develop, and maintain data pipelines and data warehousing solutions using Azure services Work with stakeholders More ❯

Role: Senior Data Engineer Salary/Rate: £600-700 per day inside IR35 Location: hybrid multiple locations available, 2 days on siteContract Duration: 4 months, potential to extend We are currently looking for a Senior Data Engineer for our government client. This Senior Data Engineer role is hybrid, based between 2 days per week … on site and the remainder of the week working remotely. Office locations include most major towns, with London being a preference. The contract for the Senior Data Engineer position is for 4-months with potential to extend, operating inside IR35. Security Clearance: Security Check ("SC Clearance") This role is inside IR35 - Due to the service of the role … it will now be based on an Umbrella solution. Essential skills/experience required: Strong experience designing, building and maintaining data pipelines using modern, cloud-based tools and practices Proficiency in Python and SQL for data engineering tasks Experience with DBT and a good understanding of data modelling approaches (e.g. star schema, dimensional modelling More ❯

Azure Data Engineer Rate - up to £450/day Outside IR35 Location - 2 days a week onsite in Central London Duration - 6 Months + Likely Extension We are working with a well-established consultancy supporting a high-impact data transformation programme for an end client based in Central London. They are seeking an experienced Azure Data … contract, with strong potential for extension. This role is Outside IR35 and offers £450/day, requiring 2 days onsite per week in Central London. Key Responsibilities - Azure Data Engineer: Design, build and maintain scalable and secure data pipelines on the Azure platform. Develop and deploy data ingestion processes using Azure Data Factory, Databricks (PySpark), and Azure Synapse Analytics. Optimise ETL/ELT processes to improve performance, reliability and efficiency. Integrate multiple data sources including Azure Data Lake (Gen2), SQL-based systems and APIs. Collaborate with Data Architects, Analysts, and stakeholders to define data models and ensure best practices in data More ❯

Python Data Engineer Location: Farnborough (Hybrid) Employment Type: Full-Time, Permanent Salary: c£50,000We are recruiting on behalf of a leading organisation based in Farnborough, currently seeking a highly skilled Python Data Engineer to join their growing data team. This is a fantastic opportunity to work with cutting-edge technologies and play a … key role in the design, development, and optimisation of scalable data solutions. The Role As a Python Data Engineer, you will be instrumental in building robust data pipelines and maintaining cloud-based databases to support a range of business functions. Working closely with analysts and data scientists, you'll help solve complex … data challenges and contribute to the company's data-driven strategy. Key Responsibilities Design and implement efficient, scalable data pipelines using Python and SQL Server Develop, test, and deploy ETL workflows and data processing solutions using Docker containers on Azure Collaborate with data stakeholders to resolve data-related More ❯

now to be part of our award-winning team at Sparta Global. About this role You'll become versatile in a wide array of tools across topics covering data focused coding, data visualisation, Cloud Services, Big Data and Machine learning. You will be equipped to work within a variety of roles including but not … limited to; Data Engineer, Data Analyst, Data Scientist, Information Engineer. You'll be designing, building, maintaining, and troubleshooting the data pipelines that enable organizations to store, process, and analyse their data, and ensuring the data is reliable through testing and debugging. Looking for efficiencies and optimising the data pipelines for scalability and performance will be a focus. You'll be handling and working with large sets of structured and unstructured data and will be responsible for ensuring that the data is organized and available for further analysis and investigation. Working with others is key, you could be working with other engineers, developers More ❯

Role: Snowflake Data Architect Location: Hove, UK Type: Permanent Role Work Mode: Hybrid Role & Responsibilities Define and implement the end-to-end architecture of Data warehouse on Snowflake. Create and maintain conceptual, logical, and physical data models in Snowflake. Design data pipelines and ingestion frameworks using Snowflake native tools. Collaborate with Data Governance teams to establish data lineage, data quality, and access control mechanisms. Engage with data stewards and stakeholders to build a comprehensive and scalable data warehouse. Implement RBAC, data masking, and encryption practices to ensure compliance with data security policies. Qualifications & Skills 10+ years of experience … in designing enterprise data platforms, with at least 5+ years in Snowflake. Strong expertise in SQL and data warehousing. Hands-on experience working in the insurance industry; prior experience with L&G is an advantage. 3+ years of experience with DBT for data transformation. Deep understanding of agile methodologies in data environments. More ❯

Job Title: Graduate Data Engineer Location: Brighton, UK About Us: We are a dynamic and innovative technology consultancy based in Brighton. We specialise in providing cutting-edge solutions to our clients, enabling them to leverage data for strategic decision-making and operational efficiency. Our team is composed of talented professionals who are passionate about technology and … committed to delivering exceptional results. Job Description: Position Overview: We are seeking a highly motivated and enthusiastic Graduate Data Engineer to join our team. As a Graduate Data Engineer, you will be an integral part of our data engineering team, working on a variety of projects that involve data collection, transformation, and … analysis. This is an excellent opportunity for a recent graduate to kick-start their career in data engineering within a supportive and collaborative environment. Key Responsibilities: Assist in the design, development, and maintenance of data pipelines and ETL processes. Collaborate with data scientists, analysts, and other stakeholders to understand data requirements and More ❯

Data Architect ££105,000 - £115,000 GBP Hybrid WORKING Location: London, Central London, Greater London - United Kingdom Type: Permanent Data Architect Location: London (Hybrid) Salary: Up to £115,000 About our client: Our client is a global technology consultancy delivering transformation across data, digital, and infrastructure. Known for driving cutting-edge solutions across both … systems at national scale and pride themselves on technical depth, agile delivery, and a culture of collaboration. The Opportunity: This is a deeply technical role for a skilled Data Architect to join a high-performing engineering team focused on the design and implementation of robust data models within analytical and real-time domains. You'll work … across multiple consulting engagements with cross-functional teams, building modern, scalable data architecture from the ground up. Important: To be considered, you must currently hold active Security Clearance or have previously held Security Clearance . Additionally, you must have demonstrable experience delivering consulting engagements end to end - from discovery through to implementation - across client environments. What You'll More ❯

Data Scientist 6 Month contract initially Based: Hybrid - Maximum of 3 days p/w - Wokingham Rate: £Market rates p/d (via Umbrella company) We have a great opportunity with a world leading organisation where you will be provided with all of the support and development to succeed. A progressive organisation where you can really make a … difference. We have a great opportunity for a highly experienced Data Scientist to join the team. We are seeking a skilled and experienced Data Scientist with expertise data scientists and LLM use cases and Data Engineering with Graph DB experience and Vector DB experience. As a Data Scientist, you will … datasets, building predictive models, and extracting meaningful insights to drive informed decision-making. Key Responsibilities: * Analyse and pre-process large-scale time series datasets, identifying relevant variables and data cleansing requirements. * Develop and implement advanced statistical models and machine learning algorithms to perform predictive analysis on time series data. * Build and validate forecasting models to predict future trends More ❯

needs of the BioPharma sector. IDBS, a Danaher company, leverages 35 years of scientific informatics expertise to help organizations design, execute and orchestrate processes, manage, contextualize and structure data and gain valuable insights throughout the product lifecycle, from R&D through manufacturing. Known for its signature IDBS E-WorkBook software, IDBS has extended its flexible, scalable solutions to … fail and fun environment which is founded on trust, empowerment and autonomy? Do you enjoy solving complex customer problems as a team? We are currently seeking a Principal Data Engineer who will be a vital cog in our team, designing and developing IDBS's data driven products. You will deliver data-rich software and … design, technical approach and implementation mechanisms adopted by the team.You will be directly involved in the development of data-centric products from ingest to egress via pipelines, data warehousing, cataloguing, integrations and other mechanisms. The role will be involved in all aspects of the software delivery lifecycle including the creation and elaboration of business requirements, functional More ❯

remote Benefits: Healthcare and great holidays The Role As a Machine Learning Operations Engineer, you will be instrumental in designing, deploying, and maintaining machine learning systems that drive data-driven decision-making in the pensions and financial services sector. Using Azure's cloud ecosystem, you will build scalable ML workflows that support actuarial modelling, risk prediction, and client … classification, regression, ensemble methods, survival models) to improve predictions and automate core processes. 2. MLOps & Model Lifecycle Management * Own the end-to-end ML lifecycle using Azure ML pipelines, Data Factory (ADF), and ML tracking tools.* Implement CI/CD pipelines with Azure DevOps, ensuring robust version control, testing, and seamless deployment.* Monitor production ML systems for performance … data drift, and anomalies using Azure Monitor or other observability tools.* Schedule and automate model retraining pipelines to maintain performance over time. 3. Data Engineering & Preprocessing * Develop and maintain scalable ETL/ELT data pipelines using Azure Data Factory, Data Lake, and SQL.* Prepare and preprocess large datasets to ensure More ❯

Job Title : Data Engineer - Snowflake | DBT | SQL £700/day (Umbrella) London - 3 days per week on-site 6 months (Initial Contract) Are you ready to play a key role in shaping the future of a high-impact enterprise data platform? Join a fast-growing data team working on a greenfield data warehousing project at one of the world's most respected financial institutions. This is an opportunity to work on cutting-edge data infrastructure using Snowflake, DBT, and Sigma, supporting business-critical data needs and enabling meaningful insights at scale. You'll be part of a collaborative team split across London and Ireland, contributing to … a platform that directly supports secure, scalable data sharing and real-time decision-making. What You'll Be Doing : Model and build robust, scalable data pipelines and warehouse structures using Snowflake and DBT Collaborate with stakeholders to ensure data solutions meet customer and business needs Develop and optimise data models to support More ❯

Senior Data Engineer London, UK About the Role We're looking for a Senior Data Engineer to lead the design and delivery of modern, cloud-based data solutions. You'll play a key role in transforming enterprise data platforms, enabling global data alignment, and driving business value through scalable architecture … and advanced analytics. Key Responsibilities Design and implement end-to-end data pipelines using Azure and Databricks. Modernize legacy data warehouses and unify data across global regions. Optimize performance, cost, and scalability of data platforms. Mentor junior engineers and promote best practices. Collaborate with cross-functional teams to translate business needs into … technical solutions. Core Skills Cloud & Platforms : Azure, AWS, SAP Data Engineering : ELT, Data Modeling, Integration, Processing Tech Stack : Databricks (PySpark, Unity Catalog, DLT, Streaming), ADF, SQL, Python, Qlik DevOps : GitHub Actions, Azure DevOps, CI/CD pipelines Please click here to find out more about our Key Information Documents. Please note that the documents provided contain More ❯

Devonshire Hayes have partnered with a fantastic organisation to help them secure a dynamic and hands-on Engineering Manager across Data & AI. You will have experience - - Leading, managing, building and scaling data teams in a SaaS, RegTech, or technology driven environment. - Hands-on expertise in modern data engineering practices, cloud data platform and full data lifecycle. - Strong programming skills would be ideal for this position, languages such as Python, Java or Scala with Data pipeline frameworks and cloud infrastructure (AWS or Azure). - A proven track record of delivering innovative, data driven products from concept to production, ideally in a regulated industry. … Deep understanding of data governance, security, privacy standards especially within a B2B or regulated context. Working arrangement: 4 days in the office/1 day from home . More ❯

Team: The Academic Information Systems ( AI S) DataOps team is a shared technology group responsible for building, administering, governing, and setting standards for a growing number of strategic data platforms and services. Our capabilities enable data to be extracted, centralized, transformed, transmitted, and analyzed across a range of products in the AIS space. Due to our … footprint across the enterprise, we are relied upon to ensure our systems are trusted, reliable and available.The technology underpinning these capabilities includes industry leading data and analytics products such as Snowflake, Astronomer/Airflow, Kubernetes , DBT, Tableau, Sisense , Collibra, and Kafka/Debezium . Our mission is to enable frictionless experiences for our AIS colleagues and customers so … that they can openly and securely consume trustworthy data, enhancing everyday interactions and decisions. About the Role: As a S enior Data Engineer I , you will be responsible for helping to creat e a data infrastructure that is secure, scalable, well-connected, thoughtfully architected while also building a deep domain knowledge of our business More ❯

Location: Guernsey, Haywards Heath, Home Office (Remote) or Manchester Salary: from £70,000 depending on experience Department: Technology and Data We're 1st Central, a market-leading insurance company utilising smart data and technology at pace. Rapid growth has been based on giving our 1.4 million customers exactly what they want: great value insurance with an … design, through development and testing, to deployment, production, monitoring and continuous improvement optimising cutting-edge AI solutions across the business. One day you might be collaborating with our Data Scientists, Software Engineers and Architects on a new GenAI capability. The next, you're fine-tuning prompts or experimenting with Computer Vision and Natural Language Processing (NLP). You … control, and continuous integration. Develop and maintain automated frameworks for testing, validation, and monitoring of AI model accuracy, performance, and reliability. Establish best practices for AI data pipeline development and integration with infrastructure to support scalable deployment. Collaborate closely with Data Scientists, Data Engineers, Software Developers, and Architects for seamless execution of AI More ❯