1 to 25 of 277 Data Pipeline Jobs in the South East

engineering principles, big data technologies, cloud computing (specifically Azure), and experience working with large datasets. Key Responsibilities : Data Pipeline Development & Optimisation: Design, develop, and maintain robust and scalable data pipelines for ingesting, transforming, and loading data from various sources … effectively to both technical and non-technical audiences. Participate in code reviews and knowledge sharing sessions. Automation & DevOps: Implement automation for data pipeline deployments and other data engineering tasks. Work with DevOps teams to implement and Build CI/CD pipelines, for environmental deployments. Promote … SQL Database. Experience working with large datasets and complex data pipelines. Experience with data architecture design and data pipeline optimization. Proven expertise with Databricks, including hands-on implementation experience and certifications. Experience with SQL and NoSQL databases. Experience with data quality More ❯

Select how often (in days) to receive an alert: Head of Data Strategy Location: UK - Hatfield, UK - London, UK - Manchester, UK - Milton Keynes, UK - Nottingham Job-ID: 213406 Contract type: Standard Business Unit: Information Technology Life on the team You will oversee the development and implementation of a … comprehensive data strategy, encompassing data governance, data management, data quality, and data analytics to ensure high-quality data outcomes, insights leading to action, maximizing data value, and treating data as an asset. What you … ll do As Head of Data Strategy, you will be responsible for defining and implementing our Data Strategy, including data governance, management, quality, master data management, and reporting/analytics/outcomes-aligned with our Technology Strategy. You will: Own, define, and More ❯

The AI Engineer is a key role in the Data Platform Portfolio team, building the data platform, driving value from data across the business. With strong technical skills and business acumen to help turn millions of potential data points into actionable insights … our customer retention rates, and drive operating efficiencies across the business. The primary goals of the team are: To build and run a data platform which can create and deliver analytics to colleagues and deliver regulatory reporting. Ingest and transform data from multiple systems, modelling data … implementing ELT pipelines using tools like dbt, with strong knowledge of data warehousing, data lake concepts, and data pipeline optimization. Skilled in SQL for data manipulation, analysis, query optimisation, and database design. Artificial Intelligence and Machine Learning Understanding of machine learning More ❯

Job Title: Lead Data Engineer (AWS) Business Unit/Segment: Data Management/Analytics Location: London, United Kingdom (Flexible hybrid working) Employment Type: Permanent - Salary -£80k to £100k Summary of the Role: A leading global data and AI company is looking for a Lead … with emerging data technologies and recommend improvements to data engineering practices. Develop and enforce best practices for data pipeline orchestration, testing, and deployment. Provide mentorship, feedback, and leadership across project and operational initiatives. Collaborate with cross-functional teams to design a consistent and … CD practices and code versioning systems. Key Skills and Attributes: Cloud & Data Engineering: Deep expertise in AWS services and data pipeline development. Data Modelling & Architecture: Strong background in data warehousing and modern modelling frameworks. Leadership: Ability to lead teams, provide feedback More ❯

Job Title: Senior Data Engineer Location: Abu Dhabi Job Summary: As a Senior Data Engineer , you will be responsible for designing, developing, and maintaining advanced, scalable data systems that power critical business decisions. You will lead the development of robust data pipelines … role requires a deep understanding of modern data engineering practices, real-time processing, and cloud-native solutions. Key Responsibilities: Data Pipeline Development & Management: Design, implement, and maintain scalable and reliable data pipelines to ingest, transform, and load structured, unstructured, and real-time data … environments. Machine Learning & Advanced Analytics Enablement: Collaborate with data scientists to prepare and serve features for ML models. Maintain awareness of ML pipeline integration and ensure data readiness for experimentation and deployment. Documentation & Continuous Improvement: Maintain thorough documentation including technical specifications , data flow More ❯

As a Tech Manager - Data Engineer with AWS experience, you will play a crucial role in the design, development, and maintenance of our data infrastructure. Your work will empower data-driven decision-making and contribute to the success of our data-driven … initiatives. Key Responsibilities: Data Integration: Develop and maintain data pipelines to extract, transform, and load (ETL) data from various sources into AWS data stores for both batch and streaming data ingestion. AWS Expertise: Utilize your expertise in AWS services such … as Amazon EMR, S3, AWS Glue, Amazon Redshift, AWS Lambda, and more to build and optimize data solutions. Data Modeling: Design and implement data models to support analytical and reporting needs, ensuring data accuracy and performance. Data Quality: Implement data More ❯

Data Engineer Inside IR35 Contract Up to £750 per day DV Clearance ESSENTIAL London We are seeking a highly skilled Data Engineer to join a prestigious defence organisation in London. This is an exciting opportunity to lead and mentor a junior Data Engineer while … pipelines and products. Key Responsibilities: Technical Leadership: Lead and guide the technical direction of data engineering projects. Data Pipeline Development: Design, develop, and maintain robust data pipelines using Azure Data Factory (ADF), Azure Synapse Analytics, and SQL Server. Data … teams to ensure seamless integration and effective communication. Essential Skills and Experience: Proven experience in data engineering roles, including data pipeline development and data integration. Strong proficiency in Azure Data Factory (ADF), Azure Synapse Analytics, and SQL Server. Solid understanding of More ❯

the complexity of modern cruising - all while keeping things collaborative and fun. Your Mission: Are you passionate about unlocking the power of customer data? We're looking for a talented Lead Data Engineer to spearhead the design, development, and optimisation of our critical CRM and customer … data transformation. You'll play a pivotal role in building the data foundations for advanced analytics, personalised customer experiences, and effective marketing activation. As a senior member of the team, you will lead data engineers and champion best practices within our data … data engineers, fostering their growth and ensuring high standards in coding, testing, and documentation. Optimise: Continuously monitor and improve data pipeline performance, reliability, and cost-efficiency. Document: Create and maintain clear, comprehensive documentation for data models, pipeline architecture, and processes. What You More ❯

platforms that handle the complexity of modern cruising - all while keeping things collaborative and fun. Your Mission: Are you eager to build the data solutions that power exceptional customer experiences? We're looking for a motivated Data Engineer to join our team and focus on developing … and maintaining our critical CRM and customer data pipelines. Working closely with the Lead Data Engineer, you'll contribute directly to the data foundations that enable advanced analytics, personalisation, and effective marketing. This is a fantastic opportunity to grow your skills in a dynamic … pipelines and processes. What You'll Be Responsible For: The successful development, testing, deployment, and maintenance of assigned customer data pipeline components. Adhering to the technical quality, scalability, and reliability standards set for the customer data platform. Following data engineering best More ❯

Data Engineer | London Market Broker | London/Hybrid | £50 - 55,000 (doe) Are you passionate about designing and optimizing data infrastructures? Do you have a flair for working with large datasets and cloud technologies? We are seeking a highly skilled Data Engineer to join … our client, who are a boutique London market broker. As a Data Engineer, you will play a vital role in developing scalable data pipelines, ensuring data quality, and enabling efficient data processing to support business intelligence and advanced analytics. You will also … be involved in their Cloud migration project. Some key responsibilities include: Design, develop, and maintain scalable and reliable data pipelines. Build and optimize data architectures, data lakes, and data warehouses. Develop ETL/ELT processes to extract, transform, and load data More ❯

Data Engineer (DV Cleared) 6 Month Contract (Inside IR35) Location: London Rate: £700 - £800 per day Skills: DV Clearance ESSENTIAL - Must already hold DV clearance, Cloud technology experience We are looking to recruit a DV Cleared Data Engineer to join a defence organisation based in Central … office based in Central London. Key Responsibilities: Technical Leadership: Lead and guide the technical direction of data engineering projects. Data Pipeline Development: Design, develop, and maintain robust data pipelines using Azure Data Factory (ADF), Azure Synapse Analytics, and SQL Server. Data … teams to ensure seamless integration and effective communication. Essential Skills and Experience: Proven experience in data engineering roles, including data pipeline development and data integration. Experience working with diverse data sets and data sources. Excellent communication and collaboration skills. More ❯

Data Science Manager 📍 London | Lloyd’s Market | Hybrid £130,000 + up to 60% + Benefits Aegis Data Science Manager - Breifing call Notes Giuseppe is head of data science and analytics DSA lost data science manager who is moving to another opprutnity … wont happen yet provess - 2 stage interview process - 1st stage technical 2nd stage values as normal Python - Azure DevOps/Azure enviroment models and pipeline experiance strogn statiscal data science techniques ML engineering gen AI would cosndier outside of insruance guiseppe likes people not from isnruanbce but … wont happen yet provess - 2 stage interview process - 1st stage technical 2nd stage values as normal Python - Azure DevOps/Azure enviroment models and pipeline experiance strogn statiscal data science techniques ML engineering gen AI would cosndier outside of insruance guiseppe likes people not from isnruanbce but More ❯

or Manchester Salary: £60,000 - £65,000 depending on experience Department: Finance We're 1st Central, a market-leading insurance company utilising smart data and technology at pace. Rapid growth has been based on giving our 1.4 million customers exactly what they want: great value insurance with an … Employer of the Year at the British Insurance Awards 2024 and our Glassdoor score is pretty mega too! Are you passionate about driving data excellence within finance? Do you thrive on enhancing data functionality that supports business success? If so, we'd love to hear from … and Technology teams to define, capture, and document data requirements. Ensure smooth integration between finance systems and the wider data pipeline, including key logic and calculations. Help shape and evolve the Data Roadmap for Finance, aligning with strategic priorities and business needs. Collaborate More ❯

AWS Data Engineer S3 Data Centre of Excellence AWS Data Engineer S3 Data Centre of Excellence Country: United Kingdom Interested in part-time, job-share or flexible working? We want to talk to you! Join our community. We have an exciting opportunity … to join our Data Centre of Excellence as an AWS Data Engineer. The AWS Data Engineer will work within a team of analysts to create the data and insights which support the functional areas of the bank. You'll use a variety … of skills, excellent communication, functional awareness and an extensive analytical capability in understanding the types of models and data frameworks that can support the business strategies around our data assets. If you're someone who's performed a similar role already, this is the perfect opportunity More ❯

Join IFS. Job Description Are you ready to make waves in the world of AI? We're on the hunt for a Principal Data Engineer to join our dynamic global R&D organization. We're looking for someone who brings the heat, fosters seamless collaboration, and is always … the ground up. From framing problems and experimenting with new data sources and tools to the grand finale of data pipeline implementation and deployment. You will ensure scalability and top-tier performance. Locking arms with ML Engineers, Data Scientists, Architects, and Product/… data processing with Spark or Dask. Strong in Python, Scala, C# or Java, cloud SDKs and APIs. AI/ML expertise for pipeline efficiency, familiar with TensorFlow, PyTorch, AutoML, Python/R, and MLOps (MLflow, Kubeflow). Solid in DevOps, CI/CD automation with Bitbucket Pipelines More ❯

Transform Healthcare with Cutting-Edge Tech! ???? Position: Data Engineer (Python/Databricks) Location: Remote Salary: Up to £80,000 + Benefits Are you driven by a passion for health tech and innovation? Do you dream of revolutionizing clinical research through advanced technology? If so, we have an incredible … opportunity for you! Join our trailblazing team as a Data Engineer and play a pivotal role in building secure, scalable microservices that power clinical research applications. This is your chance to make a significant impact on healthcare while working with the latest advancements in data engineering. … About Us We are a pioneering health tech company committed to transforming clinical research through innovative data solutions. Our collaborative team, which includes Frontend Developers, QA Engineers, and DevOps Engineers, creates high-performance data pipelines and REST APIs that drive AI applications and external data More ❯

Data Engineer London (hybrid) Salary up to £80,000 per annum An exciting opportunity has arisen to join a well established organisation in the insurance space, in a challenging Data Engineer role. There will be a lot of stakeholder engagement, so the ideal candidate will have … both hands on and business facing experience. Preferably in the insurance sector. Key responsibilities: Design Data Lake and Data Warehouse solutions. Design Data Models using Data Vault and Dimensional modelling methods Implement automated, reusable and efficient batch data pipelines and … streaming data pipelines Work closely with Governance and Quality teams to ensure metadata, catalogue, lineage and known solutions to data issues are optimised Work closely with other data specialists and technicians in the organisation, such as Actuarial and Finance, to help with their data More ❯

Role: Data & Migration Workstream Lead Location: London (Oxford Circus) Hybrid: 1-2 days per week in the office About Monument Technology Limited In building its core UK banking business, Monument has developed an open, scalable, cloud native and world-leading banking technology platform, the third pillar of the … innovate at a fraction of the cost. From front end mobile apps, through core banking and CRM to the general ledger and a data lake, the platform can be consumed in whole (“bank in a box”) or in part as required. Our technology stack has gained positive attention … co ordinate technical delivery teams and cross-functional contributors including the ability to effectively engage with non-technical audiences. Skilled in data pipeline development and reporting implementation. Experience of data governance and data quality frameworks. Basic familiarity with cloud hosted online data More ❯

Data Engineering Lead (Snowflake & AWS Environment) Hybrid working: 3 days in TW6, Middlesex offices & 2 days home/remote Salary: Negotiable to £70,000 DOE plus 40 % bonus potential Job Ref: J12869 Please note we can only accept applications from those with current UK working rights for this … role, this client cannot offer visa sponsorship. An exciting opportunity has arisen within a FTSE 100 company for a Data Engineering Lead to play a pivotal role in operating and delivering the organisation’s data products. This position holds significant responsibility within the data leadership team, ensuring the data solutions and business processes are fully aligned and contribute to the vision and strategic direction of the organisation. This is an exciting to time to join the organisation as they are in the early stages of a major programme of work More ❯

in touch with at UKI.recruitment@tcs.com or call TCS London Office number 02031552100 with the subject line: “ Application Support Request” Role: Senior GCP Data Engineer. Job Type: Permanent Location: London, UK (Hybrid - 2 days WFO) Number of hours: 40 hours per week – full time We have an exciting … opportunity for you as a GCP Data Engineer. Careers at TCS: It means more TCS is a purpose-led transformation company, built on belief. We do not just help businesses to transform through technology. We support them in making a meaningful difference to the people and communities they … learning opportunities. Lead projects and inspire both colleagues and stakeholders. Lead a team and their technical growth. The Role As a Senior GCP Data Engineer you will be responsible for providing top-tier data capabilities, leveraging your engineering skills, and embracing the possibilities offered by cloud More ❯

for developing and delivering all technology solutions for the firm’s internal use. We are taking a cloud-first approach to transform our data platforms and analytical applications across the firm. We are seeking an experienced data engineer to shape and accelerate the delivery of target … state data platform enabling GenAI use-cases. We want a passionate specialist who loves to build data solutions as part of multi-disciplinary team, working closely with digital product professionals, data scientists, cloud engineers and others. Your impact within our firm We are looking … independently. Able to work in complex and very fast paced environment Hands-on experience with Docker Solid and demonstrable background in data pipeline performance and diagnostics Interest in Generative AI and other ML topics Kedro framework experience as a plus Holds their ground, opinionated, not afraid to More ❯

Senior Data Engineer – West London (office based 4 days a week) We are currently looking for a Senior Data Engineer to join an advanced analytics business who provide insights for ecommerce customers. As an organisation they have experienced amazing success over the last few years and … space. Requirements Strong expertise in PostgreSQL and SQL. Proficient in Python for data engineering tasks. Solid understanding of ELT pipelines, data architecture, and data modeling. Experience with modern data transformation tools such as dbt . Familiarity with AWS-based data pipeline services and cloud architecture. Comfortable working in agile teams and cross-functional environments. Knowledge of statistical concepts and their application in data-driven systems. If you have the desired skills and experience and would like to find out more please respond to this advert More ❯

join us at FPSM and be part of something bigger… What we are looking for: We are seeking a highly motivated and experienced Data Product Manager to own the strategy, roadmap, and execution of our Customer Data Platform (CDP) and Single Customer View (SCV). This … is a critical role within our rapidly growing data organization, and you will be instrumental in enabling data-driven decision-making across the business. You'll work closely with engineering, marketing, product, and other internal teams to ensure our CDP is a powerful tool for understanding … with overall business objectives and stakeholder needs. Requirements Gathering: Proactively gather, analyze, and prioritize requirements from internal stakeholders (marketing, product, analytics, etc.) for data integration, access, and utilization. Data Source Integration: Oversee the integration of various data sources into the CDP, ensuring data More ❯

Job Title: Senior Data Architect Location: Hybrid (London, UK) Salary: £75,000–£135,000 per annum, plus 20% Bonus and Benefits About the Role: Our client is a leading professional services firm with a strong UK presence and global reach, delivering expert audit, tax, advisory, and risk solutions. … As part of their ongoing investment in digital transformation, they are scaling their data capabilities to better serve clients in regulated and data-intensive sectors. This newly created Senior Data Architect role will focus on designing and implementing data architecture solutions for … architecture within the insurance and asset management industries, including regulatory and compliance requirements. Proficiency in SQL , Python , and modern data pipeline design (ETL/ELT). Experience with both batch and streaming data processing architectures. Strong communication and stakeholder engagement skills, particularly with More ❯

Senior Data Engineer Job Type : Permanent Location: London – Hybrid/flexible Salary: £55,000 - £70,0000 Consultant: Mags Rendle Excellent career opportunity! The Organisation TDRC are excited to be once again working with one of the UK’s leading Wealth Management firms. Through their robust infrastructure, cutting edge … client base. They are providing the utmost level of quality and security in everything they do. The Role We are seeking a Senior Data Engineer to join our Azure-native data team in a leading UK financial services firm. This role is key to maintaining and … evolving our data platform, supporting both ongoing enhancements and greenfield development. You will work on data integration, ETL pipelines, analytics, and AI-driven insights, ensuring robust data solutions that support business growth. You will be responsible for designing, building, and optimising Azure Data More ❯