1 to 25 of 98 Data Pipeline Jobs in Southampton

Key Responsibilities: Data Pipeline Development: o Develop and maintain data pipelines for ingesting, transforming, and loading data from various sources (e.g., APIs, databases, files) into the Azure Databricks platform. o Implement data extraction, transformation, and loading (ETL) processes. o Ensure data pipelines are efficient, reliable, and scalable. Data Transformation & Processing: o Implement data transformations using Spark (PySpark or Scala) and other relevant tools. o Perform data cleaning, validation, and enrichment. o Ensure data quality and consistency. Azure Databricks Implementation: o Work … in team meetings and code reviews. Automation & DevOps: o Build CI/CD pipelines, for environmental deployments. o Contribute to automating data pipeline deployments and other data engineering tasks. o Learn and apply DevOps principles. More ❯

Job Title: Senior Data Engineer Key Requirements: 4-8 years of experience from tier 1 or 2 big tech companies Job Location: Abu Dhabi, UAE Benefits: Work with cutting-edge technology through modern infrastructure and automation projects Thrive in a growth-focused environment that prioritizes learning, innovation, and … role requires a deep understanding of modern data engineering practices, real-time processing, and cloud-native solutions. Key Responsibilities: Data Pipeline Development & Management: Design, implement, and maintain scalable and reliable data pipelines to ingest, transform, and load structured, unstructured, and real-time data … environments. Machine Learning & Advanced Analytics Enablement: Collaborate with data scientists to prepare and serve features for ML models. Maintain awareness of ML pipeline integration and ensure data readiness for experimentation and deployment. Documentation & Continuous Improvement: Maintain thorough documentation including technical specifications , data flow More ❯

the complexity of modern cruising — all while keeping things collaborative and fun. ⭐ Your Mission: Are you passionate about unlocking the power of customer data? We're looking for a talented Lead Data Engineer to spearhead the design, development, and optimisation of our critical CRM and customer … data transformation. You'll play a pivotal role in building the data foundations for advanced analytics, personalised customer experiences, and effective marketing activation. As a senior member of the team, you will lead data engineers and champion best practices within our data … data engineers, fostering their growth and ensuring high standards in coding, testing, and documentation. Optimise: Continuously monitor and improve data pipeline performance, reliability, and cost-efficiency. Document: Create and maintain clear, comprehensive documentation for data models, pipeline architecture, and processes. 🔧 What You More ❯

Data Management Specialist (Azure) - Newcastle/Hybrid You will provide expert-level troubleshooting, root cause analysis, and performance optimisation for data platforms,, such as Azure SQL databases, Fabric Warehouse and Onelake, Databricks, and Azure Data Factory. Purview or another MDM tool is highly desirable. … Skills/experience Bachelor’s degree in Information Technology, Computer Science, Data Management, or a related field. 4+ years of experience in data management, data operations, or IT support roles, with at least 2 years in Level 3/4 support or equivalent. Hands … on experience with database technologies ( Azure landscape , Oracle, PostgreSQL, etc.). Desirable: Industry certifications in professional data management, cloud platforms (AWS, Azure, GCP), metadata management, or database administration (e.g., Microsoft Certified: Azure Data Engineer Associate, DAMA CDMP) are desirable. Proven experience with ETL tools, data More ❯

Role: Data Engineering Analyst(Python & Power BI) Location: Leeds/Edinburgh(Hybrid) Duration: Contract About Us: We're seeking a skilled Data Engineering Analyst to join our team. You'll play a key role in designing, building, and maintaining our data infrastructure, ensuring seamless … data flow and insightful analytics. Key Responsibilities: Design, develop, and maintain data pipelines using Google Cloud Platform (GCP) services like BigQuery, Cloud Storage, and Cloud Dataflow. Develop and maintain reports, dashboards, and visualizations using Power BI, ensuring data-driven insights for business stakeholders. Write … and reporting. Strong programming skills in Python, with experience in data processing libraries like Pandas and NumPy. Experience with data pipeline development, data warehousing, and ETL processes. Strong analytical and problem-solving skills, with attention to detail and ability to work in a More ❯

Key aspects of the role include: Data Pipeline Development: Architect, develop, and optimise data extraction, transformation, and loading (ETL) processes for high-volume healthcare data. Integration of NHS Systems Data: Work directly with NHS data sources and ensure seamless integration of … systems like UDAL, NCDR, SUS+, HES, and ODS into our analytics platform. System Performance: Monitor and enhance the performance of our data systems to ensure reliability and responsiveness. Collaboration: Partner with data scientists, analysts, and stakeholders to ensure that data solutions align with business … needs and analytic strategies. Scalability & Security: Implement and enforce best practices for data security, quality, and scalability across our data infrastructure. Essential Skills: Healthcare Data Expertise: Proven experience in designing and managing data solutions within healthcare environments. Demonstrated ability to work with More ❯

the ease of use - they have introduced several industry-first products and maintain a strong focus on research and development. As a Junior Data Engineer, you will play a crucial role in developing the data infrastructure that supports the analytics and operational functions of our business … enterprise. Role & Responsibilities: Data Infrastructure & Support: Provide first-line support for data engineering tasks, such as managing and monitoring data pipelines, resolving issues, and ensuring data integrity. Work with both structured and unstructured datasets to design and implement data models … -related errors. Collaborate with business stakeholders to identify data requirements and deliver sustainable solutions. Monitor and troubleshoot data pipeline issues to maintain data integrity and accuracy. Assist in the development, maintenance, and optimization of ETL (Extract, Transform, Load) processes for efficiency More ❯

join us at FPSM and be part of something bigger... What we are looking for: We are seeking a highly motivated and experienced Data Product Manager to own the strategy, roadmap, and execution of our Customer Data Platform (CDP) and Single Customer View (SCV). This … is a critical role within our rapidly growing data organization, and you will be instrumental in enabling data-driven decision-making across the business. You'll work closely with engineering, marketing, product, and other internal teams to ensure our CDP is a powerful tool for understanding … with overall business objectives and stakeholder needs. Requirements Gathering: Proactively gather, analyze, and prioritize requirements from internal stakeholders (marketing, product, analytics, etc.) for data integration, access, and utilization. Data Source Integration: Oversee the integration of various data sources into the CDP, ensuring data More ❯

London (links Piccadilly in the south to Oxford Street in the north) **Work Arrangement:** In-office only, five days a week. **Job Title:** Data Architect/Data Engineer **Experience Required:** 7-8 Years **Role Overview:** We are seeking a Senior Data Architect/Data Engineer to design and implement scalable data solutions, including real-time and ETL/ELT data pipelines. Responsibilities include optimizing our data warehouse, creating Power BI reports, and developing AI-driven solutions using OpenAI APIs. **Key Responsibilities:** - Design and implement robust real … time data pipelines from diverse sources. - Maintain a high-performance data warehouse for reporting and analytics. - Develop insightful Power BI dashboards and reports for stakeholders. - Use Azure Data Factory, Azure Functions, and Logic Apps for data integration. - Employ Python and SQL for More ❯

make better use of it. We provide advice, support and delivery services across various industry sectors covering a multitude of areas, ranging from Data Strategy, Governance and Security to Data Platform Modernisation, Cloud and Customer Intelligence. We do everything with a view to creating tangible business … measure how we are performing and ensures we deliver the value that our customers deserve. Summary We are seeking a skilled and proactive Data Engineer with experience in Microsoft Fabric to join our growing data team. You will be responsible for developing, maintaining, and optimising data … lakehouse architectures. Hands-on experience with Power BI for data integration and visualisation. Familiarity with DevOps practices, particularly around data pipeline deployment. Good problem-solving skills and the ability to work under minimal supervision. Effective communicator with stakeholders of varying technical expertise. Benefits Annual Leave More ❯

Who are Inference Group? Are you ready to be a part of a company that’s redefining the future of data and AI? Welcome to Inference Group, where we’re not just a tech startup – we’re a team of passionate innovators dedicated to empowering businesses through the … transformative power of data, machine learning, and AI. At Inference Group, we’re on a mission to help businesses unlock their true potential and drive sustainable growth. As we rapidly expand, we’re looking for like-minded, creative technical experts who want to make a real impact. Here … of MLOps practices and tools – e.g., version control for data/models, CI/CD for machine learning, containerization (Docker, Kubernetes), and pipeline orchestration. Understanding of data engineering principles (ETL/ELT processes, data warehousing) to oversee data pipeline development. More ❯

ideally, or any part of the world, as long as they are willing to relocate to KL, Malaysia. Are you passionate about using data to drive innovative solutions in a fast-paced environment? We're looking for a Senior Data Engineer to join a cutting-edge … technology company based in Kuala Lumpur! As a Senior Data Engineer, your mission will be to support data scientists, analysts, and software engineers by providing maintainable infrastructure and tooling for end-to-end solutions. You’ll work with terabytes to petabyte-scale data, supporting … multiple products and data stakeholders across global offices. Key Responsibilities Design, implement, operate, and improve the analytics platform Design data solutions using various big data technologies and low latency architectures Collaborate with data scientists, business analysts, product managers, software engineers and other More ❯

Azure Data Engineer/Consultant – Up to £75K + Bonus Location: Remote (UK-based, with very occasional travel) Salary: Up to £75,000 + bonus Company & role Join one of the UK’s most advanced Microsoft partners, specialising in Azure-led Data & AI solutions. This consultancy … is known for transforming data into tangible value through innovative platforms and strategic data services. With a collaborative team of experts, they deliver impactful solutions across enterprise clients, leveraging the very latest in Microsoft technology. Why This Role Stands Out Play a key role in shaping … across the Azure Data Platform (Fabric, Synapse, Databricks, Data Lake, Data Factory, Purview, etc.) Strong data pipeline and ETL skills, including data modelling and transformation Experience implementing Medallion Architecture (Bronze, Silver, Gold layers) in lakehouse or Databricks environments Familiarity More ❯

Azure Data Engineer Remote (UK-based) Up to £60,000 per annum About the Company We are a leader in the insurance sector, committed to data-driven decision making and modernising our technology landscape. As part of our continued growth, we’re seeking a skilled Azure … Data Engineer to support our data platform and engineering initiatives. Role Overview As an Azure Data Engineer, you will be instrumental in designing, building, and optimising data pipelines and architecture in our Azure environment. This role is ideal for someone with strong … SQL skills and a background in Azure data services, looking to play a key role in transforming and scaling our data operations. Key Responsibilities Develop and maintain scalable data pipelines in Azure. Build and optimise data integration workflows using Azure Data More ❯

Data Engineer/Data Architect Location: Penryn, Cornwall- On premise About Us: Aspia Space is building the next generation of planetary intelligence. We transform observational data into trusted, consumable intelligence for agriculture, finance, environmental planning and policy. From our offices in Harwell and Cornwall … we push Earth data to its limits to deliver insights no one else can, developing products that combine satellite imagery and expert ground truth with the very best observational science and deep learning tools to support clients across the world, powering everything from smallholder farming programmes in Africa … impact. Our ambition is global, but our focus is always local, measurable, and relevant. Role Overview: We’re looking for a highly skilled Data Engineer/Data Architect who can hit the ground running and join us in our Penryn office. You will be instrumental in More ❯

Position Overview Our client, a leading player in the insurance sector, is seeking a seasoned Azure Data Factory (ADF) Architect to spearhead the design, implementation, and optimization of data pipelines within the Azure ecosystem. This pivotal role involves providing strategic consulting and technical leadership to ensure … ADF features, troubleshooting, and continuous improvement of data workflows in production environments. Mentor Teams: Lead and mentor development teams on secure ADF pipeline development, API management, and compliance with enterprise security standards. Ensure Best Practices: Establish and enforce best practices for data integration, transformation, and … governance within the Azure platform. Required Qualifications Technical Expertise: Extensive experience with Azure Data Factory, including pipeline design, data flow, and integration runtime configuration. Proficiency in Azure services such as Azure Data Lake, Azure Synapse Analytics, Azure SQL Database, and Azure Key Vault. More ❯

Data Engineer - Leading Pharmaceutical Company - Manchester (Tech Stack: Data Engineer, Databricks, Python, Power BI, AWS QuickSight, AWS, TSQL, ETL, Agile Methodologies) About the Role: We are seeking a talented and experienced Data Engineer on behalf of our client, a leading Software House. This is … work with cutting-edge technologies and contribute to exciting projects in a collaborative environment. About the Role: Our client is seeking an experienced Data Engineer to join their team in Manchester. This hybrid position involves working within the pharmaceutical industry, focusing on the design, development, and maintenance of … data pipelines, ETL processes, and databases. The role is ideal for someone passionate about improving processes, ensuring data quality, and maintaining compliance with regulatory standards. focusing on designing, developing, and maintaining data pipelines, ETL processes, and databases. If you are passionate about driving continuous More ❯

A high performing, multi-billion dollar AUM hedge fund are looking to make a newly created Data Analyst hire into their rapidly growing centralised Data Management team. Reporting into the Global Head of Data and taking on a highly visible role within the team … the hire will be working very closely with Operations, Product Control, Risk, Trading and Research teams to ensure the smooth transmission of data, in … a role that has a highly technical/engineer angle. You will optimize the operation and quality management of the internal data pipeline with oversight of raw data initially being onboarded through to the delivery of datapoints to front office/investment teams. You will More ❯

advertised date, so please apply as soon as you can. What you’ll be doing Contribute to the design and execution of the data migration and integration strategy within the Mortgages area, supporting the broader integration programme objectives. Collaborate with internal stakeholders, business analysts, and architects to translate … functional and non-functional requirements into technical data solutions. Design, build and maintain robust, scalable and high-performance ETL/ELT pipelines using modern data engineering tools and practices, ensuring quality, lineage and auditability. Support data discovery activities and analysis to inform data mapping, profiling, and migration planning, especially in the context of mergers or acquisitions. Develop and maintain detailed technical design documents and data flow diagrams aligned with architectural principles and data governance standards. Assist in the implementation and enforcement of best practices in data More ❯

our organisation for the next stage of its journey. To drive towards this ambition, we are seeking a motivated individual to join our Data Platform team and support Aztec's new technology strategy using Azure Databricks. You will lead our Data Engineering capability and collaborate with … others passionate about solving business problems. Key responsibilities: Data Platform Design and Architecture Design, develop, and maintain a high-performing, secure, and scalable data platform, leveraging Databricks Corporate Lakehouse and Medallion Architectures. Utilise our metadata-driven data platform framework combined with advanced cluster management … data architectures, such as Corporate DataLake, Medallion Architecture, Metadata Driven Platform, Event-driven architecture. Proven experience of ETL/ELT, including Lakehouse, Pipeline Design, Batch/Stream processing. Strong working knowledge of programming languages, including Python, SQL, PowerShell, PySpark, Spark SQL. Good working knowledge of data More ❯

collaboration, recognition and inclusivity and offer a wide range of benefits to support staff wellbeing. Your Future Starts Here PURPOSE OF JOB: The Data & AI Solutions Deployment Architect will be instrumental in architecting and deploying Azure Data & AI Solutions. A solid, technical, ‘hands-on’ (not just … grasp of Azure Fabric is essential as we navigate clients through all of the infrastructure, coding, and product development lifecycle challenges around successful Data and AI implementation and deployment. This customer-centric position involves delivering projects, conducting workshops and physically working on client’s environments via Python, and … C#. This role would suit an experienced Data platform engineer or Data Solutions Architect who thrives in working across multiple projects with a diverse range of clients, and who places the importance of bridging the gap between complex data problems and real life product More ❯

Job Title: Senior Data Engineer Salary: up to £65,000 Location: Hybrid – 1 day per fortnight in the Wilmslow offices Working hours 0900 to 1730 The Role Following recent investment, The Citation Group are looking to build out their data capabilities to be Best in Class. … As such, a fantastic opportunity has arisen for a strong data engineer to work in the data engineering team, to expand and maintain the data infrastructure that provides the foundations from which our BI, Data Science and Operational Data teams … insights and revenue to the functions across the wider group. We’re looking for an individual who has experience working with enterprise grade data pipelines and infrastructure, who can implement processes on our modern tech stack with robust, pragmatic solutions. Responsibilities develop, and maintain ETL/ELT data More ❯

Role Title: Lead Data & Insights Engineer Salary: £65,00-£80,000 Location: UK Remote Full-time | Permanent | 37.5 hours/week Apply by: 12/06/2025 Hiring Manager : Phil Walter | Recruiter: Rikki Spencer-Cox Help us “Elevate Education, Everywhere.” Team: Technology & Digital | Etio At Etio, we … We manage both internal and partner delivery streams, bringing resilience to complex systems and driving continuous improvement across live services. The Role: Lead Data & Insights Engineer We're looking for a strategic and delivery-focused data leader to join Etio as our Lead Data … who combines deep technical expertise with strong autonomy and a sharp eye for operational value. This is the perfect opportunity for a senior data professional who wants to own and evolve data architecture across a growing organisation. You’ll act as the primary in-house expert More ❯

Senior Data Engineer Bristol 6-month contract Paying up to £80p/h (Outside IR35) *** Please note that due to the nature of the work you must hold or be eligible to obtain DV level security clearance *** Responsibilities : Design, develop, and maintain secure and scalable data pipelines using the Elastic Stack (Elasticsearch, Logstash, Kibana) and Apache NiFi. Implement data ingestion, transformation, and integration processes, ensuring data quality and security. Collaborate with data architects and security teams to ensure compliance with security policies and data governance standards. … Manage and monitor large-scale data flows in real-time, ensuring system performance, reliability, and data integrity. Develop robust data models to support analytics and reporting within secure environments. Perform troubleshooting, debugging, and performance tuning of data pipelines and the Elastic Stack. More ❯

Job Title: Snowflake Data Architect Job Location: Hove, UK/Hybrid Is it Permanent/Contract: Permanent Job Description Who we are? We are a leading global IT Services company, dedicated to driving digital transformation and innovation for businesses around the world. Founded in 1990, Client has grown … and excellence. We foster a collaborative environment where creativity and continuous learning are encouraged, enabling our employees to thrive and grow. The Opportunity. Data & AI practice is experiencing rapid growth, and we're on the lookout for delivery leaders. Our ideal candidates should be passionate about data … responsibilities in challenging engagements, gaining exposure to clients in a variety of industries across the UK and Europe. If you're a dynamic Data Solution Architect leader who excels in a collaborative team environment and enjoys working with talented individuals, this could be the perfect opportunity for you. More ❯