21 Data Pipeline Jobs in the South East

We are searching for an experienced and talented Head of Data Engineering for a critical role working for a successful and expanding business, a company that is a market leader in its field, producing award-winning software solutions and services for a diverse portfolio of customers. The role … UK without company sponsorship. In this role you will be responsible for the following: - Creating and managing a complex service model that delivers data timely and accurately to a wide variety of stakeholders and downstream consuming applications and services. Defining and managing the cost model for the Data Platform and the staff who work on the platform. Working hand-in-hand with Data Operations and Data Insight teams to build key data pipelines and platform processes, deploying models and implementing orchestrations. Implementing governance models for data access, data more »

Description Due to continued growth we are currently looking for Senior Data Engineer to join our Professional Services division - you will be part of a cross functional Data consulting team spanning data engineering, visualisation, analytics and science. You will be working with our clients … across multiple sectors, helping them to explore next-generation data techniques and tools to drive incremental business value from their data assets. A day in the life of an Aiimi Senior Data Engineer: You will be responsible for collaborating with business subject matter experts … to discover the information hidden in various sources of content and data, helping our clients make smarter decisions to reduce service failure and deliver better outcomes to their customer base Your primary focus will be in connecting to and extracting data from source systems, applying business more »

Data Engineer Remote working, West Sussex, Manchester Up to £54,000 per annum + Bonus An exciting opportunity has arisen to join a leading company in the Insurance and technology sector, renown for its outstanding working culture, in a Data Engineering role. The role will require … who has strong Pyspark and SQL experience alongside an Azure background. Key responsibilities: Develop low to high complexity, secure, governed, high quality, efficient data pipelinesfrom a variety of on and off premise, internal and external data sources Ensure that data is cleansed, mapped, transformed … and optimised for storage to meet requirements for business and technical use cases. Build data observability and data quality by design into all Data pipelines, promoting self-testing pipelines that proactively identify processing issues or discrepancies. Build solutions that pipe transform data more »

using your expertise to uncover valuable insights that drive strategic decision-making and fuel business growth? Do you thrive in a high volume data-oriented environment, where every record holds the potential to uncover radical insights and opportunities? Do you have exceptional skills in big data technologies, real-time data processing, and advanced analytics, and would be capable of handling and extracting insights from a highly dynamic dataset growing by 8 billion records each day? If youre a Big Data Analyst used to working with big data and … high-volume records, and have what it takes to transform complex data into actionable recommendations, informed decisions, and impactful outcomes, then this could be the perfect position for you! Salary & Benefits: Competitive salary between £60,000 - £80,000 per annum depending on relevant technical skills and experience Fully more »

My Prop-Fintech client is looking for a Data Engineer to join the Head of Data and 2 data analysts in the building, extraction, and refining of new and innovative datasets. This Geospatial Dataset contains billions of datapoints and has infinite possibilities of growth. … bonus, private healthcare for you and the family, Gym Benefits, and home office equipment. KEY RESPONSIBILITY Design and uphold an efficient data pipeline architecture. Enhance data storage platforms for optimal speed, scalability, reliability, and performance. Oversee critical functions such as data acquisition, extraction … preparation, and processing of extensive datasets for predictive modelling purposes. Conduct web scraping and proficiently manipulate data as required. Take the lead in pioneering modelling techniques utilising emerging technologies. Collaborate closely with data subject specialists to enhance data functionality and accessibility. Provide technical support more »

change the status quo, we’ll help you make your moment. Join Team Purple. Join IFS. Job Description As a Principal/Senior Data Engineer, you will spearhead our data initiatives, transforming raw data into strategic assets that propel our business forward. This role … who excel in a dynamic environment, offering the freedom to innovate and the responsibility to deliver. Core Responsibilities: Identify and harness high-value data opportunities within our product suite, converting raw data into impactful features and reusable assets. Act as the authoritative voice on data … Strong programming skills in Python, Scala, C#, or Java, and adeptness with cloud SDKs and APIs. Deep understanding of AI/ML to enhance pipeline efficiency, with experience in TensorFlow, PyTorch, AutoML, Python/R, and MLOps platforms like MLflow and Kubeflow. Solid background in DevOps , including CI/ more »

Senior Data Engineer Location: Hybrid, with 2 days per month to be onsite (Abingdon) We have the pleasure to be representing a business that is on a mission to increase the number of people who can live longer, healthier lives and reduce unnecessary medical costs while doing so. … To help with the success of this strategy They are recruiting for a Senior Data Engineer that will take a pivotal role in driving the Data team forward. Responsibilities: Lead Cloud Data Warehouse Project : Take charge of leading the Cloud data warehouse … project through its next phase of development, ensuring its scalability, reliability, and performance meet the highest standards. Ownership of Data Management : Assume ownership of all aspects of data management, serving as the go-to person for any data queries from the internal team. Your more »

people just like you! About the Role We are DataOps advocates and use software engineering best practices to build scalable and re-usable data solutions to help clients use their data to gain insights, drive decisions and deliver business value. Clients don’t engage BJSS to … which means we get to work with a wide range of tools and technologies and there are always new things to learn. BJSS data engineers are specialist software engineers that build, optimise and maintain data applications, systems and services. This role combines the discipline of software … engineering with the knowledge and experience of building data solutions in order to deliver business value. As a BJSS data engineer you’ll help our clients deploy data pipelines and processes in a production-safe manner, using the latest technologies and with a DataOps more »

people just like you! About the Role We are DataOps advocates and use software engineering best practices to build scalable and re-usable data solutions to help clients use their data to gain insights, drive decisions and deliver business value. Clients don’t engage BJSS to … which means we get to work with a wide range of tools and technologies and there are always new things to learn. BJSS data engineers are specialist software engineers that build, optimise and maintain data applications, systems and services. This role combines the discipline of software … engineering with the knowledge and experience of building data solutions in order to deliver business value. As a BJSS data engineer you’ll help our clients deploy data pipelines and processes in a production-safe manner, using the latest technologies and with a DataOps more »

to enhance their technical skills, fostering a culture of continuous learning and professional growth. Collaborate closely with cross-functional teams , including ML Engineers, Data Scientists, and Product Managers, to architect, refine, and deploy sophisticated AI solutions. Act as an ambassador of innovation , delivering keynote speeches, hosting webinars, and … authoring insightful publications that influence the industry and drive transformation. Qualifications A robust foundation in computer science , emphasizing AI and Data infrastructure design. A minimum of 10 years’ experience as an engineer, designing and executing large-scale solutions in cloud environments, ensuring seamless integration with legacy systems. S … model development, deployment, and mastery of MLOps practices. Experienced in crafting enterprise-scale cloud-native AI/ML solutions Proficiency in data pipeline orchestration across hybrid environments, leveraging the latest in Azure and allied technologies. Expertise in data processing with tools like Spark or Dask more »

Xpertise is seeking a series of Data Engineers with cloud experience to join a fledgling team in Reading. As part of our client's growing engineering division, you will play a pivotal role in leading the data engineering capabilities, working closely with Platform Engineers, Developers, and … and even venture into the world of MLOps/AIOps. Key experience desired/what you will learn: Design, develop, and maintain scalable data pipelines and ETL processes leveraging GCP, AWS, and/or Azure services Cloud Database Management: BigQuery, Snowflake, Databricks, and Redshift Architect data … practices to ensure compliance and data integrity. Monitor and troubleshoot data pipelines, ensuring high availability and reliability. CI/CD pipeline automation utilising GitLab DevOps: GKE (Kubernetes), Terraform, Ansible Leadership capabilities: mentorship, management Role overview: If you're eager to collaborate with a driven team more »

Xpertise is seeking a series of Data Engineers with cloud experience to join a fledgling team in Milton Keynes. As part of our client's growing engineering division, you will play a pivotal role in leading the data engineering capabilities, working closely with Platform Engineers, Developers … and even venture into the world of MLOps/AIOps. Key experience desired/what you will learn: Design, develop, and maintain scalable data pipelines and ETL processes leveraging GCP services such as BigQuery, Dataflow, and Cloud Storage. If you have experience across AWS and Azure, we can … practices to ensure compliance and data integrity. Monitor and troubleshoot data pipelines, ensuring high availability and reliability. CI/CD pipeline automation utilising GitLab DevOps: GKE (Kubernetes), Terraform, Ansible Leadership capabilities: mentorship, management Role overview: If you're eager to collaborate with a driven team more »

love to see: Proficiency in developing ML models with Python or TensorFlow Familiarity with NLP and AI technologies, including LLMs Experience building robust data pipelines Experienced with DevOps practices, including CI/CD, TDD, SDLC Cloud experience – Azure or GCP This is an opportunity to take charge of more »

team. This team is responsible for and enhancing the business’s lead management and CRM capabilities BI analysts with accurate and relevant application data business insights and predictive models developed by the BI team into self-optimising, productionised systems within the wider business. The ideal candidate will be … responsible for developing and maintaining data pipelines and queries to extract complex datasets needed for our business intelligence and reporting needs. Additionally, the candidate will be involved in writing production-level code to support the organisation’s various CRM and lead-management applications and systems. Key Responsibilities: 1. … Data Extraction and Transformation: - Design and develop data pipelines and queries, for efficient data extraction and transformation. - Collaborate with cross-functional teams to understand data requirements and ensure accurate and timely delivery of BI solutions. 2. Business Intelligence: - Work closely with business more »

Python or TensorFlow ) Familiarity with natural language processing or AI technologies ( e.g LLMs such as ChatGPT or BARD , Embeddings , Prompt Engineering ) Experience with data pipelines and preferably processing of document s ( e.g using Google Cloud Fusion/Azure Data Factory ) Strong background in either Python/… Service Orientated Architecture, API-Centric Design, Twelve-Factor App, FAIR, etc.). Experience supporting digital platforms, including Integrations, Release Management, Regression Testing, Integrations, Data Obfuscation, etc. Knowledge of Azure Data Factory/GCP Cloud Data Fusion , Microsoft Azure Machine Learning or GCP Cloud ML … Engine , Azure Data Lake , Azure Databricks or GCP Cloud Dataproc . Familiarity with big data technologies and distributed computing frameworks, such as Hadoop, Spark, or Apache Flink. Experience scaling an “API-Ecosystem ”, designing, and implementing “API-First” integration patterns. Experience working with authentication and authorisation protocols more »

. Familiarity with natural language processing or AI technologies ( e.g LLMs such as ChatGPT or BARD, Embeddings, Prompt Engineering ) Experience with machine learning, data pipelines and preferably processing of document s (e.g using Google Cloud Fusion/Azure Data Factory ) Experience in new database technologies such … as vector databases advantageous Strong background in either Python/Typescript Experience supporting digital platforms, including Integrations, Release Management, Regression Testing, Integrations, Data Obfuscation, etc. Experience scaling an “API-Ecosystem”, designing, and implementing “API-First” integration patterns. Experience working with authentication and authorisation protocols/patterns. Other Information more »

. Familiarity with natural language processing or AI technologies (e.g LLMs such as ChatGPT or BARD, Embeddings, Prompt Engineering) Experience with machine learning, data pipelines and preferably processing of documents (e.g using Google Cloud Fusion/Azure Data Factory) Experience in new database technologies such as … automation and generative AI applications.(e.g SpaCy, NLTK, or Hugging Face). Experience supporting digital platforms, including Integrations, Release Management, Regression Testing, Integrations, Data Obfuscation, etc. Experience scaling an “API-Ecosystem”, designing, and implementing “API-First” integration patterns. Experience working with authentication and authorisation protocols/patterns. Other more »

be able to demonstrate strong capabilities in Azure DevOps methodologies with the ability to implement CI/CD pipelines in collaboration with the data engineering team. Core responsibilities: Lead the implementation and maintenance of CI/CD pipelines for data projects, ensuring efficiency and scalability. Collaborate … for process improvement and implement solutions to streamline development workflows. Monitor system performance and implement optimizations to ensure the reliability and availability of data pipelines and services. Collaborate with cross-functional teams to troubleshoot issues, implement fixes, and ensure timely resolution of technical challenges. Stay up-to-date … with industry trends, emerging technologies, and best practices in DevOps and data engineering. Document processes, configurations, and best practices to ensure knowledge sharing and maintainability of data infrastructure and pipelines. Key technical skill-set: Proven leadership capabilities in regards to project implementation coupled with the ability more »

My client is hiring for a Data Architect on a permanent basis. This role is working with Defence and does require SC/DC clearance, it is essential. This role is hybrid - 2 days in the office and 3 days working from home. They have 2 locations, one … will be driven by solving varied and interesting client problems. You will be responsible for the strategy, architecture and high-level design of data intensive solutions, and will also support Client Managers directly in a pre-sales capacity to develop relationships and solutions that can support future business … growth. Skills: Experience of building specifying Data Pipelines in a Cloud environment (Azure, AWS, GCP) Knowledge of commercial and contractual frameworks SC/DV clearance is essential for this role Responsibilities include: Help Defence customers develop their strategy for data as a strategic asset and define more »

My client is hiring for a Data Architect on a permanent basis. This role is working with Defence and does require SC/DC clearance, it is essential. This role is hybrid - 2 days in the office and 3 days working from home. They have 2 locations, one … will be driven by solving varied and interesting client problems. You will be responsible for the strategy, architecture and high-level design of data intensive solutions, and will also support Client Managers directly in a pre-sales capacity to develop relationships and solutions that can support future business … growth. Skills: - Experience of building specifying Data Pipelines in a Cloud environment (Azure, AWS, GCP) - Knowledge of commercial and contractual frameworks - SC/DV clearance is essential for this role Responsibilities include: - Help Defence customers develop their strategy for data as a strategic asset and define more »