1 to 25 of 31 Data Ingestion Jobs in the South East

We are seeking a strategic and hands-on Lead Data Engineer to own the development of Aerial Direct’s cloud-native data platform. Working primarily within the Microsoft Azure ecosystem, you will be responsible for building and maintaining scalable data pipelines, optimising architecture for performance, and enabling actionable insights across the business. In addition to technical delivery … you will lead a small team of data engineers and analysts, helping to embed best practices, mentor talent, and deliver enterprise-grade data services that power automation, reporting, and transformation initiatives. You will play a central role in shaping how data is stored, governed, and consumed across the business, ensuring security, scalability, and compliance are built in … Programme offering confidential Wellbeing Support Regular events and activities throughout the year including our famous Christmas Party, summer party & Employee Awards! Company pension scheme You’ll get involved in: Data Strategy, Architecture & Engineering Owning and evolving Aerial Direct’s Azure-based data platform, ensuring performance, security, and scalability. Design, management, and optimisation of data warehouse and data More ❯

Job Description: We are seeking a highly skilled and visionary Director of Data Platforms and Architecture to lead the design, development, and governance of our enterprise-wide Snacking data platform ecosystem. This individual will be responsible for driving platform strategy, fostering innovation, and ensuring robust data architecture that empower multiple products and capabilities to power business decisions … scalability, and operational excellence. As a strategic leader, the Director of Data Platforms and Architecture will oversee the implementation and optimization of cutting-edge data technologies & solutions, establish best practices in data platforms and governance, and provide technical leadership to multidisciplinary teams. You will work closely with stakeholders across business, IT, and analytics teams to align data architecture with organizational goals, enabling seamless data acquisition and advanced analytics capabilities. What will be your key responsibilities? Strategic Leadership: Develop and drive the long-term vision for enterprise data platforms and architecture, ensuring alignment with organizational goals and priorities. Collaborate with C-level executives and business leaders to identify opportunities for leveraging data as a More ❯

and improving in everything they do. We support our clients with incubation services, offering ready-made capabilities supported by smart, execution-focused practitioners. The Role Are you passionate about data and transforming raw information into valuable insights? We’re seeking a talented Fabric/Data Consultant with expertise in Azure and Power BI to join our dynamic team. … This role suits someone who enjoys working with advanced technology, developing robust data architectures, and leading strategic business intelligence initiatives. If you're excited about unlocking the full potential of data for organizations, we want to hear from you. Proven experience in data engineering, with exposure to Azure Synapse, Azure Data Factory, or Databricks. Familiarity with … Microsoft Fabric or a strong willingness to learn and adapt to its ecosystem. Proficiency in Python programming, especially for data manipulation and engineering. A genuine interest in data science and a desire to expand skills into predictive analytics, machine learning, or statistical modeling. Strong analytical and problem-solving skills with a passion for optimizing workflows. Excellent communication and More ❯

My client is based in the London area and is currently looking to recruit an experienced Fabric Data Engineer to join their team. They are one of the leaders within the Insurance space. They are currently going through a period of growth and are looking for an experienced Data Engineer to join their team. They are backed by … implement and utilise the most in-demand and cutting-edge technology on the market right now. Your role will include: Collaborate closely with stakeholders to understand and interpret their data and analytics needs. Design and deploy end-to-end data solutions using Microsoft Fabric, encompassing data ingestion, transformation, and visualisation workflows. Construct and refine data models, pipelines, and ETL frameworks within the Fabric ecosystem. Leverage Fabric's suite of tools to build dynamic reports, dashboards, and analytical applications. Maintain high standards of data integrity, consistency, and system performance across the data environment. Advise on data governance, security protocols, and compliance strategies aligned with industry best practices. Diagnose and resolve technical challenges More ❯

high-ambiguity environment. • Build, train, and deploy state-of-the-art models (e.g., deep learning, NLP, computer vision, reinforcement learning, or relevant domain-specific architectures). • Design infrastructure for data ingestion, annotation, experimentation, model versioning, and monitoring. • Collaborate closely with product, design, and DevOps to integrate AI features into our platform. • Continuously evaluate new research, open-source tools … and grow an AI/ML team as we scale beyond our seed round. ⸻ Key Responsibilities 1. Architecture & Hands-On Development • Define and implement end-to-end AI pipelines: data collection/cleaning, feature engineering, model training, validation, and inference. • Rapidly prototype novel models (e.g., neural networks, probabilistic models) using PyTorch, TensorFlow, JAX, or equivalent. • Productionize models in cloud … availability inference. 2. Strategic Leadership • Develop a multi-quarter AI roadmap aligned with product milestones and fundraising milestones. • Identify and evaluate opportunities for AI-driven competitive advantages (e.g., proprietary data, unique model architectures, transfer/few-shot learning). • Collaborate with business stakeholders to translate “big problems” into technically feasible AI solutions. 3. Data & Infrastructure • Oversee the creation More ❯

Senior Data Engineer - London/Hertfordshire - Hybrid Are you a technically strong Azure Data Engineer looking to join a forward-thinking, innovative company? Here's your chance to work with a leading construction and infrastructure company who foster a culture of collaboration, support, and strong communication. You'll be responsible for ensuring seamless integration and optimization of diverse … business data streams - from CRM, finance, procurement to fleet and commercial data - enabling reliable, high-quality data for analytics and business insights. While this role won't be client-facing, strong communication and collaboration skills within their agile team are essential. Requirements: Experience orchestrating data pipelines with Azure Data Factory (ADF) Proficiency managing API data ingestion and orchestration using Azure Databricks Working with Azure Data Lake (ADL) Storage, on-prem SQL, and Python Vast Azure data engineering experience Proficient in ADF, Databricks, ADL, SQL, Python Desirable: Exposure to D365 or similar CRM environments Azure Synapse experience Power BI knowledge Benefits: Salary: £70,000 - £80,000 depending on experience Hybrid working More ❯

Senior Machine Learning Operations Engineer Location: Guernsey, Haywards Heath, Home Office (Remote) or Manchester Salary: £65,000 - £75,000 - depending on experience Department: Technology and Data We're 1st Central, a market-leading insurance company utilising smart data and technology at pace. Rapid growth has been based on giving our 1.4 million customers exactly what they want: great … for our colleagues too; we won Insurance Employer of the Year at the British Insurance Awards 2024 and our Glassdoor score is pretty mega too! We're big on data: it gives us the insights we need to give the right cover to the right customers at the right price. But it's the people inside and outside our … business that power us and were currently on the hunt for 2 experienced Senior Machine Learning Operation Engineers to join our Data Function. You'll play a significant role within our Data Function, working on the design and implementation of machine learning model engineering frameworks, solutions, and best practices. You'll be technically proficient in machine learning and More ❯

Join a Leading Growth business on its transformative journey to become the UK's most recommended businesses in their industry Lead Data Engineer Salary: £70-90k Location: Reading – Hybrid Benefits: Competitive Role Overview: As Lead Data Engineer , you will be part of the Corporate Data team, reporting directly to the Senior Leadership team. You will lead … a small team of data engineers, overseeing the design, build, and maintenance of the infrastructure and systems that collect, store, and process corporate data for reporting and analysis. Key Responsibilities: • Lead and manage a team of data engineers. • Design, build, and maintain data infrastructure and systems. • Ensure effective use of Microsoft Fabric and Synapse analytics platform. … Collaborate with various stakeholders, including Data and Systems owners, Data Analysts, Data Architects, Infrastructure Architects, and external suppliers. Experience required • Dynamic and hands-on leader who excels in managing and coaching a team. • Thrives in a collaborative environment and adapts to changing priorities with ease. • Excellent communication and presentation skills, capable of conveying complex ideas to both More ❯

Data Engineer - Integration Location: London-based (Hybrid - 1 day per week in the office) Contract: FTC - 23 months £56,500 per annum Overview: We are currently recruiting on behalf of a well-established nationwide charity for a Data Engineer - Integration . This is an exciting opportunity to play a key role in designing and developing data products … and services that improve how data is accessed and used across the organisation. You'll help ensure data is accurate, timely, and fit for purpose, supporting a wide range of departments and strategic goals. Role Summary: As a Data Engineer, you will work to connect systems, streamline business processes, and optimise data warehouse platforms. You'll … collaborate with stakeholders to deliver bespoke BI solutions, ranging from custom queries to dashboards and reports. Your work will transform raw data into actionable insights that drive the charity's mission forward.In addition to building and maintaining data integration pipelines, you'll help ensure the ongoing performance and reliability of the charity's core data platforms, which More ❯

Business Intelligence Farnborough 27.06.2025 URL of this page was copied to paste buffer Data Analyst INNOVATION IS IMAGINING WHAT NOONE ELSE CAN. SHARE YOUR PASSION. At the BMW Group, everything starts with passion. It turns a profession into a vocation. It drives us to keep reinventing mobility and bring innovative ideas onto the roads. Enthusiasm for joint projects turns … It is only when expertise, highly professional processes and enjoyment of work unite that we can shape the future together. BMW Group Financial Services UK is looking for a Data Analyst. This role is based at the Summit ONE Campus in Farnborough. What awaits you? Analyse data from transactional systems to understand the generation and transformation of data, and its relationship to business processes and models. Develop a detailed understanding of transactional system data at the attribute level to prepare for effective transformation into a structure suitable for analytics. Identify and quantify business stakeholders' requirements which we can convert to re-usable Data Products. Design and implement suitable database schemas to be integrated into our More ❯

Because we're breaking new ground, it often means that we're working on new technological challenges. We are building in-house banking products using modern technologies. Role Description Data is a key asset to Allica and is deeply embedded in all our teams and products. With data, we assist and work with Finance, Operations, Credit Risk, Product … and Management. We are looking for ambitious and curious people who can help us understand our data better, contribute to building robust data pipelines, and help model our data so that stakeholders and team members can extract better insights from it. You will be an integral part of the data engineering team, building and developing the … tooling and pipelines that form our data ingestion and modelling machine for our partnership agreements. Principal Accountabilities Understand our data landscape and architecture. Strong skillset in data modelling. Build and maintain data pipelines. Work with the team to engineer tools based on data requirements, manage development from prototyping to productioniSing. Identify, design, and implement More ❯

Social network you want to login/join with: At Travelex we are developing modern data technology and data products. Data is central to the way we define and sell our foreign currency exchange products. Our relationship with our customers is deeply data-driven. The data engineering manager ( DEM ) owns the design and delivery of … all data flows to and from our data platform, underpinning crucial data products. The DEM is responsible for a significant transformation Travelex is going through, with event-driven and transactional enhancements to our enterprise data architecture, alongside expansion of our data warehousing function to support a wide range of data integrations. Main Responsibilities: Work … with the Director of Data Engineering and IT Leadership to enhance the company's data maturity. Maintain strong business alignment by engaging with leaders across domains and geographies. Ensure data engineering initiatives match evolving business priorities. Promote a product mindset, balancing technical efficiency with clear business value. Team Management Lead the Data Engineering team, providing supervision More ❯

Social network you want to login/join with: At Travelex we are developing modern data technology and data products. Data is central to the way we define and sell our foreign currency exchange products. Our relationship with our customers is deeply data-driven. The data engineering manager ( DEM ) owns the design and delivery of … all data flows to and from our data platform, underpinning crucial data products. The DEM is responsible for a significant transformation Travelex is going through, with event-driven and transactional enhancements to our enterprise data architecture, alongside expansion of our data warehousing function to support a wide range of data integrations. Main Responsibilities: Work … with the Director of Data Engineering and IT Leadership to enhance the company's data maturity. Maintain strong business alignment by engaging with leaders across domains and geographies. Ensure data engineering initiatives match evolving business priorities. Promote a product mindset, balancing technical efficiency with clear business value. Team Management Lead the Data Engineering team, providing supervision More ❯

Social network you want to login/join with: Search 5.0 is delighted to partner with an innovative AI tech start-up in the financial data sector, seeking a Rust Developer to join their exciting journey! Our client equips traders with tools that are faster, smarter, and easier to use. This is a fantastic opportunity to join a leading … AI company whose Machine Learning product suite creates new trading opportunities through real-time analytics, supported by a world-class, scalable, ultra-low latency architecture. The Data Platform is crucial to all capabilities and services, enabling data ingestion, analysis, querying, and storage at near-real-time speeds, setting our client apart from competitors. While most companies process … data in days, hours, or minutes, our client operates at nano to millisecond scales, a speed typically reserved for stock market trading systems. Job: Rust Developer Joining an incredible, small tribe you will play a pivotal role in driving the technical direction ensuring requirements are met and building a scalable product. Develop and maintain the platform services that underpin More ❯

Search 5.0 is delighted to partner with an innovative AI tech start-up in the financial data sector, seeking a Rust Developer to join their exciting journey! Our client equips traders with tools that are faster, smarter, and easier to use. This is a fantastic opportunity to join a leading AI company whose Machine Learning product suite creates new … trading opportunities through real-time analytics, supported by a world-class, scalable, ultra-low latency architecture. The Data Platform is crucial to all capabilities and services, enabling data ingestion, analysis, querying, and storage at near-real-time speeds, setting our client apart from competitors. While most companies process data in days, hours, or minutes, our client … driving the technical direction ensuring requirements are met and building a scalable product. Develop and maintain the platform services that underpin our products, in tandem with SME’s in data science and quantitive finance. Contribute new products and technical ideas, providing feedback and collaboration sessions. Work closely with QA to ensure that systems are engineered accordingly What we’re More ❯

development of scalable trading platforms and internal developer tools. Collaborate with quant researchers, traders, and infrastructure teams to deliver performant solutions. Maintain and optimize trading infrastructure (low-latency systems, data pipelines, monitoring, etc.). Manage a team of software engineers, mentor junior developers, and drive best practices across the stack. Ensure high code quality through code reviews, CI/… workflows. Own deployment pipelines across cloud providers (AWS, GCP, Azure) using containerization (Docker, Kubernetes). Design and implement microservices and APIs for internal tools and external integrations. Work with data at scale (market data ingestion, real-time analytics, historical storage). Required Skills & Experience: Core Engineering: 5+ years of experience in software development, with exposure to both … and storage configurations. Familiarity with monitoring/logging tools (e.g., Prometheus, Grafana, ELK stack). Trading Systems & Finance: Solid understanding of trading infrastructure, latency optimization, execution systems, and market data feeds. Experience working in or with quantitative research, HFT, or hedge fund teams is highly desirable. Leadership: Proven experience managing and mentoring a team of 3–10 engineers. Ability More ❯

Senior Data Engineer - £82,000 - Hybrid & Remote Flexibility No Sponsorship Available Company We are a forward-thinking technology company dedicated to simplifying complex processes through innovative solutions. With a focus on enhancing efficiency and transparency, we strive to deliver outstanding experiences for our customers. Our team operates with a flexible hybrid working model, fostering collaboration and innovation in a … supportive environment. Responsibilities As a Senior Data Engineer, you'll play a pivotal role in advancing our data infrastructure to support our expanding scope and responsibilities. Key responsibilities include: Enhancing the reliability and stability of our data infrastructure. Managing data ingestion pipelines using tools such as Azure Data Factory (ADF) and Python. Ensuring the … quality of raw datasets to empower Data Analysts in creating robust data models. Deploying and managing data tools on Kubernetes (Airflow, Superset, RStudio Connect). Supporting Data Analytics through the management of DBT, DevOps, and deployment rules. You will have the opportunity to work end-to-end, making meaningful contributions within a small, agile team. Experience More ❯

Job Title: Data Engineer (NPPV3 + SC Clearance Required)Location: Hybrid (UK-based)Contract: 3 Months (with potential extension/move to other projects)Rate: £550 per day (Outside IR35)A bout the Role:We are seeking an experienced Data Engineer with NPPV3 + SC clearance to join our dynamic team on a 3-month contract basis. This … role offering the flexibility of working both remotely and onsite. The contract has potential for extension or transition onto other exciting projects. Key Responsibilities:* Design, build, and maintain scalable data pipelines using Azure Data Factory, Databricks, and Azure Synapse.* Collaborate with cross-functional teams to ensure data quality and integration.* Support data ingestion, transformation, and … storage in cloud environments.* Troubleshoot and optimise data workflows for performance and reliability.* Maintain compliance with security protocols and clearance requirements. Essential Skills & Experience:* Must hold NPPV3 + SC clearance (this is a mandatory requirement).* Proven expertise in Azure Data Factory, Databricks, and Azure Synapse Analytics.* Strong experience in building and managing cloud-based data solutions. More ❯

poster from Digital Waffle Tech Recruitment Manager (DevOps, Cloud & Infrastructure) for the UK & US . Get in touch to find out how I can solve your hiring problems! Senior Data Insights Analyst Data-Driven Strategy | Revenue Optimisation | Business Impact PowerBI SQL VBA Are you an intellectually curious, highly motivated analyst with a passion for turning data into … real business value? We’re on the hunt for a Senior Data Insights Analyst to join our fast-growing team in Oxford — someone ready to roll up their sleeves and make a meaningful impact from day one. This role sits at the core of our business , where data isn’t just a resource — it’s the engine powering … our product innovation, customer experience, and revenue growth. About the Role As part of our high-performing Data Insight Team , you’ll be working with large datasets to extract critical trends, uncover growth opportunities, and drive fast-paced, data-informed decision-making. You'll influence key business strategies, using insights to directly shape product performance and customer satisfaction. More ❯

Data Pipeline Development: Design and implement end-to-end data pipelines in Azure Databricks, handling ingestion from various data sources, performing complex transformations, and publishing data to Azure Data Lake or other storage services. Write efficient and standardized Spark SQL and PySpark code for data transformations, ensuring data integrity and accuracy across … the pipeline. Automate pipeline orchestration using Databricks Workflows or integration with external tools (e.g., Apache Airflow, Azure Data Factory). Data Ingestion & Transformation: Build scalable data ingestion processes to handle structured, semi-structured, and unstructured data from various sources (APIs, databases, file systems). Implement data transformation logic using Spark, ensuring data is cleaned, transformed, and enriched according to business requirements. Leverage Databricks features such as Delta Lake to manage and track changes to data, enabling better versioning and performance for incremental data loads. Data Publishing & Integration: Publish clean, transformed data to Azure Data Lake or other cloud storage solutions for consumption by analytics and reporting More ❯

a dynamic self-starter to join their innovative team in the heart of London. If you thrive in a fast-paced environment and want to contribute to cutting-edge data science applications, this is the perfect opportunity for you! About Client: Our client is a leading global payments technology company that enables businesses of all sizes to accept payments … geographies. They handle billions of transactions annually, offering secure, scalable solutions that support innovation in the financial ecosystem. With deep industry expertise, they empower merchants to grow through advanced data insights and reliable infrastructure. What You'll Do : Design, develop, and deploy machine learning models that solve real-world problems. Collaborate with cross-functional teams to improve production systems … and ensure seamless integration of ML models. Drive the ML life cycle from data ingestion to model deployment and monitoring. Implement CI/CD pipelines and leverage DevOps practices to enhance operational efficiency. What You Need : A relevant BSc/MSc degree in a related field with strong industry experience. Proven expertise in ML engineering and MLOps, with More ❯

Social network you want to login/join with: We’re seeking a proactive Trading Developer to architect and enhance the data and process backbone of our trading environment. You’ll design and implement backend APIs, real-time data pipelines, and monitoring frameworks—ensuring reliable, low-latency access to market and operational data. While your core focus is … tasks and responsibilities API & Service Development Design, develop, and maintain robust internal RESTful and event-driven APIs. Establish best practices for versioning, documentation, security, and monitoring. Build and manage data streaming pipelines using frameworks such as Spark, Kafka, or equivalents. Ensure high-availability and low-latency delivery of market and industry data. Deploy and operate services on Azure using … Functions, App Services, AKS (Kubernetes), and related services. Data Ingestion & Management Ingest diverse data sources (exchange feeds, third-party APIs, FIX/WebSocket interfaces). Implement ETL processes and data models to drive downstream analytics and trading strategies. Uptime & Monitoring Implement health checks, alerting, and observability (metrics, logs, tracing) to maintain 24/7 uptime. Rapidly More ❯

Data Engineer (Snowflake) Hybrid working pattern 2 days per week in the London office My client, a large global Insurance company, are seeking a talented AWS Data Engineer with Snowflake experience to join their growing data practice within the UK. As a key member of this team, you will play a pivotal role in designing and implementing … data warehousing solutions using Snowflake and AWS. The ideal candidate must have: Strong experience as an AWS Data Engineer Lead the design and implementation of Snowflake [and Redshift] based data warehousing solutions within an AWS environment Extensive experience in AWS services, e.g. EC2, S3, RDS, DynamoDB, Redshift, Lambda, API Gateway Develop and optimize ETL processes using AWS … services (e.g. AWS Glue, Lambda) to ensure efficient data ingestion, transformation, storage, and cost optimization. Strong SQL skills for complex data queries and transformations ETL Pipe-lining experience Experience with Python AWS and Snowflake Certifications are a plus Insurance experience or Financial services experience advantageous Must have experience of working closely with business teams and communicating with More ❯

This is an exciting opportunity for a Power BI Specialist. Based in London, you will use your expertise to create insightful dashboards and reports that support data-driven decision-making. Client Details A £100million turnover company. Description Design, build, and refine dashboards focusing on profit and loss, margin analysis, cost control, and operational KPIs across the Group. Enhance existing … dashboards to integrate with the new Microsoft Data Links repository, replacing scheduled refreshes with live data feeds. Use Power Query (M language) for data ingestion and transformation. Develop advanced DAX measures, including time intelligence calculations, dynamic reporting layers, and calculated tables. Manage the full dashboard development lifecycle, from stakeholder engagement and data modelling to prototyping … business units to gather requirements and shape outputs. Ensure dashboards are user-friendly, commercially relevant, and actionable for both finance and non-finance users. Work alongside internal ERP and data teams (who manage SQL and connectors) to maintain data consistency with Unit4 and Tagetik systems. Provide ad-hoc support for wider reporting and transformation projects, including benchmarking and More ❯

management Skills and Experience AI/ML Expertise: Strong understanding of LLMs, NLP, and AI-driven recommendations. Technical Leadership: Experience in AI product scalability, DevOps, and cloud architecture. Backend & Data Skills: Proficiency in Python, Node.js, PostgreSQL, MongoDB, API security. AI Model Deployment: MLOps, scaling, data engineering, and AI ethics awareness. Strategic Mindset: Ability to align technology with business … objectives and cost efficiency. Security & Compliance Knowledge: GDPR, API authentication, and observability best practices. Big data processing: Understanding data lakes, warehouses, and tools like Spark, Kafka, and Airflow. ETL Pipelines: Ability to evaluate data ingestion, transformation, and cleaning processes. DevOps & CI/CD: Hands-on knowledge of Jenkins, GitHub Actions, Terraform, and CloudFormation. What's in More ❯