51 to 72 of 72 Data Ingestion Jobs in the UK excluding London

Junior–Mid Software Engineer (Data Pipelines) 📍 Onsite – London | Full-time We’re helping one of our partner — a seriously exciting AI company started by a team of top scientists — find a Software Engineer to join their data team. They’re building powerful platforms that help organisations make smarter … decisions, and the data ingestion & pipelines team plays a huge part in that. You’d be working on real infrastructure, solving real problems, and making an actual impact from day one. What you’ll be up to: Building and optimising data pipelines (think ETL, scraping, ingestion) Keeping data accurate, clean, and flowing smoothly Collaborating with engineers, analysts, and data scientists Playing around with modern tools, cloud platforms, and a very smart team You might be a good fit if you: Have 1–3 years of experience writing software (Python, Java, Scala — whatever works More ❯

We are currently working with a leading, private equity-backed music rights investor that is seeking a Data Engineer to join their growing team. The data team builds, owns, and maintains proprietary data assets that support the full data lifecycle — from raw data ingestion … pipelines. The team plays a critical role across the business. Responsibilities: Develop, maintain, and optimise robust ETL/ELT pipelines in Python Collaborate with data teams to translate business requirements into scalable data solutions Work cross-functionally to support analytics, reporting, and product development Write clean, maintainable, and … testable code with a strong emphasis on clarity and structure Contribute ideas and insights to help evolve the client’s data platform for scale and efficiency To be successful in this role, you will have: A minimum of 2 years' of relevant experience Proficiency in Python Experience building ETL More ❯

We are looking for a Lead Data Engineer to take on a leadership role within our client's Data Engineering team, driving technical excellence and transforming their data ingestion processes. You will be responsible for implementing Data Engineering strategy, working closely with the Head of … Data Engineering to define best practices for the team whilst also working closely with the Project team to ensure technical specifications and tasks are clear for all team members. Responsibilities Lead data engineering projects, provide guidance & oversight to junior + mid-level data engineers, and establish best … practices to improve the software development life-cycle Work closely with the Head of Data Engineering to ensure the data engineering strategy is implemented by the team Help define clear testing frameworks for the team to follow for validating/transforming synthetic data Define best practices for More ❯

Job Title: Elastic Data Engineer Location: Birmingham 3 days per week. We are looking for a talented Elastic Data Engineer to help build and maintain scalable data solutions. Key Responsibilities: Design, develop, and maintain ElasticSearch clusters. Implement data ingestion pipelines using ETL tools. Optimise search … and indexing performance. Collaborate with seniors and analysts to support data-driven projects. Ensure data security and compliance. Qualifications: Proven experience with ElasticSearch and data engineering. Proficiency in programming languages. Strong understanding of ETL processes and data modelling. Excellent problem-solving skills and attention to detail. More ❯

Data Engineer (Python Spark SQL) *Newcastle Onsite* to £70k Do you have a first class education combined with Data Engineering skills? You could be progressing your career at a start-up Investment Management firm that have secure backing, an established Hedge Fund client as a partner and massive … growth potential. As a Data Engineer you'll be instrumental in transitioning the core data platform to Azure, managing 3rd party data ingestion and integration. You'll collaborate and be supported by a small team, seeking ways to improve the data quality and introduce event … top tier university, Computer Science or similar technical/scientific discipline, backed by minimum A A B grades at A-level You have commercial Data Engineering experience working with technologies such as SQL, Apache Spark and Python including PySpark and Pandas You have a good understanding of modern data More ❯

We are seeking an experienced Kafka Real-Time Architect to design and implement scalable, high-performance real-time data processing systems leveraging Apache Kafka. In this role, you will be responsible for architecting and managing Kafka clusters, ensuring system scalability and availability, and integrating Kafka with various data processing platforms. This position requires a deep understanding of Kafka and related technologies, along with a proactive approach to addressing business data needs and ensuring optimal system performance. Key Responsibilities: Design & Architecture: Architect and design scalable, real-time streaming systems using Apache Kafka, ensuring they are robust, highly … available, and meet business requirements for data ingestion, processing, and real-time analytics. Kafka Cluster Management: Configure, deploy, and maintain highly available Kafka clusters, ensuring scalability and fault tolerance across distributed environments. Manage configurations, optimize cluster performance, and troubleshoot issues to maintain smooth operations. Integration & Data Processing More ❯

for the role. Full Job Description About the role: The primary responsibility of this role is to integrate log sources into Sentinel, using standard data connectors, troubleshooting and enhancing data connectors, developing custom connectors where required and optimising log ingestion. The key responsibilities are: Responsible for end-to … test and advise clients on audit configuration settings for log sources, to ensure that the right logs flow into Sentinel for threat detection. Deploy data connectors and troubleshoot data ingestion, including deployment of Function Apps, customisation and enhancement of Function App code where required, and development of … custom log ingestion solutions. Validation of log parsing, fixing and enhancing existing parsers, and development of new parsers. Optimisation of collected logs to ensure the right events are collected and unnecessary events are filtered out to manage consumption and cost. Documentation of solution design, and development of technical processes More ❯

and internal developer tools. Collaborate with quant researchers, traders, and infrastructure teams to deliver performant solutions. Maintain and optimize trading infrastructure (low-latency systems, data pipelines, monitoring, etc.). Manage a team of software engineers, mentor junior developers, and drive best practices across the stack. Ensure high code quality … cloud providers (AWS, GCP, Azure) using containerization (Docker, Kubernetes). Design and implement microservices and APIs for internal tools and external integrations. Work with data at scale (market data ingestion, real-time analytics, historical storage). Required Skills & Experience: Core Engineering: 5+ years of experience in software … monitoring/logging tools (e.g., Prometheus, Grafana, ELK stack). Trading Systems & Finance: Solid understanding of trading infrastructure , latency optimization , execution systems , and market data feeds . Experience working in or with quantitative research, HFT, or hedge fund teams is highly desirable. Leadership: Proven experience managing and mentoring a More ❯

GBP/Day (Inside IR35) Role Description: A Security Solutions Architect + GenAI would design and implement secure AI/ML solutions, ensuring data protection and compliance while leveraging AI's capabilities. This role combines expertise in cybersecurity, AI/ML, and cloud technologies to architect secure and scalable … AI solutions. Responsibilities : Security Architecture: Design and implement security solutions for AI/ML applications, focusing on data privacy, confidentiality, and integrity. GenAI Solution Design: Develop and deploy AI-powered solutions, integrating security measures throughout the life cycle, from data ingestion to model deployment. Skills and Qualifications … understanding of cybersecurity principles, threat modelling, vulnerability assessment, and penetration testing. Cloud Computing: Familiarity with cloud platforms (AWS, Azure, GCP) and their security features. Data Privacy and Compliance: Knowledge of data privacy regulations (GDPR, CCPA) and industry standards. Problem-solving and Analytical Skills: Ability to identify, analyze, and More ❯

The Role We are seeking an experienced Software Development Manager with extensive expertise in AI, Data Engineering, and Architecture. You will lead the development of scalable, AI-driven solutions, combining deep technical knowledge of AWS Cloud Services, MLOps, and core Data Science concepts with proven leadership and strong … up for quality for our customers then we want to talk to you. Key responsibilities Lead engineering teams to deliver high-quality AI and Data solutions. Define and oversee solution and technical architecture decisions. Drive the technical vision, including the selection of appropriate technologies, frameworks, and architectural patterns. Review … designs, providing feedback to Product Management and Architects. Oversee the end-to-end lifecycle of AI projects including data ingestion, ML model development, deployment, and monitoring. Implement and refine MLOps processes, ensuring robust and scalable delivery of models into production. Plan, initiate, manage, and monitor technical projects from More ❯

London-based tech company helping clients engage in marketing strategies with audiences in virtual worlds. Your Focus: As a Senior Backend Engineer focused on Data Pipelines and API Optimisation, you’ll be central to our platform's performance and scalability. You’ll design, build, and iterate on the systems … that handle data ingestion, transformation, and delivery - ensuring they are scalable, reliable, and performant. You should have a strong background in backend engineering, particularly around distributed systems, database optimisation, and API development. You’ll work with technologies like JavaScript and TypeScript to build robust services that meet high … standards for performance and reliability. This role is ideal for someone experienced, self-driven, and excited about leading backend projects involving large-scale data handling and API improvements. Your technical leadership will help evolve our infrastructure and mentor other engineers as we scale. Skills, Knowledge, and Expertise Experience 5+ More ❯

AI platform in their market. They help their clients service customers with unmatched speed and accuracy. The Opportunity: There're looking for a Senior Data Scientist with deep experience in computer vision to join a high-performance team. Youll help push the boundaries of visual defect detection and predictive … optimising workflows in Azure Machine Learning (AML) for both dev and prod Develop and execute best practices for experimentation enabling the team to make data informed decisions Contributing to the end-to-end AI pipeline from data ingestion to production deployment Were looking for: 4+ years in … applied data science with a proven record of delivering value Hands-on experience with Azure , yolo , and OpenCV Proven track record training custom image models on real-world datasets Experience deploying to Azure ML production pipelines Experience with LLMs in production settings Strong understanding of model evaluation, data More ❯

peers. Autonomy: Authority over technical strategy, decisions and implementation approach, so you can deliver using practices that align with your preferred ways of working. Data Driven: Utilise rich logs, metrics, and data to monitor and improve system performance & reliability. Culture: Enhancing a diverse engineering culture by taking part … look for in you: Experience working in large and small agile teams of engineers, and eager to collaborate with other disciplines, such as Marketing, Data Analytics & Product Owners. Good understanding of Cloud technologies, and their application to solve problems such as Event/Data Ingestion and 3rd More ❯

B2B environment, ideally within insurance Experience with Agile Project delivery methods, and associated software tools including Confluence and Jira Experience with Product that is data driven and heavy in calculation logics Experience in working with a large dataset, and having experience working with data extraction processes is a … plan for some of our roles. What is Insurwave? Insurwave is a disruptive Insurtech company leveraging the power of AI to consolidate and visualise data, helping clients to understand risk and make smarter risk transfer and insurance decisions. Our platform offers an integrated insurance management experience, from ai-driven … data ingestion through to collecting and consolidating risk data providing insight on business exposure changes in real-time. What’s in it for me? You’ll be part of a supportive team, working in a Values led culture, doing the exciting work that you thrive on, making More ❯

and resolve problems. Bachelor’s degree in Computer Science, Information Technology, or related field. Proficiency in configuring Sprinklr Modern Care's/CCaas including data ingestion, workflow automation, and reporting. Experience with integrating Sprinklr Modern Care with external systems, including CRM systems, ticketing systems, and custom applications. Desirable … social media management software. Knowledge of industry-standard frameworks (e.g., Contact center, ITIL). Certification in Sprinklr Modern Care or related technologies. Experience with data analytics and reporting tools. Familiarity with cloud-based technologies and software-as-a-service (SaaS) solutions. Rewards & Benefits TCS is consistently voted a Top More ❯

opportunities to make a big impact in scaling the people, process and technology in service of that mission. We welcome passion and expertise around data quality, systems thinking and cutting through the noise to deliver great experiences to customers. About the Role As a Staff Software Engineer in our … experienced engineer. 7 years+ experience with Python. Deep understanding of one or more languages, primarily Python. Special interest in the areas of large scale data ingestion, enrichment, lineage and consensus. Experience with scaling and debugging scaled databases. Knowledge of cloud computing platforms, such as AWS, GCP and Azure More ❯

clients, ensuring timely and successful implementation. Collaborate with internal product, engineering, and customer success teams to tailor onboarding workflows. Configure client accounts and support data ingestion, validation, and reconciliation processes. Provide hands-on support with Advanced Excel and work with SQL queries for data troubleshooting. Educate clients More ❯

Engineering, or a related field. 2+ years of professional development experience as a Software Engineer in either C++, Python or Java. Strong understanding of data structures, algorithms, and software design principles. Experience with multi-threaded programming and real-time systems (C++/Java) Experience writing software for massive scale … for maximum speed and efficiency, minimising latency. Implement and enhance trading algorithms in collaboration with quantitative researchers and traders. Handle large volumes of market data in real-time, ensuring accurate and fast data ingestion, processing, and storage to feed trading algorithms. If you are a talented Software More ❯

generation. What You’ll Actually Be Doing Engineering research tools in Python and C++ , built for scale and speed Designing systems to handle massive data ingestion , feature engineering, and backtesting Collaborating directly with researchers to productionize models — from linear regressions to machine learning pipelines Leveraging HPC and distributed … a systematic trading or quant research environment Fluency in Python and/or C++, with a strong engineering mindset Hands-on with distributed systems , data pipelines , or ML tooling Ability to partner with researchers without translation layers Comfortable with ownership, ambiguity, and pace More ❯

NodeJS Developer - NodeJS Engineer - Data - TDD - AWS - 6 Months - Paying up to £650 (Inside IR35) - Hybrid You will be an integral part of the backend engineering team, building and developing the tooling and pipelines that form their data ingestion and modelling machine. Paying up to £650 (Inside More ❯

Increasing with length of service The Role: We are looking for an experienced SQL Database Administrator. Key Responsibilities: Support and maintain a large-scale data warehouse environment, handling high volumes of structured data. Manage and process incoming data streams from external sources (e.g. cameras and monitoring systems). … Develop and support server-side data processing workflows to ensure timely and accurate data ingestion. Collaborate with 2nd and 3rd line support teams, providing expertise on complex data-related issues. Maintain and optimise the central database infrastructure, ensuring performance, reliability, and scalability. Work closely with stakeholders to … deliver custom reports and analytics. Ensure data integrity, transformation, and storage aligns with business and regulatory requirements. Participate in long-standing, enterprise-level projects requiring consistent support and system knowledge. Troubleshoot data warehouse and integration issues and implement enhancements for improved performance and reliability. Contribute to the design More ❯

primary interface between the consultancy and the client’s Head of Engineering, ensuring strategic objectives are met while leading multi-disciplinary teams across Digital, Data, DevOps, and Cloud Engineering. The role requires a proven leader who can drive engineering excellence, reduce technical debt, and streamline delivery processes. You will … . Champion engineering best practices , promoting CI/CD, DevOps, and modern cloud-native architectures . Technical Excellence & Delivery Oversight Provide technical oversight across data pipelines, digital platforms, DevOps practices, and cloud migrations . Assess and mitigate technical debt , ensuring a balance between quick wins and long-term stability … and modernization of legacy applications , ensuring observability, performance, and security . Collaborate with Chief Engineers and engineering teams to define best practices for ETL, data integration, and cloud adoption . Ensure engineering maturity measurement through relevant KPIs such as code quality, deployment frequency, incident resolution time, and PR review More ❯