26 to 50 of 54 Data Ingestion Jobs in the UK excluding London

Junior Data Engineer - Research & Technology Role Overview This role supports the development and maintenance of a scalable data platform underpinning private equity investment activity across EMEA. Working closely with investment teams, you will build and optimise data pipelines, automate workflows, and transform fragmented market and real estate … data into structured intelligence that informs underwriting, portfolio strategy, and capital allocation decisions. Role Responsibilities * Build and maintain scalable data pipelines, managing large data sets from internal and external sources. * Develop and optimise ETL processes using AWS, SQL, and modern cloud technologies. * Manage API integrations and automate ...

OpenAI services on Azure Function Apps Monitor and resolve API bugs and performance issues to ensure high availability and reliability Oversee and troubleshoot data ingestion processes supporting the chatbot’s functionality Collaborate with solution architects, product teams, and the cloud team to deliver a reliable and scalable solution … OpenAI APIs Hands-on experience with Azure Function Apps and cloud-native development Strong background in API design, monitoring, and performance optimisation Experience managing data ingestion pipelines and integrating data sources Excellent problem-solving skills and a proactive, solution-oriented mindset Strong communication and leadership skills, with ...

Lead Data Engineer - Gambling Sector (No Sponsorship) Salary: £125,000 Location: Remote (London Office) This is an opportunity to step into a hands-on leadership role where you will shape a modern data function, influence technical direction, and build high-impact data infrastructure from the ground up. … base across the UK. With strong growth, a clear product vision, and a collaborative engineering culture, they offer the chance to directly influence how data powers the next phase of their evolution. The business values responsible innovation, high technical standards, and a genuine passion for building great digital products. ...

life cycle. Should have managed Risks/Issues, should have managed Agile project. Must have used agile management tool. Needs to have knowledge around Data specific project executions like - Data Warehousing, Data Migrations, Business Intelligence, Analytics, Big Data, Cloud (AWS/Azure) transformations. Key Responsibilities … technical delivery roadmap across data ingestion, ETL/ELT pipelines, data lake/warehouse models, streaming workloads, and analytics products. Own and Supervise Quality, Testing & Release Governance Required Expertise:- Overall experience: 12 to 14 years, 6+ years relevant project management experience with focus on delivering cross-functional ...

Client Gamma RRF ID 2026-18851 £400 per day Outside IR35 Duration 3 months initial Start ASAP Overview We are looking for an experienced Data Engineer to design and deliver scalable, secure, and high performance data solutions. The role focuses heavily on Snowflake architecture, cloud data platforms … modern data pipeline development. Key Responsibilities Design and architect data solutions using Snowflake Optimise workflows and integrations using Openflow Develop and maintain ETL and ELT pipelines from Salesforce, relational databases, files, and NoSQL sources Implement dimensional modelling to support data warehousing and BI reporting Build and optimise ...

Data Engineer £60 0-700/day outside IR35 3-month initial contract Remote working - occasional site visits Working with a leading financial services client who are looking for a Data Engineering Consultant. Looking for someone who can act independently and deliver complex data products - work with … stakeholders to understand requirements, engineer and deliver products. Data Engineering Consultant, key responsibilities: Take ownership of end-to-end of complex data products Work cross-functionally with non-technical stakeholders to understand data requirements Expert knowledge of SQL to query, analysis and model data Experience with ...

Data Platform Engineer – Warton – SC Cleared – 12 Month Contract About Your Role: As a Data Platform Engineer in a highly regulated environment, you will be responsible for designing, building, and maintaining secure and scalable data infrastructure that supports both cloud and on premises platforms. You will play … role in ensuring that all data systems comply with industry regulations and security standards while enabling efficient access for analytics and operational teams. A strong command of Apache NiFi is essential for this role. You will design, implement, and maintain data flows using NiFi, ensuring accurate, efficient ...

DATA ENGINEER 6-MONTH CONTRACT £500-£550 PER DAY OUTSIDE IR35 FULLY REMOTE This role offers a great opportunity for an AWS Data Engineer to join a fast-paced media company working on a cutting-edge data ingestion and transformation project. You'll take ownership … pipeline development, integrating real-time data streams and third-party APIs into a modern analytics stack. The environment promotes autonomy, modern tooling, and close collaboration across technical and business teams. THE COMPANY This is a leading media organisation leveraging data to drive audience insights, engagement strategies, and targeted ...

Data Engineer Hybrid: Birmingham Salary: £45000 to £50000. This is an exciting opportunity to step into a hands on Data Engineer role within an analytics team that is growing in both scope and influence. You will play a key part in stabilising and evolving a broad data … healthcare space with a strong reputation for product quality, research, and customer care. The organisation is scaling quickly, expanding internationally, and investing in data as a core enabler for decision making. The culture is collaborative and people focused, with a close knit feel, modern office space, and regular exposure ...

ready to play a key role in a modern data transformation? We are seeking a BI Solutions Analyst to join our Data Centre of Excellence. In this role, you will help us transition from legacy systems to a cutting-edge Microsoft Fabric platform, ensuring our reporting assets … measures. Semantic Model Support: Maintain enterprise-level semantic models (including Sales and Finance models) and develop efficient DAX measures aligned with governance standards. Data Triage & Operational Support: Monitor data ingestion pipelines across Bronze, Silver, and Gold layers. You'll diagnose load failures and inconsistencies within our metadata ...

Data Engineer | Outside IR35 | £450 - £500 | 6 months | Hybrid London We’re supporting a company who are looking for a Data Engineer to build and enhance the data processing capabilities within their Databricks environment. You’ll be responsible for developing the code that drives their data … Python, Spark, and Databricks Workflows to deliver new platform functionality and ensure efficient execution. Key Responsibilities Develop reliable Python and PySpark code to support data ingestion, transformation, and end-to-end processing. Deliver new technical features and components aligned to approved solution designs and business requirements. Enhance, extend ...

With a strong commitment to user-centred design and agile delivery, and more to deliver innovative digital services that matter Role Description As a Data Technical Architect, you will lead the design and governance of Google Cloud Platform data architecture. You will define scalable, secure and reusable data … BigQuery, Cloud SQL and Cloud Composer. You will ensure interoperability, portability and alignment with multi-cloud considerations where required. You will embed strong data governance using Dataplex and Dataform, ensure compliance with GDPR and public sector security standards, and provide architectural assurance through appropriate governance forums. You will also ...

Practice. This is an opportunity to work end-to-end on high-impact analytics solutions that directly support established organisations in making better, faster, data-driven decisions. The Role: As an Analytics Engineer, you'll play a key role in transforming high-quality data into trusted, scalable insight … semantic models Develop high-quality Power BI reports, including advanced DAX Apply best practices across modelling, performance optimisation, and governance Collaborate closely with Data Engineering teams using Microsoft Fabric Provide responsive support for analytics products used across the organisation Requirements: Hands-on experience with Power BI (data ingestion ...

subject matter experts to identify automation opportunities and define technical and functional requirements. Coding and Programming: Develop automation scripts and build pipelines for data ingestion, preprocessing and feature engineering using languages such as Python, Java or R. Testing and Maintenance: Conduct thorough testing, resolve issues, monitor accuracy … system performance, and implement updates or improvements as needed. Collaboration: Partner with cross-functional teams including data scientists, engineers, software developers and IT operations to ensure seamless integration of AI/automation solutions. Documentation and Reporting: Produce and maintain clear technical documentation and prepare performance reports, insights and recommendations ...

security matter as much as feature velocity. Working embedded with AI, Backend, Frontend, and DevOps, you'll shape how they validate agent behaviours, data pipelines, and end-to-end operational workflows - from research prototypes through to production deployments. Quality is built in from day one, not inspected … safety-conscious, and security-first delivery expectations. Act as the primary quality authority across projects, advising engineering and productleadership on risk and readiness. AI & Data-Focused Testing Design approaches for testing multi-agent workflows, including orchestration logic, memory/state handling, and tool integrations. Define validation strategies ...

Yorkshire Working Hours: 40 hours per week - Full timeA globally established organisation is seeking an Azure Solution Architect to support a large-scale Manufacturing Data Management programme. This is a senior architecture role focused on reviewing technical designs, validating configurations and ensuring seamless edge-to-cloud integration across Azure … aligns with enterprise cloud and MDM standards Coordinate with networking and site IT teams to ensure connectivity and firewall readiness Validate edge-to-cloud data flows including payload format, frequency and ingestion design Facilitate Azure subscription access, permissions and resource configuration Own go-live readiness and drive resolution ...

London Working Hours: 40 hours per week - Full timeA globally established organisation is seeking an Azure Solution Architect to support a large-scale Manufacturing Data Management programme. This is a senior architecture role focused on reviewing technical designs, validating configurations and ensuring seamless edge-to-cloud integration across Azure … aligns with enterprise cloud and MDM standards Coordinate with networking and site IT teams to ensure connectivity and firewall readiness Validate edge-to-cloud data flows including payload format, frequency and ingestion design Facilitate Azure subscription access, permissions and resource configuration Own go-live readiness and drive resolution ...

strong track record of innovation, operational excellence, and a culture that values collaboration, flexibility, and diversity. This opportunity sits within a modern Data Science team and will play a pivotal role in designing and building an AI-enabled reporting and reconciliation platform that will transform the client's Fund … Administration processes. Role Overview The successful candidate will report into the Head of Data Science and work closely with a dedicated Product Manager in the Fund Administration team. This is a hands-on engineering role where you will architect, build, and deliver production-grade automation solutions in a real ...

helping people live longer, healthier lives. We partner with life sciences organisations which include, pharmaceutical, biotech and healthcare leaders, to build transformative AI and data driven technologies addressing real-world health challenges. From strategy and research to UX design and agile development, we deliver and validate impactful solutions using … Skills/Experience Understanding of MLOps, model serving, scaling and monitoring workflows (e.g., BentoML, MLflow, Vertex AI, AWS Sagemaker) Experience building streaming + batch data ingestion and transformation pipelines (Spark/Airflow/Beam) Mobile and front-end web application development experience What We Offer At Newpage ...

hiring an experienced AI Architect to lead the design and governance of enterprise-grade AI/ML and Generative AI solutions across data, application and cloud infrastructure layers. This is a senior-level architecture role focused on delivering production-ready LLM, RAG and agentic AI platforms within complex enterprise … environments. Key Responsibilities Define and own AI reference architecture across data ingestion, model orchestration, inference services and application integration Architect and deploy LLM solutions (GPT, BERT, Transformers) including RAG pipelines and vector databases Lead LLMOps/MLOps strategy including model lifecycle, CI/CD for ML, model registry ...

this, the team are also responsible for the development of core products that can be utilised across the business, such as a standardised data ingestion and distribution exchange for engaging with 3rd party data providers. Finally, you will also be a proponent of agile practises and clean ...

HANA. Plan and run database upgrades, patching, migrations, and disaster recovery processes. Implement and enforce database best practices, standards, and governance. Collaborate with data engineering, analytics, and business teams to support high-volume data ingestion and reporting. Troubleshoot and resolve complex database issues to ensure high availability … reliability. Requirements: Extensive SAP IQ (Sybase IQ) DBA experience. Strong experience with analytics-focused databases optimized for large-scale data warehousing. Hands-on with performance tuning, monitoring, and troubleshooting for Sybase IQ NLS and SAP HANA integration experience is a plus. ECS Recruitment Group Ltd is acting ...

outsourced to a modelling partner). Multiple squads are part of the MMM initiative and loosely ladder up into the follow different areas: - Modelling. - Data Ingestion & automation. - Internalisation of external developed architecture. - Visualisation. Background - As the business looks to increase the coverage of MMM via their internally built … results and work with local stakeholders to provide actionable insights with immediate impact. Scope - Work with local stakeholders to scope MMM deliveries (including what data to include and aligning on the scope of the models). - Once the Data Scientists have complete models, deliver these models ...

core systems behind the intelligence engine. You’ll help shape architecture, product direction, and engineering standards while building across the stack, from signal ingestion pipelines and AI analysis layers through to the frontend experience. This is a role for someone who enjoys operating in ambiguity, shipping quickly, and solving … building production software systems Python development TypeScript development React frontend development LLM integration in production environments Asynchronous systems (Celery, Redis, task queues) PostgreSQL databases Data ingestion pipelines and AI-driven analysis AI-assisted development tools and modern LLM ecosystems Strong computer science fundamentals Startup mindset with high ownership ...

engineering experience, ideally in GCP and its AI ecosystem * Strong Python engineering ability and experience building production-grade services * Experience working with vector search, data ingestion, observability, and experimentation frameworks * A STEM background or equivalent technical capability What They Offer * Salary up to 90,000 plus benefits * Hybrid ...