251 to 275 of 420 Data Ingestion Jobs in the UK

a small, high-impact team responsible for the firm's most critical datasets - including corporate actions, fundamentals, and index membership data. Rather than chasing breadth, this team focuses on data depth: building a clean, unified, and intelligent data layer that underpins all major decisions across the business. We build distributed pipelines to ingest, clean, and expose daily datasets … to internal teams. This is an engineering-first team with strong autonomy - ideal for someone who enjoys taking ownership of data quality, system design, and intelligent automation. What You'll Do Own and improve automated data ingestion and processing pipelines Design and maintain distributed systems for reliable, scalable data access Identify and fix data anomalies … using statistical and algorithmic approaches Collaborate directly with internal users across research, trading, and operations Anticipate future data needs and help evolve a minimal, robust data schema Hard Requirements Strong coding skills in a statically typed language (e.g. Go, Java, C++) Solid background in distributed systems and relational databases Comfortable with data analysis; knowledge of statistics or More ❯

Our talented people empower us, and we believe in being part of a team that is open, collaborative, entrepreneurial, passionate, and above all, fun. About the team The Enterprise Data & AI (EDAI) Team, led by the FIS Chief Data & AI Officer (CDAO), is focused on delivering cutting-edge Data, AI, and Machine Learning platforms and models. The … group is organized into four engineering Towers: Data Ingestion, Data Engineering, Data Warehousing, and Artificial Intelligence/Machine Learning. These are supported by three enabling Towers: Strategy & Operations, Customer Success, and Data Ops/Compliance. Based between the US, UK, and India, the team is 250 strong and committed to delivering value for FIS through … data and AI. What you will be doing You will play a pivotal role in the success of the new enterprise Data and AI platform developed by the EDAI team. This platform, built with industry-leading technology and the highest security standards, will become the enterprise's new benchmark – allowing FIS to anonymize and commercialize data to More ❯

modern Silicon Valley innovation and time-tested Japanese quality craftsmanship. We leverage these complementary strengths to amplify the capabilities of drivers, foster happiness, and elevate well-being. Team Our data platform team is working on accelerating autonomous driving by providing access to petabytes of data collected by our fleet of autonomous and non-autonomous vehicles. Efficient, fast and … cost-effective access to data at large scale is key to tackle the hardest problems in AD/ADAS, from developing the Machine Learning (ML) models for perception and prediction of human driving patterns, to increasing the sophistication of our validation and simulation by identifying rare and interesting real-world driving situations. The data ecosystem developed by the … London team is a key building block for developing and testing modern AD/ADAS products that will impact millions of customers. Our ML and Data pipelines are built on-top of the open-source Flyte orchestration framework and are deployed to AWS. Pipeline code is written in Python. We use SQS and Kafka to automate data connections More ❯

Hybrid role based in London Role Overview The Data Scientist role will be responsible for designing, developing, and implementing advanced AI and machine learning models, utilizing both traditional and emerging approaches to address complex business challenges. This position is well-suited for with a PhD/Masters or equivalent in Statistics, Machine Learning, or Artificial Intelligence with a passion … including credit, commercial, product, and operations, with a significant emphasis on credit risk. ML Ops Leadership: Define and execute an ML Ops framework to streamline model lifecycle management, including data ingestion, data transformation, model training, deployment, and monitoring. Collaborative Problem Solving: Work with commercial and product teams to align ML solutions with business goals, ensuring risk considerations … compliance with industry standards. Who you are Educational Background: PhD/Master's or equivalent in Statistics, Machine Learning or Artificial Intelligence. Deep Experience: 5+ years of experience in data science, including hands-on ML model development and production deployment. ML & Statistical Expertise: Expert in traditional machine learning and statistical methods (e.g., classification, regression, time series models), with deep More ❯

s challenges of today and tomorrow. Informed and validated by science and data. Superpowered by creativity and design. All underpinned by technology created with purpose. Your Role As a Data & AI Operating Model Consultant, you will play a key role in setting organisations up for the fourth industrial revolution, helping them to effectively leverage data and AI to … drive business outcomes: You will play a central role in the designing of effective data, analytics & AI operating models, covering all components required to deliver an effective data & AI capability. You will be work with our clients to define their strategic objectives, and then build a picture of the requisite data & analytical capabilities required to realise those … strategic objectives. You will provide guidance and advisory services to clients on a wide range of data & analytics capabilities at all points of the data & analytics value chain. You will work with stakeholders at all levels of the organisation to assess the maturity of existing data capabilities, identifying opportunities for re-use and corresponding gaps in capability. More ❯

Quantitative Developer - Java (Risk Technology) Millennium is a top-tier global hedge fund committed to leveraging innovations in technology and data science to solve complex business problems. The Risk Technology team is seeking a Quantitative Developer who will utilize Java, AWS, and data manipulation libraries to deliver data-driven risk management solutions for stakeholders such as Portfolio … London, and Singapore to develop risk analytics solutions for various asset classes (fixed-income, commodities, equities, etc.). Build, enhance, and maintain Java REST services and related systems. Develop data ingestion pipelines and core data systems to enable risk management analytics access via programmatic interfaces and web applications. Create and manage cloud applications on AWS. Work with … solutions to Portfolio and Risk Managers. Required skills/experience: Strong analytical and mathematical skills with interest or experience in quantitative finance. Good understanding of design patterns, algorithms, and data structures. Significant experience with modern Java. Experience with REST APIs and cloud services. Relational SQL database development experience. Unix/Linux command-line proficiency. Ability to work independently in More ❯

you will play a key role in ensuring that critical business services have the capacity and resilience to meet regulatory and operational demands. You will focus on developing robust data pipelines and solutions that support capacity monitoring, reporting, and forecasting across complex IT environments. Role Overview: Develop ETL processes to extract and normalise capacity data from infrastructure sources. … Automate data ingestion and processing into SQL Server databases. Normalising raw infrastructure and monitoring data into structured formats suitable for graphing and trending Collaborate with DBAs and technical teams to ensure transparency and supportability. Build data flows that support capacity reporting and trending, using modern technologies. Skills & Experience Required: 3+ years' experience in ETL/data processing roles. Proficiency with ETL tools, SQL Server, RESTful APIs. Experience working with infrastructure data (e.g., Nutanix, VMware, Dell EMC desirable). Scripting experience in Python, SQL (T-SQL), Excel VBA. Knowledge of data optimisation, large datasets, JSON/XML formats. Experience of setting up SQL DTSS and Control/M batch jobs useful Excellent communication, analytical More ❯

This is an exciting opportunity for a Power BI Specialist. Based in London, you will use your expertise to create insightful dashboards and reports that support data-driven decision-making. Client Details A £100million turnover company. Description Design, build, and refine dashboards focusing on profit and loss, margin analysis, cost control, and operational KPIs across the Group. Enhance existing … dashboards to integrate with the new Microsoft Data Links repository, replacing scheduled refreshes with live data feeds. Use Power Query (M language) for data ingestion and transformation. Develop advanced DAX measures, including time intelligence calculations, dynamic reporting layers, and calculated tables. Manage the full dashboard development lifecycle, from stakeholder engagement and data modelling to prototyping … business units to gather requirements and shape outputs. Ensure dashboards are user-friendly, commercially relevant, and actionable for both finance and non-finance users. Work alongside internal ERP and data teams (who manage SQL and connectors) to maintain data consistency with Unit4 and Tagetik systems. Provide ad-hoc support for wider reporting and transformation projects, including benchmarking and More ❯

Chat API using Python, LangChain, and OpenAI services on Azure Function Apps Monitor and resolve API bugs and performance issues to ensure high availability and reliability Oversee and troubleshoot data ingestion processes supporting the chatbot's functionality Collaborate with solution architects, product teams, and the cloud team to deliver a reliable and scalable solution Champion best practices in … strong knowledge of LangChain and OpenAI APIs Hands-on experience with Azure Function Apps and cloud-native development Strong background in API design, monitoring, and performance optimisation Experience managing data ingestion pipelines and integrating data sources Excellent problem-solving skills and a proactive, solution-oriented mindset Strong communication and leadership skills, with the ability to work cross More ❯

financial sector. The ideal candidate will have a deep experience and understanding of test automation frameworks, and an understanding of COTS (Commercial Off-The-Shelf) products and experience in Data Modernization/Transformation programmes. As an Automation Test Lead, you will be responsible for designing and executing test plans, coordinating with cross-functional teams, and ensuring the quality of … support Coordinate with cross-functional teams, including developers, business analysts, and project managers, to understand project requirements and develop test strategies Potentially also support, advise, assist to prepare test data, migrations testing CI/CD, NFT and UAT Potentially create automated solutions around data inputs, outputs and data source-target comparisons Qualifications and Skills: Bachelor's degree … experience as an Automation Test Lead, with at least 5 years of experience and automation testing in the banking and financial sector Experience with cloud technologies, such as Azure Data Factory In depth experience, knowledge and understanding of test automation frameworks with Karate, JSON, NUnit, Pytest, Eggplant; along with Github Actions pipeline and IntelliJ IDE. In-depth understanding of More ❯

financial sector. The ideal candidate will have a deep experience and understanding of test automation frameworks, and an understanding of COTS (Commercial Off-The-Shelf) products and experience in Data Modernization/Transformation programmes. As an Automation Test Lead, you will be responsible for designing and executing test plans, coordinating with cross-functional teams, and ensuring the quality of … support Coordinate with cross-functional teams, including developers, business analysts, and project managers, to understand project requirements and develop test strategies Potentially also support, advise, assist to prepare test data, migrations testing CI/CD, NFT and UAT Potentially create automated solutions around data inputs, outputs and data source-target comparisons Qualifications and Skills: Bachelor's degree … experience as an Automation Test Lead, with at least 5 years of experience and automation testing in the banking and financial sector Experience with cloud technologies, such as Azure Data Factory In depth experience, knowledge and understanding of test automation frameworks with Karate, JSON, NUnit, Pytest, Eggplant; along with Github Actions pipeline and IntelliJ IDE. In-depth understanding of More ❯

Software Engineer, Systematic Equity Millennium is a top tier global hedge fund with a strong commitment to leveraging market innovations in technology and data to deliver high-quality returns. About Us We are a well established systematic equity trading group within a famous and prestigious global investment firm. Our team develops and maintains a plethora of sophisticated trading strategies … collaborate closely with quantitative researchers, portfolio manager, and supporting teams to build scalable, high-performance trading systems. Your primary responsibility will be to develop, optimize, and maintain complex algorithms, data pipelines, and software infrastructure that underpin our trading strategies while ensuring robust risk management and compliance standards. Location London Principal Responsibilities Lead the development and evolution of the quantitative … translate trading strategies into efficient, production-grade code Evaluate and integrate emerging technologies, libraries, and tools to drive continuous improvement Design, code, test, and deploy robust trading algorithms and data processing pipelines Develop real-time data ingestion systems and analytics frameworks for market data, risk metrics, and performance Optimize existing codebases for speed, reliability, and maintainability More ❯

What is Human Native? At Human Native, we're building an AI data marketplace that ensures creators and rights holders are fairly compensated for their work while providing AI developers with high-quality, responsibly licensed training data. We believe in building AI the right way - ensuring transparency, fairness, and accessibility. This is a hard problem, and we need brilliant … minds to help us solve it. The Opportunity As an ML Engineer, you'll help us index, benchmark, and evaluate training datasets at scale. Your expertise with data, AI and ML training methodologies and evaluation techniques will advance the state of the art for developing AI. You will work across: Designing and developing benchmarks that allow our customers to … understand their value of data for training ML (quantifying dataset quality and biases). Deploy these benchmarks by implementing end-to-end data evaluation pipelines to be run on different datasets and ML models. Tools to visualise, analyze, and understand the attributes of datasets based on the evaluations. Develop ML models to transform, clean and understand data. Collaborating More ❯

management Skills and Experience AI/ML Expertise: Strong understanding of LLMs, NLP, and AI-driven recommendations. Technical Leadership: Experience in AI product scalability, DevOps, and cloud architecture. Backend & Data Skills: Proficiency in Python, Node.js, PostgreSQL, MongoDB, API security. AI Model Deployment: MLOps, scaling, data engineering, and AI ethics awareness. Strategic Mindset: Ability to align technology with business … objectives and cost efficiency. Security & Compliance Knowledge: GDPR, API authentication, and observability best practices. Big data processing: Understanding data lakes, warehouses, and tools like Spark, Kafka, and Airflow. ETL Pipelines: Ability to evaluate data ingestion, transformation, and cleaning processes. DevOps & CI/CD: Hands-on knowledge of Jenkins, GitHub Actions, Terraform, and CloudFormation. What's in More ❯

own Terraform to manage and provision our cloud infrastructure for machine learning operations. Oversee the transition to a real-time streaming architecture for our machine learning applications, ensuring efficient data ingestion, feature engineering, and model serving in a streaming context. Develop and implement a comprehensive monitoring framework to track model performance, identify potential issues, and ensure optimal model … health in production. Monitor model performance and update them as needed to adapt to new data and changing conditions. Collaborate closely with data scientists and engineers to ensure seamless integration of models into our existing systems and workflows. Stay abreast of the latest MLOps trends and technologies to continuously improve our processes and tools. Your Story You have … Kubernetes) and using orchestration tools such as Kubeflow (our preferred tool) or similar frameworks like Apache Airflow to manage and automate ML workflows. You have experience with real-time data streaming technologies such as Kafka and Confluent and feature stores in such settings. You are skilled in building and maintaining monitoring systems for machine learning models. You have excellent More ❯

management Skills and Experience AI/ML Expertise: Strong understanding of LLMs, NLP, and AI-driven recommendations. Technical Leadership: Experience in AI product scalability, DevOps, and cloud architecture. Backend & Data Skills: Proficiency in Python, Node.js, PostgreSQL, MongoDB, API security. AI Model Deployment: MLOps, scaling, data engineering, and AI ethics awareness. Strategic Mindset: Ability to align technology with business … objectives and cost efficiency. Security & Compliance Knowledge: GDPR, API authentication, and observability best practices. Big data processing: Understanding data lakes, warehouses, and tools like Spark, Kafka, and Airflow. ETL Pipelines: Ability to evaluate data ingestion, transformation, and cleaning processes. DevOps & CI/CD: Hands-on knowledge of Jenkins, GitHub Actions, Terraform, and CloudFormation. What's in More ❯

the end-to-end architecture of RMN platforms, covering sponsored ads, programmatic DSP, self-service portals, and dynamic ad placements. • Design scalable and modular RMN solutions, integrating Ad-Serving, Data Management Platforms (DMP), Customer Data Platforms (CDP), and Demand-Side Platforms (DSP). • Ensure real-time bidding (RTB) and programmatic advertising capabilities within the RMN ecosystem. • Develop a … analytics, and automated campaign optimization. • Implement Generative AI for automated ad creation, content personalization, and A/B testing. • Enhance Dynamic Creative Optimization (DCO) capabilities for personalized ad experiences. Data & Integration Framework • Define data ingestion, processing, and activation strategies across first-party, second-party, and third-party data sources. • Ensure seamless API integrations with RMN partners … attribution models to measure ad performance effectively. Ad-Tech Stack & Cloud Architecture • Design RMN solutions leveraging Google Cloud, AWS, or Azure, ensuring high availability and low-latency performance. • Optimize data pipelines, real-time analytics, and identity resolution mechanisms. • Oversee privacy-compliant data handling (GDPR, CCPA) and secure identity frameworks. Stakeholder Collaboration & Roadmap Execution • Work closely with engineering, product More ❯

patches, and enhancements with minimal disruption to business operations. Collaborate with stakeholders to define requirements for system integrations, user needs, and architectural designs. Identify and implement solutions to improve data quality, ingestion/extraction processes, and overall data accessibility across SQL and other databases. Execute system failovers and disaster recovery procedures to minimize risk and downtime. Manage … with strong experience customizing Symitar for reporting and system enhancements. Expertise in Jack Henry system releases, including planning, testing, and deployment. In-depth experience with SQL Server management including data ingestion, query optimization, and backups. Strong project leadership skills and experience managing complex, cross-functional technical initiatives. Advanced troubleshooting, analytical, and strategic problem-solving abilities in high-pressure More ❯

You will need to be self-directed, action-oriented, and comfortable making high-impact decisions quickly. Your focus will include consulting on new product developments, refining complex real-time data pipelines handling millions of data points, and driving operational excellence across our SaaS platform. Key Responsibilities: Production Ownership: Manage and optimise our live software environment, ensuring stability, scalability … Leadership: Consult on new feature and product development, integrating these into the existing production infrastructure with minimal disruption. Performance Optimisation: Continuously monitor, benchmark, and enhance system performance and reliability. Data Pipeline Management: Oversee and improve our extensive real-time data ingestion pipelines, ensuring they are resilient, scalable, and future proof. Process Innovation: Identify and implement improvements across … on ability to execute technical solutions. Passion for continuous learning, improvement, and operational excellence. Deep technical expertise in: Databases such as MySQL, Singlestore, and BigQuery. Real-time, high-throughput data pipelines. Cloud platforms, particularly Google Cloud. What we offer: Supportive, friendly colleagues who work together to achieve shared goals and contribute directly to mission-critical systems that underpin the More ❯

and ensure the environment is compliant. Evaluate and pilot emerging capabilities such as Microsoft Copilot and AI-driven productivity tools. Oversee the operational integrity, performance, and availability of core data platforms and pipelines that support reporting, analytics, and AI readiness across the firm. Collaborate closely with the Data & AI teams to ensure cloud infrastructure supports scalable, secure, and … reliable data ingestion, transformation, and delivery. Budgeting & Licensing Own and manage the platform budget, ensuring efficient use of resources and cost optimisation. Manage and forecast consumption charges, ensuring compliance and maximising license utilisation. Liaise with procurement, finance, and vendors for renewals and cost planning. Governance, Risk & Compliance Define and enforce policies for workload management and governance across our … cloud estate. Maintain compliance with internal policies and regulatory requirements. Ensure data governance best practices across cloud environment. Stakeholder Engagement Collaborate with business units, Technology senior leadership team, and product teams to gather requirements and improve platform services. Champion change management initiatives that drive behavioural adoption of digital tools across the firm. Act as an evangelist for Modern cloud More ❯

and ensure the environment is compliant. Evaluate and pilot emerging capabilities such as Microsoft Copilot and AI-driven productivity tools. Oversee the operational integrity, performance, and availability of core data platforms and pipelines that support reporting, analytics, and AI readiness across the firm. Collaborate closely with the Data & AI teams to ensure cloud infrastructure supports scalable, secure, and … reliable data ingestion, transformation, and delivery. Budgeting & Licensing Own and manage the platform budget, ensuring efficient use of resources and cost optimisation. Manage and forecast consumption charges, ensuring compliance and maximising license utilisation. Liaise with procurement, finance, and vendors for renewals and cost planning. Governance, Risk & Compliance Define and enforce policies for workload management and governance across our … cloud estate. Maintain compliance with internal policies and regulatory requirements. Ensure data governance best practices across cloud environment. Stakeholder Engagement Collaborate with business units, Technology senior leadership team, and product teams to gather requirements and improve platform services. Champion change management initiatives that drive behavioural adoption of digital tools across the firm. Act as an evangelist for Modern cloud More ❯

Data Engineer – Crypto Client, London, Hybrid, £Competitive We're hiring on behalf of a leading crypto client for an experienced Data Engineer to join their fast-growing team in London. This is an exciting opportunity to work on cutting-edge blockchain data infrastructure in a highly dynamic environment. What We're Looking For: 5+ years of engineering … experience Strong expertise in data engineering: – Proficient in Python , Spark – Advanced SQL skills Proven experience with data ingestion pipelines Hands-on with cloud platforms – ideally AWS Solid experience working with Big Data and time series databases You’ll be part of a team shaping the future of decentralized finance through robust, scalable data systems. Apply More ❯

Data Engineer – Crypto Client, London, Hybrid, £Competitive We're hiring on behalf of a leading crypto client for an experienced Data Engineer to join their fast-growing team in London. This is an exciting opportunity to work on cutting-edge blockchain data infrastructure in a highly dynamic environment. What We're Looking For: 5+ years of engineering … experience Strong expertise in data engineering: – Proficient in Python , Spark – Advanced SQL skills Proven experience with data ingestion pipelines Hands-on with cloud platforms – ideally AWS Solid experience working with Big Data and time series databases You’ll be part of a team shaping the future of decentralized finance through robust, scalable data systems. Apply More ❯

Design, implement, and support full life-cycle data engineering projects including large data ingestion, persistence, transformation, and retrieval. Design, develop, and maintain multiple code bases to service realtime data over a distributed data pipeline. Responsible for Kubernetes deployments and namespaces, MemSQL and Oracle databases, Victoria Metrics instances, on-prem S3, and codebases in Python, Typescript … and Kotlin. Maintain and manage pipeline between real-time data from streaming/non-streaming data sources such as Apache Pulsar/Oracle/MemSQL to application data stores and services. JOB REQUIREMENTS MINIMUM EDUCATION: Bachelor's Degree in Computer Science, CIS, MIS or Information Technology. MINIMUM EXPERIENCE: Five (5) years of experience as Data Engineer … Engineer, Oracle Engineer, Systems Engineer, or Software Engineer. ALTERNATE EDUCATION: Master's Degree in Computer Science, CIS, MIS, or Information Technology. ALTERNATE EXPERIENCE: Three (3) years of experience as Data Engineer, Database Engineer, Oracle Engineer, Systems Engineer, or Software Engineer. SPECIAL REQUIREMENTS: The required five (5) years of experience with a Bachelor's degree or three (3) years of More ❯

across various asset classes. They have grown into a global firm with offices in major financial hubs, including New York, Chicago, London, and Hong Kong. They have a very Data-Driven Culture with a heavy focus on technology, data science, and research. They're hiring for their enterprise data team who are building a cloud native data platform used by data analysts, scientists, and quantitative investors. The platform involves technologies like Docker, Kubernetes, REST APIs, and multi-language client libraries. Key Responsibilities: Build cloud-based, distributed data solutions. Understand and prioritize investment data needs. Collaborate with users and team members to refine requirements. Design high-performance data ingestion and delivery systems. … and infrastructure-as-code . API design experience; front-end experience with React is a plus. Cloud experience (especially AWS ) is highly valued. Additional Requirements: Experience working within the data space within a Hedge Fund Environment Happy to work onsite 4 days a week in central London Educated to Degree Level, ideally Computer Science or similar. More ❯