26 to 50 of 74 Data Ingestion Jobs in the UK

Data Engineer Embark on a Rewarding Journey with MIKI Travel: Where Your Career Takes Flight. At MIKI Travel, our 55+ years of expertise make us a trailblazing wholesale travel provider, curating unforgettable global adventures. As a premier private travel company with a presence in 36 offices, we invite you … Fuel your career with opportunities for growth, powered by values of quality, innovation, and commitment that shape the Miki way. Job Purpose As a Data Engineer you will be responsible for designing, developing, and maintaining data pipelines and infrastructure to enable efficient data ingestion, processing, storage … and analysis. The ideal candidate will have a strong background in data engineering, including expertise in building and optimizing ETL processes, data warehousing, and working with big data technologies. Main Responsibilities ● Collaborate with business stakeholders to gather and analyse business requirements, translating them into specific data more »

we serve and transform leading enterprises, including the Fortune Global 500, with our deep business and industry knowledge, digital operations services, and expertise in data, technology, and AI. Inviting applications for the role of Senior Principal Consultant, Data Architect/Data Engineer! In this role, you will … play a pivotal role in designing and implementing robust data solutions. Your technical expertise and leadership will drive the success of our big data initiatives. Good level of experience is required. Responsibilities: Build robust, high-performance, and adaptable solutions for modern platforms (GCP Big Query,Relational, OLAP) Finance … data modelling extensive experience desired as the data lead. Talend/Informatica transformation logic experience is required Technical thought leadership on architecture, data modeling, and Big Data strategy Expertise in technical principles, vision, and standards for data warehouse. Scalability to handle increasing data demands more »

Data engineer - Python - Azure - Inside IR35 - £500 per day - 6 months Exalto Consulting are currently recruiting for a contract Data Engineer for a client, 100% remote working inside IR35 paying £500 per day, initially 6 months Skills required for the role: Big data experience Data inventory … and data familiarisation Efficient data ingestion and ingestion pipelines Data cleaning and transformation Databricks (ideally with Unity Catalog) Python and PySpark CI/CD (ideally with Azure Devops) Unit testing (PyTest) If you have the above experience and are looking for a new contract role … please send your CV for immediate consideration as our client are looking to hire ASAP Data engineer - Python - Azure - Inside IR35 - £500 per day - 6 months more »

to £100,000 dependent on experience, plus bonus and other benefits. An overview of the role: We're seeking talented individuals to join a Data & AI Strategy team of a consultancy that we're supporting, where you'll design and build strategies and operating models to transform organizations across … all industries through data & AI. All Managers and Senior Managers work as trusted advisors with senior clients, leading teams of consultants and technical specialists on large and complex projects. We're looking for candidates with consulting experience in sectors such as Transport, Energy, Utilities, Government, Defence, Consumer Products & Retail … the opportunity to take on leadership roles within the consultancy practice, honing your skills before taking them to clients. Why this role? As a Data & AI Operating Model Manager, you will play a leadership role in setting organisations up for the fourth industrial revolution, helping them to effectively leverage more »

We're on a mission to empower experts with groundbreaking knowledge discovery tools. If you're a seasoned Software Engineer - Data with a passion for building AI-native data processing pipelines, we want you on our team! Your Role: Design, optimize, and maintain robust data pipelines for … complex knowledge-intensive documents. Curate and manage large-scale data ingestion and indexing pipelines, ensuring data quality and error handling. Implement end-to-end encryption and data isolation for secure data handling. Collaborate with a diverse team to integrate search infrastructure for large language models. … Your First 3 Months: Enhance data ingestion framework, boosting efficiency and scalability. Improve scalability and performance of data infrastructure, setting new benchmarks. Mentor team members and promote best practices in data engineering. What You Bring: Experience building and optimizing search infrastructure in cloud environments. Proficiency in more »

to change the status quo, we’ll help you make your moment. Join Team Purple. Join IFS. Job Description As a Principal/Senior Data Engineer, you will spearhead our data initiatives, transforming raw data into strategic assets that propel our business forward. This role is designed … those who excel in a dynamic environment, offering the freedom to innovate and the responsibility to deliver. Core Responsibilities: Identify and harness high-value data opportunities within our product suite, converting raw data into impactful features and reusable assets. Act as the authoritative voice on data technology … steering our organization towards cutting-edge platforms and methodologies. Architect and execute comprehensive data projects, ensuring robust scalability and performance from conception to deployment . Collaborate with a team of ML Engineers, Data Scientists, and other key stakeholders to design, implement, and maintain data pipelines that fuel more »

Data Platform Engineer - London/Hybrid - £70,000 FTC I am currently working with a well-established, global organisation within the education sector who are looking for a Data Platform Engineer to join their established IT team. Their current project is based around the creation and implementation of … a brand new Azure Data Platform. You will take responsibility for the design, development and maintenance of the data platform utilising your experience with the Azure tech stack. You will join a growing team. This is an opportunity to join a team who are serious about the way … that they use data and work within an organisation who are providing significant investment into all of their data teams. As part of this role, you will be responsible for some of the following areas. Develop and maintain the organisation Azure data warehouse infrastructure Create robust data more »

Senior Data Engineer - Investment Firm - London - Up to £150k base + bonus About Us: We are working with a prominent investment firm based in London, renowned for their innovative approach and commitment to delivering exceptional results for our clients. As part of our continuous growth and evolution, we are … seeking a talented Senior Data Engineer to join our dynamic team. Role Overview: As a Senior Data Engineer, you will play a pivotal role in managing and optimising our data infrastructure, ensuring the reliability, security, and efficiency of our data processes. You will work closely with … various teams to design, implement, and maintain robust data pipelines, leveraging cutting-edge technologies to drive insights and inform strategic decisions. Key Responsibilities: Develop and maintain data pipelines for security and index data, sourced from Bloomberg, Reuters, and other relevant providers. Utilise Python and Java (both preferred more »

Data Engineer - Manchester - £35,000 - Hybrid Working My client is hiring a full time Data Engineer to contribute to their expanding data science and analytics initiatives. Your role involves designing, building, and maintaining data pipelines, ensuring seamless data flow into their central Microsoft Azure repository. … Collaborate with Data Analysts and Scientists to optimise data quality, reliability, security, and automation. Skills & Responsibilities: Configure and troubleshoot Microsoft Azure, manage data ingestion Develop ETL scripts using Python, handle web scraping, APIs, and relational databases Familiarity with Power BI (you will not have to use … Power BI, but being aware of how to collaborate with people that do is important) Work closely with data analysts and scientists, contribute to data models and algorithms, and participate in data events Working knowledge of Git If you have the required skills and experience and would more »

Data Engineer - Fixed Term Contract About our Team At LexisNexis Intellectual Property (LNIP), our mission is to bring clarity to innovation by delivering better outcomes to the innovation community. We help innovators make more informed decisions, be more productive, and ultimately achieve superior results. By helping our customers achieve … their goals, we support the development of new technologies and processes that ultimately advance humanity About the Role As a Data Engineer at LexisNexis Intellectual Property (LNIP), you'll contribute to new product development. Your role will focus on complex natural language datasets, implementing cutting-edge data solutions … like Data Lakehouses and embracing DataOps for operational excellence. Responsibilities Developing and maintaining data infrastructure supporting real-time data processing in streaming architectures Implementing scalable data ingestion and ML pipelines, incorporating Data Lakehouse concepts for unified data management Designing APIs for diverse business more »

Founding Data Engineer Central London 3 days a week, hybrid NB – we are looking for someone with specific experience around fintech, finance, banking, eCommerce transactions, insurance, payments, fraud, consumer scams or related areas. Please only apply if you have specific/relatable experience. You should also be fine to … they have validated some of the big problems to solve. The mission is big. They have a new approach tested with retail banks. The data engineer will be key to bringing some of the solutions and the early stage product to life. The role > We’re looking for someone … that really fancies building a data platform from scratch alongside some very solid technical founders > Designing, building, and maintaining scalable data pipelines. > Collaborating with various teams to understand data requirements and implement solutions. > Optimizing data workflows and processes to enhance data quality, reliability, and performance. more »

Software Engineer (Data) £110,000-£120,000 London, Hybrid (2x days) Permanent role I am seeking an experienced Software Engineer (Data) who can be passionate about designing, optimising, and maintaining AI-native data processing pipelines. Your role will involve tackling complex challenges with robust data pipelines … across diverse sources, enhancing the quality of data for our systems, and readiness to meet the challenges of scale. Responsibilities: Manage cloud orchestration and data infrastructure, applying your expertise with various data processing and indexing tools to ensure scalability and reliability. Design robust data processing and … indexing pipelines handling large amounts of data across several modalities, ensuring robustness and timeliness. Curate and manage large-scale data ingestion and indexing pipelines, ensure data quality and error handling. Ensure the secure handling of data through end-to-end encryption, data isolation, and more »

Exciting opportunity to work with a Big4 Tech company as a Data Scientist, based in London! 12 months contract Key Qualifications: • Postgraduate degree (MS or PhD) in a quantitative field of study including economics, statistics, applied mathematics, data science, business, or a related discipline. • 5-7 years professional … experience developing advanced Tableau data visualizations, dashboards and reporting. • Programming skills in SQL and Python required to develop and maintain custom ETL processes. • Skills in exploratory data analysis and familiarity with advanced quantitative analysis. • Experience reporting on web analytics in Adobe, or other web analytics tools. • Comfortable using … modern data technologies. Familiarity with database modelling and data warehousing principles (Snowflake preferred). • Knowledge of the digital advertising industry, the mobile app economy or related field. • Familiarity with automated data ingestion processes including APIs. • Ability to partner effectively with data science and engineering stakeholders more »

Principal Data Engineer – National Salary Circa £80,000. London Salary Circa £100,000. Do you like working with the latest technology and are interested in enhancing your tech abilities? We have an exciting opportunity for an Engineer Manager with significant experience of managing data pipelines. As well as … being an expert in cloud platform, you’ll have a strong background in Data Ingestion and Integration, designing and implementing ETL pipelines on various technologies, Data Quality monitoring and a rounded understanding of data operations. Aviva believes strongly in experimentation leading to industrialisation and we are … individuals who are focussed on using their skills to drive out real business value for our customers. A bit about the job: At Aviva, data serves as the lifeblood that propels our organizations forward, and we are no exception. As a Principal Data Engineer within our esteemed Data more »

minded subject matter experts, solving interesting problems, and building the products of tomorrow, this opportunity is for you. On our mission to revolutionize the Data Strategy & Management space, we are seeking a dynamic and seasoned Solution Architect with a stellar track record in Data Products. As our client … navigates the complexities of a legacy IT landscape, we need someone who can not only conceptualize, design and build robust data platforms but also provide strategic guidance and stakeholder management expertise. Leveraging your software engineering expertise, you'll play a pivotal role in ensuring the integrity, efficiency, and compliance … of the data architecture while spearheading the migration from legacy systems to modern solutions. What you'll bring to us: At least 5 years of experience in a similar role, with 10 years in software engineering and solution architecture. Extensive knowledge of database structure systems, data mining, data more »

Job title: Lead Data Engineer Client: Data Driven Hedge Fund Salary: Up to £160,000 Location: London/Hybrid My client are seeking a Lead Data Engineer to own delivery of scalable data pipelines that are critical to the entire firms infrastructure. Being a Lead Data … cutting edge technologies and boast a world class technical team where you will work with and learn from the best. Responsibilities: As a Lead Data Engineer, you will engage with data scientists, software engineer's and product managers to deliver high-quality product based on very specifically tailored … data requirements. Architect and implement data ingestion, transformation, and storage processes. Optimise data pipeline performance. Keep up with market developments in the data engineering and AI space and implement these best practices. Requirements and Skills: Bachelor's Degree in Computer Science, Engineering (or other related more »

Data Architect - AWS Digital Consultancy £650-£700 per day (outside IR35) UK remote This is an exciting opportunity to join a digital consultancy that operate throughout Europe, predominately working with technology led businesses to build new products and platforms. They're about to embark on a project for a … customer in the automotive/EV industry and so are looking for a Data Architect to help lead on the design and build of a customer data platform. The platform will sit on AWS and they plan on using Snowflake as the data warehouse however you'll … however there will be the occasional requirement to meet with the team in London. What you'll do: Plan architecture and build infrastructure for data ingestion, processing, analytics, visualisations and business intelligence systems. Integrate AI/Machine learning inference as part of a data platform. Data more »

basis. Purpose: Supporting the Business and Finance Team you will be integral to establishing, testing and maintaining a comprehensive suite of reports, workflows and data models across all business areas. You will work with internal and external stakeholders to guide data strategy and best practice to ensure solutions … existing Finance reporting via NetSuite, Vantage & Sugar systems. - Suggest and implement ways to improve current Finance reporting processes. - Own and manage the company’s data systems such as Alteryx and Power BI. Manage users and data ingestion and be responsible for the data held and reports … generated. - Drive efficiency by identifying customer profitability and supplier billing queries. - Manage, Transform, and Interpret data, and analyze results using statistical techniques and provide ongoing reports. - Design and implement data analysis, data collection, and other processes that deliver the key objectives of your stakeholders. E.g. improving existing more »

A Financial Service Company is currently seeking a Lead Data Engineer to join its EMEA Technology team. This person will lead the design, development, and maintenance of scalable and efficient data pipelines to support data-driven applications The Lead Data Engineer will also collaborate with cross … functional teams, including data scientists, software engineers, an product managers, to understand data requirements and deliver high-quality solutions as well as architecting data ingestion, transformation, and storage processes using tools such as Apache Spark, Azure Data Factory, and other similar technologies. Other core duties … include optimizing data pipeline performance, ensuring data accuracy, reliability, and timely delivery. Requirements; Significant experience working as a Lead Data Engineer within Financial Services Certifications in relevant technologies, such as Azure Data Engineer or Databricks Certified Developer Experience with real-time data processing and streaming more »

customers Your world of expertise In this role you will be part of the Standalone Applications team who manage several key operational systems for data ingestion, data processing and product generation. For example; satellite data reception and radar data processing. These systems not only provide … essential data used for the models that run on the supercomputer, but also use model data from the supercomputer to produce essential products used by forecasters and customers. As our Application Operations Engineer, the key duties are: Provide support, as part of the team, to a range of more »

TwentyAI are supporting a lead commodity trading house with a hire needed in their London office. They are looking for a talented Data Engineer to work a number of greenfield initiatives as part of a trade monitoring department. This is an exciting opportunity to work within a fast-paced … and strategy of this team and play a leading role in the growth of the team. The role: Work on designing, implementing & continuously evolving data platform's including pipeline & repository Carry out data ingestion, curation, preparation and access required for systematic alpha ideas exploration, analytics & reporting Management … and ongoing development of an enterprise data platform built on Azure Development of new data products and services for commercial and operational teams Tech Stack Python and Pyspark SQL Pipelines, Big Data, Data architectures App Services, Functions, Cosmos DB, Databricks, Synpase and associated tools automation, PyTest more »

purpose of making the world a better and safer place. Element, a well-established organization that touches billions of lives, is searching for a Data Scientist in London (hybrid) to help build a startup within their organization. You will design and train performant models from scratch and create a … drive our product's analytics engine. Operating in an Agile setting, you will be part of a cross functional team, collaborating closely with other Data Scientists, Data Engineers, Software Engineers, Product Owners and UX/UI Designers, to innovate and drive our product to new heights. Core Responsibilities … learning models & design robust analytics engines for high-end digital products Create repeatable, interpretable, dynamic and scalable models that are seamlessly incorporated into analytic data products Document code, results, and details of the approaches in a systematic way to promote knowledge sharing and code re-use Collaborate with other more »

Data Engineer Overview: An Exciting opportunity has arisen with one of the UK’s largest life insurance brokers to join as a Data Engineer. Reporting to the Data Platform Manager, you will be responsible for ensuring that all of the raw data is prepared into clear … department to develop and maintain products Working cross-functionally across the business to understand the requirements of the products Designing and implementing performance related data ingestion pipelines from multiple sources using Apache Spark Integrating end-to-end data pipelines ensuring a high level of quality is maintained … Working with an Agile delivery/DevOps methodology to deliver additional data products Delivering and presenting proofs of concept of key technology components to the wider data team and relevant stakeholders Essential Skills & Experience: Strong experience with SQL & Python Competent with designing and delivering solutions using Azure services more »

I am partnered with an innovation & transformation consultancy that are on the lookout for an SC Cleared Data Engineer in their Bristol office. Join one of the UK's largest innovation & transformation consultancies dedicated to shaping the future of public sector engagements. We pride ourselves on delivering cutting-edge … solutions that drive impactful change and foster growth in the community. Key Responsibilities: Utilize your expertise in Azure to design, develop, and maintain data solutions tailored to the unique requirements of public sector projects. Engage in end-to-end data engineering processes, including data ingestion, transformation … Requirements: SC clearance is essential for this role. Bachelor's degree in Computer Science, Engineering, or related field. Minimum of 4 years' experience in data engineering roles, preferably within a consulting firm. Proficiency in Azure data services such as Azure SQL Database, Azure Data Factory, Azure Databricks more »

A new opportunity has risen for a Data and Insights Manager to join our D2C team. The Data and Insights team is a small but dynamic and experienced group of SMEs who work across Google Analytics, Bloomreach CDP and BI/data engineering. The team is the … personalised and segmented approach to every customer touchpoint to empower our customers to reach their financial goals. Plus, we know the best decisions are data-driven so we give our senior management team the best business intelligence data at their fingertips. Were looking for an experienced, driven and … Youll be responsible for managing the day-to-day work of the team, working with the team to triage incoming requests, gather requirements for data tasks (whether that be dashboards, new functionality, analysis pieces or tracking requests) and manage the teams backlog on JIRA. Youll take on running sprint more »