1 to 25 of 107 Data Ingestion Jobs

Come build your future, while being the reason millions of people find a brighter financial future with Discover. Job Description: T he Lead Business Data Analyst role is crucial in helping the company understand and improve customer satisfaction (CX). They will be responsible for supporting the customer satisfaction … and escalates risk and customer-impacting issues within the day-to-day role to management. Responsibilities: Partner with business and technology teams to develop data applications. Identify business data ingestion and processing frameworks. Coordinates and obtains data application requirements from the business. Translates business requirements to … development teams. Assist with development and testing processes. Coordinate data application and model deployments and validations. Assists with the development of models, analytic processes, and reports. Provides guidance on the development of data consumption processes. Ensures data governance policies are followed by implementing and validating data more »

BI Developer/Data Engineer required for a Global Software provider, offering cloud based solutions. Joining a Data Warehouse (DW) and Business Intelligence (BI) team to support a range of growth. The role will be to support the BAU activities which includes the ingestion of multiple data sources to the main DW and the preparation of datasets for reporting requests. Primary types of data will be financial, CRM, telephony & support desk, with others as the need arises. The work is diverse, every report is different, every acquisition uses different source systems with different processes & terminologies. … Resulting in a need to be creative, inventive and reactive to how different data sources are managed and prepared etc. Main Responsibilities Data ingestion from multiple sources. Excel/FlatFile (csv/txt) Web (JSON/XML) SQL Databases Direct Integrations Support testing of existing & new dataset more »

Data Engineer - Manchester - £35,000 - Hybrid Working My client is hiring a full time Data Engineer to contribute to their expanding data science and analytics initiatives. Your role involves designing, building, and maintaining data pipelines, ensuring seamless data flow into their central Microsoft Azure repository. … Collaborate with Data Analysts and Scientists to optimise data quality, reliability, security, and automation. Skills & Responsibilities: Configure and troubleshoot Microsoft Azure, manage data ingestion Develop ETL scripts using Python, handle web scraping, APIs, and relational databases Familiarity with Power BI (you will not have to use … Power BI, but being aware of how to collaborate with people that do is important) Work closely with data analysts and scientists, contribute to data models and algorithms, and participate in data events Working knowledge of Git If you have the required skills and experience and would more »

London office on a hybrid basis. As a Contract Senior Golang Developer, you will play a crucial role in assessing, analysing, and enhancing our data ingestion, transformation, and storage layers. Your primary focus will be on Go Programming and DBT, with secondary skills in Google Cloud Services. The … successful candidate will bring 5 to 10 years of experience in developing robust data pipelines, conducting unit testing, and providing support for production deployments. Key Responsibilities: Assess and analyse data ingestion, transformation, and storage layers, including RAW, Exploratory, Curated, and application layers. Understand existing codebase and make … alterations or fixes as per design requirements. Develop and optimize data ingestion pipelines using Go Programming and DBT. Conduct unit testing and validation of the developed code to ensure reliability and performance. Collaborate with cross-functional teams to support production deployments and resolve any issues that may arise. more »

Job Title: Senior Data Engineer About Luupli Luupli is a social media app that has equity, diversity, and equality at its heart. We believe that social media can be a force for good, and we are committed to creating a platform that maximizes the value that creators and businesses … up of passionate and dedicated individuals who are committed to making Luupli a success. Job Description: We are seeking a highly skilled and motivated Data Engineer with expertise in designing and implementing robust data pipelines on the Amazon Web Services (AWS) platform. As a Data Engineer at … Luupli, you will play a critical role in developing and maintaining a scalable and efficient data infrastructure. You will work closely with cross-functional teams, including data scientists and software engineers, to ensure the smooth flow of data throughout our organisation. Responsibilities: Design, build, and maintain a more »

Senior Data Engineer About us HENI is a technology company pioneering art markets and information. We work with world-leading artists and estates across various sectors, including printmaking, book publishing, NFTs, digital content, and art research – all underpinned by cutting-edge research in analytics, AI, and machine learning. Our … team comprises of talented web developers, data engineers, data scientists, and infrastructure specialists who thrive on solving complex problems. We enjoy developing machine learning tools and embracing the latest in infrastructure and coding methodologies. We maintain high standards of technical quality via thorough testing and a dedication to … and commercial decisions are driven by this innovative collection of art market data. Role overview We are seeking a highly skilled and experienced Senior Data Engineer. You would be joining a strong team of roughly 20 people (software and data engineers, data scientists, analysts), in a fast more »

we do things differently. We’re focused on our core values; using these we’ve seen significant growth across our practices and our Digital, Data and Cloud team is preparing for the next phase of expansion. This creates new opportunities for driven and skilled individuals to join one of … the fastest-growing consultancies globally. Job Description This is an exciting opportunity for an experienced developer of large-scale data solutions. You will join a team delivering a transformative cloud hosted data platform for a key Version 1 customer. The ideal candidate will have a proven track record … as a senior/self-starting data engineer in implementing data ingestion and transformation pipelines for large scale organisations. We are seeking someone with deep technical skills in a variety of technologies, specifically SPARK performance\tuning\optimisation and Databricks, to play an important role in developing and more »

client are a tech for good business dedicated to promoting transparency in the corporate world they are seeking a highly skilled and experienced Lead Data Engineer. You will be the first hire in the Data Engineering team allowing you to take the drivers seat regarding the direction taken … by the data team to follow. Key Responsibilities: Platform Development: Design, develop, maintain, test, and optimise a scalable and reliable data platform for collecting, processing, and transforming large volumes of company data from various sources. Collaboration: Work with cross-functional teams to understand data platform requirements … and deliver solutions that meet business needs. Optimisation: Enhance data storage and retrieval systems for high performance and data integrity. Quality Assurance: Implement and maintain a data quality and observability framework to ensure data accuracy, completeness, and consistency. Documentation: Develop and maintain comprehensive documentation for data more »

The role As a Data Engineer at Storm you will work alongside other software and cloud infrastructure engineers to design, build and maintain data pipelines, data management solutions and analytics components for our client’s projects across the healthcare, justice and local government sectors. About you We … know that many Data Engineer's will come from a variety of backgrounds, and we encourage and value people who bring unique perspectives and knowledge to our team. Your previous experience will include working daily with disparate data sources, ingesting, transforming and processing data streams into actionable … use or for onward processing. You will have a proven track record of delivery and a passion for adherence to best practices in the data engineering discipline. You will be passionate about data in all its forms and see the value in transforming data into insights. You more »

Senior Data Engineer Up to £70k plus bonus Manchester Are you looking to take your Data Engineer career to the next level?? This company use extremely modern technologies, and you can be certain you will grow within a technical environment. The career advancement and progression opportunities for you … focus is building, planning, and constructing pipelines, in a technical role, working predominantly within the Azure Cloud. Your new challenge Working alongside the Lead Data Engineer, you’ll be working within a small, friendly team, working on exciting technical projects. You will be using your technical expertise with Microsoft … Azure to conduct data ingestion, extractions, and transformations in Data Lake, and developing new database structures and data warehouse strategies. This company need someone who can work on business procedures and data models, ensuring that any necessary changes or improvements are applied. About you First more »

is a prominent player within the Lloyd’s (Insurance) Market Syndicate, embarking on a complex transformation and distribution programme they are looking for a Data Architect to play an integral role in data architecture, integration/data life cycle management. You must have experience in: · Knowledge of … Data and Cloud platform technologies utilised by the Lloyd’s (Insurance) Market. · Proven track record of successful delivery of greenfield data initiatives. · Substantial experience of designing and optimising data models using cloud technologies. · Proven track record of successful delivery of MS Azure cloud technology initiatives. Knowledge in … Blueprint Two (B2P). Strong experience in data modelling, data ingestion and data governance (BI) · Gravitas and presence to provide confidence to key insurance business and technology stakeholders. · Excellent communicator, both written and verbally. · Strong organisation skills. · Experience in greenfield warehouse projects. more »

Senior Azure Data Engineer - 6 Month Contract - Inside IR35 This is an Azure Data Engineer 6 month contract position for a financial services end client helping them on delivering and implementing new data pipelines. You will need prior financial services experience as well as having strong Azure … Data Factory and Databricks experience. Work Involved: Implementing and delivering data pipelines Providing mentorship to junior members of the team Working with the Platform team defining capabilities for the lake framework Developing business conceptual data models Producing new frameworks using design patterns and data ingestion Requirements: Strong Data Pipeline experience Databricks Azure Data Factory Spark Azure cloud Financial Services If you're interested get in touch ASAP more »

Senior Data Engineer 💰 £85,000-£100,000 🖥️ Snowflake, AWS, dbt, Terraform 🌍 London, 2-3 days a week in London office We are partnered with one the UK's most exciting e-commerce platforms, operating within the luxury beauty and wellness industry, and supporting them on their search for a … Senior Data Engineer. The role... As a Senior Data Engineer you'll be at the forefront of transforming and interpreting complex data into valuable insights that drive the business. Your expertise will be crucial in leveraging key technologies like Snowflake, AWS, DBT, and Terraform to enhance understanding … of customer behaviors, market trends, and operational efficiencies. What you'll be doing... You will gather, analyse, and transform data from diverse sources, ensuring cleanliness and quality. You will be architecting and building the next generation Data Platform. You will construct data models, pipelines, and stores to more »

framework. Core java and Spring framework hands on experience is must. Strong understanding of database design and feature engineering to support huge volume of data on premises and external cloud. Should have an experience as “Data and Cloud Architect”. Should be able to design the large-scale … cloud native applications. Must have got hands on experience in Google Cloud. Setup reliable data ingestion pipelines for new data sources and integrate them effectively with existing data sets. Hands on experience in at least one programming language. Writing POCs and defining strategic platforms for “Future … experience in Spring framework Working experience in Google Cloud tools. Like for example – Looker Studio Experience in performance optimizations of applications which processes large data (TB scale more »

their field for over two decades. Building upon their continued success, they are launching the search for an Analytics Engineer within a 10-strong Data team to help support the development of their BI reporting and Data Platform. This will be a business-facing role, with a …/50 split between BI reporting/analysis and data ingestion, and would suit someone with a couple years experience across Python , ETL, AWS and any BI tool (PowerBI, Tableau, Qlik, QuickSight, or GoodData - it's more about the transferable skills!). This is an ideal role for more »

are high-performing and scalable Leverage Azure services to create and manage cloud-based applications (SQL DB, App Services, Azure FrontDoor, SignalR) Create efficient data ingestion and indexing pipelines using Elasticsearch Design, develop, and maintain efficient, reusable, and reliable C# code Implement necessary infrastructural changes for successful application more »

and have huge influence on the strategic and technical direction of the system moving forward. Responsibilities Design, build, and maintain software for real-time data ingestion, processing, and presentation. Collaborate with quants, traders, and developers to understand needs and deliver effective solutions. Ensure data accuracy and clarity more »

re proud to be one of the largest personal insurers in the UK. Role Description We have a fantastic opportunity for a Senior Geospatial Data Scientist to join our fast Growing and hugely successful Data Science team, made up of over 50 talented and diverse Data Engineers … striving to work with the best talent and technologies in order to stay at the forefront to beat the best. As a Senior Geospatial Data Scientist, you will lead the development of geospatial solutions to solve complex problems across business functions, leveraging your expertise in geospatial data analysis … geospatial datasets. Develop predictive machine learning models, evaluate model performance, iterate model improvements to enhance accuracy. Assist in the design and implementation of scalable data processing pipelines for geospatial data ingestion, processing & storage. Stay up to date with the latest advancements in geospatial analytics, machine learning & data more »

re proud to be one of the largest personal insurers in the UK. Role Description We have a fantastic opportunity for a Senior Geospatial Data Scientist to join our fast Growing and hugely successful Data Science team, made up of over 50 talented and diverse Data Engineers … striving to work with the best talent and technologies in order to stay at the forefront to beat the best. As a Senior Geospatial Data Scientist, you will lead the development of geospatial solutions to solve complex problems across business functions, leveraging your expertise in geospatial data analysis … geospatial datasets. Develop predictive machine learning models, evaluate model performance, iterate model improvements to enhance accuracy. Assist in the design and implementation of scalable data processing pipelines for geospatial data ingestion, processing & storage. Stay up to date with the latest advancements in geospatial analytics, machine learning & data more »

re proud to be one of the largest personal insurers in the UK. Role Description We have a fantastic opportunity for a Senior Geospatial Data Scientist to join our fast Growing and hugely successful Data Science team, made up of over 50 talented and diverse Data Engineers … striving to work with the best talent and technologies in order to stay at the forefront to beat the best. As a Senior Geospatial Data Scientist, you will lead the development of geospatial solutions to solve complex problems across business functions, leveraging your expertise in geospatial data analysis … geospatial datasets. Develop predictive machine learning models, evaluate model performance, iterate model improvements to enhance accuracy. Assist in the design and implementation of scalable data processing pipelines for geospatial data ingestion, processing & storage. Stay up to date with the latest advancements in geospatial analytics, machine learning & data more »

Principal Data Engineer - Northwest £70,000 to £75,000 VIQU are seeking Principal Data Engineer to join a leading social enterprise working to transform lives by developing and managing vibrant communities. They excel in placemaking, regeneration, development, investment, property management, and leisure services. With a dedicated team across … numerous specialist subsidiaries, they create thriving environments and reinvest profits to enhance community projects, ensuring sustainable and flourishing communities. The successful Principal Data Engineer will be responsible for ensuring robust and compliant data processes on Google Cloud Platform. The key responsibilities of the role will include data ingestion, quality, modelling, transformation, security, and monitoring. The Principal Data Engineer will also be involved in automation, process improvement, mentoring engineers, setting coding standards, and optimising queries. The Principal Data Engineer will work in collaboration with the Principal Cloud Engineer and Data Architect, as it more »

maintenance of our Kafka ecosystem, ensuring its scalability, reliability, and performance to meet the evolving needs of our organization. You will collaborate closely with data engineers, software developers, and other stakeholders to architect robust solutions and streamline data pipelines. Your key responsibilities will include: Installing, configuring, and managing … health, performance metrics, and throughput, and proactively identifying and addressing potential bottlenecks or issues. Implementing security measures, access controls, and encryption protocols to safeguard data privacy and integrity within the Kafka ecosystem. Managing Kafka topics, partitions, replication, and consumer groups, and optimizing configurations for efficient resource utilization and high … availability. Collaborating with cross-functional teams to design and implement data ingestion pipelines, Real Time processing workflows, and event-driven architectures. Performing capacity planning, scaling, and disaster recovery planning to ensure scalability, fault tolerance, and business continuity. Automating routine tasks, such as cluster provisioning, deployment, monitoring, and alerting more »

driving the uptake and success of electric vehicles with GRIDSERVE Car Leasing. Everything we do is underpinned and connected by a technology-agnostic, proprietary data platform that grows and improves every day. Get in touch ROLE OVERVIEW Join GRIDSERVE as the Head of Data. This will be a critical … member of our leadership team, responsible for establishing and driving the data vision and strategy across the company. This role involves harnessing the power of data to optimize our operations, enhance decision-making, and unlock innovative solutions in the renewable energy space. You will lead our journey towards … becoming a data-mature organisation, building a team to leverage data for operational excellence, strategic insights, and market innovations.You will be able to communicate complex data concepts in simple business language, cutting out the jargon and getting to the point of what really matters for the decision more »

we do things differently. We’re focused on our core values; using these we’ve seen significant growth across our practices and our Digital, Data and Cloud team is preparing for the next phase of expansion. This creates new opportunities for driven and skilled individuals to join one of … the fastest-growing consultancies globally. Job Description As a Data Engineer you will join a team delivering transformative cloud hosted data platforms for a Version 1 customers. You will have a proven track record as a Data Engineer and be adept in your ability to implement data ingestion and transformation pipelines for large scale organisations. This career may align to individuals who like to work with a variety of technologies and data services, that will develop and deliver early proofs of concept and production implementation. On a day to day basis you'll: Design more »

we do things differently. We’re focused on our core values; using these we’ve seen significant growth across our practices and our Digital, Data and Cloud team is preparing for the next phase of expansion. This creates new opportunities for driven and skilled individuals to join one of … the fastest-growing consultancies globally. Job Description As a Data Engineer you will join a team delivering transformative cloud hosted data platforms for a Version 1 customers. You will have a proven track record as a Data Engineer and be adept in your ability to implement data ingestion and transformation pipelines for large scale organisations. This career may align to individuals who like to work with a variety of technologies and data services, that will develop and deliver early proofs of concept and production implementation. On a day to day basis you'll: Design more »