1 to 25 of 65 Permanent Data Ingestion Jobs

Data Engineer Tech for good - up to £52,000 1 day a week - London My Client is in the tech for good space working to protect children globally. They are looking for a Data Engineer who will report directly to the Head of Data and is responsible … for building and maintaining of the Enterprise Data Platform, including data ingestion pipelines and data models. Responsibilities/skills required: Experience working the entire data lifecycle, building relevant data models and pipelines that meet business requirements Minimum of 3 years experience in a Data Engineering role Must have strong experience with Snowflake and SQL Implement data solutions making sure the data is accessible, accurate and up-to-date Strong experience with data management, data migration, data ingestion, data transformation, data storage, data quality, data more »

JOB ALERT FOR! We are currently recruiting for a LEAD DATA ENGINEER for one of our clients in the insurance industry. The Lead Data Engineer will be responsible for: Leading a team of developers and data engineers who build and deliver the components of modern Data platform – Data Lakehouse Architecture in Azure, leveraging data lake, delta Lake for data storage Utilizing Azure Data Factory for data integration Databricks for data transformation Data governance using purview Profisee for MDM Synapse and Power BI for end-user data analytics … layer and reporting. In the pipeline – Co Pilot and Fabric. Key responsibilities Lead data engineer responsible of: • Mixture of hands on and lead a newly established data engineering team through both strategic development projects and day-to-day data management (BAU) activities. o Defining and orchestrating tasks more »

Job description: hands-on experience in business intelligence development or data engineering experience,Solid experience with data modelling, data warehouse design and data lake concepts and practices; Exposure working in a Microsoft Azure Data Platform environment. Exposer inAzure Data Factory Azure Storage Data Ingestion and Storage including Azure Data Factory, Azure Databricks, Azure Data Lake, Kafka and Spark Streaming, Azure EventHub/IoT Hub, and Azure Stream Analytics • Building new data pipelines and optimizing data flows using the Azure cloud stack. • Building data products from scratch. … Support Business Analysts and Data Architects with discovery and best practices. • Self-directed and comfortable supporting the data needs of multiple teams, systems and products. What’s required ? • Advanced Azure knowledge and experience working and migrating data products from on-prem to Azure. • Experience building and optimizing more »

Our client, a prominent multi-strategy hedge fund, is seeking a talented Data Engineer to join their dynamic team in the London office. In this role, you will be a key contributor to building a robust data platform that services a wide range of investment teams and internal … clients. You will collaborate with data sourcing experts, data specialists, data scientists, and fellow data engineers to manage and deliver a diverse set of financial and alternative datasets and insights across asset classes and strategies. Responsibilities: Data Pipeline Design: Partner with the data strategy … and sourcing team to understand data requirements and design efficient data pipelines and delivery structures. Real-Time Data Capture: Develop components to enhance the existing platform for capturing real-time data. Data Ingestion and Evaluation: Work with vendors and technical teams to systematically ingest, evaluate more »

support of BTs Strategic SIEM development. We are seeking a skilled Software Engineer ideally with knowledge of Elasticsearch (or other SIEM and/or Data platforms) to join our dynamic team. As a Software Engineer you will play a critical role in designing, developing, and maintaining our security information … focus will be on leveraging Elasticsearch and related technologies to enhance threat detection, incident response, and overall security posture. What you’ll be doing • Data Ingestion and Enrichment: o Configure data pipelines for data ingestion from various sources, primarily from Kafka o Enhance data … work towards achieving these. You'll have the following experience • A strong background in Software Engineering, working on projects that involve large amounts of data • Understanding of ETL/Data pipelines Design, develop, and maintain data pipelines Automate data ingestion, transformation, and loading tasks. • Understanding more »

Contractor Rate: £400-£450 per day Duration: 6 months Number of Positions: 1 Role Description: As a database expert, you will design solutions for data ingestion, transform data, optimize data storage, and ensure data integrity. You will also structure data for easy consumption by … BI tools. This role significantly impacts the organization by leveraging cutting-edge technology to enhance data management and analysis. Key Responsibilities: Develop ETL processes to extract data from various sources, transform it into the desired format, and load it into Azure Data Lake. Transform raw data into structured datasets, create models, and manage data pipelines using DBT Lab. Create scalable data transformation processes using dbt models for analytics-ready data. Ensure data exposed in business tools is populated by version-controlled, quality-assured dbt models. Configure and manage databases in the cloud. more »

Job Title: Data Warehouse Developer Location: Rotherham Salary: Up to £40,000 Hours: Monday to Friday 9 am - 5.30 pm Benefits: 25-day annual leave entitlement, plus bank holidays. A Continuous Learning Program to keep you in the loop with relevant progression within the business and changes to economic … because we may consider covering the cost of your annual subscription/membership. Workplace Pension is provided by Scottish Widows About the position of Data Warehouse Developer Working for a leading home improvements lender driving positive market change, we have a newly created vacancy available for a Data Warehouse Developer, reporting to the Head of IT. You will be responsible for the design, development, and maintenance of the company's data warehouse. This role involves working closely with business stakeholders, data analysts, and the IT team to ensure the data warehouse meets business needs more »

office Permanent Key Responsibilities: As a Database Developer, you will: Design, develop, and maintain high-performance databases. Collaborate with cross-functional teams to understand data requirements. Optimize and enhance database performance. Troubleshoot and resolve database issues. Stay updated on the latest trends and technologies in database development. Requirements: Deliver … aspects of issues, reporting and development in an efficient and effective fashion, aligned to critical business timeframes and deadlines. Be an ambassador for the Data Warehouse team across the business through appropriate meetings around BAU and projects. Deliver analysis and development of business requirements through the following: Support ETL … architecture for overall data migration, data models, data mappings, business logic and data cleansing methods. Designing and scheduling complex SSIS Packages, PowerShell scripts for transferring data from multiple data sources to SQL Server. Data ingestion into the data warehouse using SSIS more »

week. Key Responsibilities : As a Azure Database Developer, you will: Design, develop, and maintain high-performance databases. Collaborate with cross-functional teams to understand data requirements. Optimize and enhance database performance. Troubleshoot and resolve database issues. Stay updated on the latest trends and technologies in database development. Requirements: Deliver … aspects of issues, reporting and development in an efficient and effective fashion, aligned to critical business timeframes and deadlines. Be an ambassador for the Data Warehouse team across the business through appropriate meetings around BAU and projects. Deliver analysis and development of business requirements through the following: Support ETL … architecture for overall data migration, data models, data mappings, business logic and data cleansing methods. Designing and scheduling complex SSIS Packages, PowerShell scripts for transferring data from multiple data sources to SQL Server. Data ingestion into the data warehouse using SSIS more »

our vision. Key Responsibilities : As a Database Developer, you will: Design, develop, and maintain high-performance databases. Collaborate with cross-functional teams to understand data requirements. Optimize and enhance database performance. Troubleshoot and resolve database issues. Stay updated on the latest trends and technologies in database development. Requirements: Deliver … aspects of issues, reporting and development in an efficient and effective fashion, aligned to critical business timeframes and deadlines. Be an ambassador for the Data Warehouse team across the business through appropriate meetings around BAU and projects. Deliver analysis and development of business requirements through the following: Support ETL … architecture for overall data migration, data models, data mappings, business logic and data cleansing methods. Designing and scheduling complex SSIS Packages, PowerShell scripts for transferring data from multiple data sources to SQL Server. Data ingestion into the data warehouse using SSIS more »

week. Key Responsibilities : As a Azure Database Developer, you will: Design, develop, and maintain high-performance databases. Collaborate with cross-functional teams to understand data requirements. Optimize and enhance database performance. Troubleshoot and resolve database issues. Stay updated on the latest trends and technologies in database development. Requirements: Deliver … aspects of issues, reporting and development in an efficient and effective fashion, aligned to critical business timeframes and deadlines. Be an ambassador for the Data Warehouse team across the business through appropriate meetings around BAU and projects. Deliver analysis and development of business requirements through the following: Support ETL … architecture for overall data migration, data models, data mappings, business logic and data cleansing methods. Designing and scheduling complex SSIS Packages, PowerShell scripts for transferring data from multiple data sources to SQL Server. Data ingestion into the data warehouse using SSIS more »

days a week. As a Database Developer, you will: Design, develop, and maintain high-performance databases. Collaborate with cross-functional teams to understand data requirements. Optimize and enhance database performance. Troubleshoot and resolve database issues. Stay updated on the latest trends and technologies in database development. Requirements of the … aspects of issues, reporting and development in an efficient and effective fashion, aligned to critical business timeframes and deadlines. Be an ambassador for the Data Warehouse team across the business through appropriate meetings around BAU and projects. Deliver analysis and development of business requirements through the following: Support ETL … architecture for overall data migration, data models, data mappings, business logic and data cleansing methods. Designing and scheduling complex SSIS Packages, PowerShell scripts for transferring data from multiple data sources to SQL Server. Data ingestion into the data warehouse using SSIS more »

Senior Data Engineer About us HENI is a technology company pioneering art markets and information. We work with world-leading artists and estates across various sectors, including printmaking, book publishing, NFTs, digital content, and art research – all underpinned by cutting-edge research in analytics, AI, and machine learning. Our … team comprises of talented web developers, data engineers, data scientists, and infrastructure specialists who thrive on solving complex problems. We enjoy developing machine learning tools and embracing the latest in infrastructure and coding methodologies. We maintain high standards of technical quality via thorough testing and a dedication to … and commercial decisions are driven by this innovative collection of art market data. Role overview We are seeking a highly skilled and experienced Senior Data Engineer. You would be joining a strong team of roughly 20 people (software and data engineers, data scientists, analysts), in a fast more »

we do things differently. We re focused on our core values using these we ve seen significant growth across our practices and our Digital, Data and Cloud team is preparing for the next phase of expansion. This creates new opportunities for driven and skilled individuals to join one of … the fastest-growing consultancies globally. Job Description As a Data Engineer you will join a team delivering transformative cloud hosted data platforms for a Version 1 customers. You will have a proven track record as a Data Engineer and be adept in your ability to implement data ingestion and transformation pipelines for large scale organisations. This career may align to individuals who like to work with a variety of technologies and data services, that will develop and deliver early proofs of concept and production implementation. On a day to day basis you'll: Design more »

we do things differently. We re focused on our core values using these we ve seen significant growth across our practices and our Digital, Data and Cloud team is preparing for the next phase of expansion. This creates new opportunities for driven and skilled individuals to join one of … the fastest-growing consultancies globally. Job Description As a Data Engineer you will join a team delivering transformative cloud hosted data platforms for a Version 1 customers. You will have a proven track record as a Data Engineer and be adept in your ability to implement data ingestion and transformation pipelines for large scale organisations. This career may align to individuals who like to work with a variety of technologies and data services, that will develop and deliver early proofs of concept and production implementation. On a day to day basis you'll: Design more »

client are a tech for good business dedicated to promoting transparency in the corporate world they are seeking a highly skilled and experienced Lead Data Engineer. You will be the first hire in the Data Engineering team allowing you to take the drivers seat regarding the direction taken … by the data team to follow. Key Responsibilities: Platform Development: Design, develop, maintain, test, and optimise a scalable and reliable data platform for collecting, processing, and transforming large volumes of company data from various sources. Collaboration: Work with cross-functional teams to understand data platform requirements … and deliver solutions that meet business needs. Optimisation: Enhance data storage and retrieval systems for high performance and data integrity. Quality Assurance: Implement and maintain a data quality and observability framework to ensure data accuracy, completeness, and consistency. Documentation: Develop and maintain comprehensive documentation for data more »

Job Description: Data Engineer (Specialist/Professional) Work Location: London/Birmingham, UK Platform: GCP Position Overview: We are seeking a skilled and experienced GCP Data Engineer to join our team. As a GCP Data Engineer, you will be responsible for designing, implementing, and maintaining our data infrastructure and pipelines on the Google Cloud Platform (GCP). Key Responsibilities: Design & Development: Design and implement data ingestion pipelines, data processing workflows, and data storage solutions using GCP services such as Cloud Storage, Big Query, Dataflow, Composer etc. Work with other DEs to integrate … data solutions into existing or new applications. Develop and manage data integration processes to extract data from various sources, transform and cleanse it, and load it into appropriate data storage systems. Implement robust ETL processes to ensure data quality and consistency. Data Modeling and more »

Working with a top tier Digital Asset Management firm who are looking for an adept and experienced Data Engineer to play a critical role in shaping and managing their data engineering platform. This role demands strong technical skills and a proven ability to work closely with quantitative researchers … ensuring seamless integration of data engineering processes to support our quantitative investment strategies. Key Responsibilities: Collaborate closely with quantitative researchers to understand their data needs, offer technical solutions, and effectively integrate data engineering with quantitative strategies. Design, develop, and optimize robust data pipelines, facilitating ETL processes … that support quantitative research, analysis, alpha forecasting, and execution. Implement stringent data quality assurance measures to ensure the accuracy, reliability, and consistency of data inputs and outputs. Evaluate, select, and manage appropriate data engineering tools, technologies, and infrastructure, with a focus on scalability and automation. Work with more »

market. They are a dynamic financial service business that provides innovative solutions in credit risk assessment and sustainable finance. This newly created role of Data Warehouse Developer reports to the Head of IT and is responsible for the design, development, and maintenance of the company s data warehouse. … This role involves working closely with business stakeholders, data analysts, and the IT team to ensure the data warehouse meets business needs and performance standards. Responsibilities: Design and develop data models and schemas for the data warehouse. Build and maintain ETL (Extract, Transform, Load) processes to … integrate data from various sources. Develop complex SQL queries and scripts to ensure efficient data retrieval and manipulation. Responsible for the build and deployment of data warehouse capabilities. Ensure data quality and integrity through validation and cleansing processes. Implement data warehousing best practices for data more »

An exciting opportunity for an experienced Data Engineer to join the Engineering team working within the Digital & Product Department. Location: Chiswick (West London) Work Type: 3 days in office, 2 day from home Salary: £80k - £90k dependent on experience The successful candidate will lead a team of 2 data engineers to design and create scalable data pipelines. This hands-on role involves discussing requirements, designing solutions, and coding to automate data ingestion, cleansing, and transformation. You'll collaborate with Products, Engineering, Data, and Business Analytics teams to understand data requirements and develop integration … solutions for necessary migrations. The Data Engineer will support software developers, PowerBI developers, data analysts, and data scientists, ensuring a consistent and optimal data delivery architecture across projects. Self-direction and comfort in supporting multiple teams, systems, and products are essential. DUTIES & RESPONSIBILITIES Understand, build and more »

of a successful, dynamic, and creative global business then we want to hear from you. The Opportunity: We have an exciting opportunity for a Data Warehouse Architect to join a rapidly growing data team in Clarion. The role will report into the Senior Architect, with responsibility to analyse … to overcome complex challenges, and grow your design through developing skills with the new initiatives we are looking to explore. Key Responsibilities: Analyse existing data design and suggest improvements that promote performance, stability, and interoperability. Work with product management and business subject matter experts to translate business requirements into … good database design. Work with internal and external resources to develop robust Extract/Transform/Load (ETL) processes for disparate data sources. Strong knowledge of setting up and maintaining synapse pipelines within Azure. Good knowledge of acquiring data from various Rest, Soap, SFTP, Landing Zones and obtaining more »

market. They are a dynamic financial service business that provides innovative solutions in credit risk assessment and sustainable finance. This newly created role of Data Warehouse Developer reports to the Head of IT and is responsible for the design, development, and maintenance of the company’s new data warehouse. This role involves working closely with business stakeholders, data analysts, and the IT team to ensure the data warehouse meets business needs and performance standards. Responsibilities: Design and develop data models and schemas for the data warehouse. Build and maintain ETL (Extract, Transform, Load … processes to integrate data from various sources. Develop complex SQL queries and scripts to ensure efficient data retrieval and manipulation. Responsible for the build and deployment of data warehouse capabilities. Ensure data quality and integrity through validation and cleansing processes. Implement data warehousing best practices more »

benefits Please note: you need to have the right to work in the UK no sponsorship, unfortunately About the job Reporting to the Senior Data & Analytics Lead, the BI Developer will be responsible for driving the direction and maintaining the business data model ensuring it aligns with business … activities and processes. The role will work closely with key internal and external stakeholders in organising and creating a data model that makes data and information accessible with appropriate channels of access controls. Fostering a relationship of trust and playing an advisory role in the use of data to improve business performance. The role will require playing a highly collaborative role by overseeing cross-functional data governance while ensuring the adoption and adherence to data quality are understood by stakeholders. Key Responsibilities Responsible for designing effective, efficient data models to support various types of more »

Title: Principle Data Engineer Location: Hybrid - Once per Month Travel to office in North West. Salary: £75,000 + Package Role Overview: The Principal Data Engineer holds a key position within the Data Engineering function, reporting directly to the Head of Data & Platform Engineering. This role … requires a deep understanding of Google Cloud Platform to ensure that all production data processes are reliable, efficient, and compliant. Key responsibilities include overseeing data ingestion, quality, integrity, modeling, warehousing, transformation, security, encryption, monitoring tools, and batch management. The Principal Data Engineer will: Identify opportunities for … automation and process improvement. Mentor and guide data engineers. Establish coding standards and best practices. Implement and document data integrity and quality checks. Optimize queries. Facilitate collaboration within the data engineering team. This role involves close collaboration with the Principal Cloud Engineer and the Data Architect more »

Solution Architect (Presales) Remote (Travel for client visit) Permanent As a Solution Data Architect, you will be responsible for designing, implementing, and overseeing comprehensive data solutions that align with our business objectives. Your expertise in AI, cloud technologies, Databricks, data lakes, and data strategy will be … pivotal in driving our data initiatives forward. Additionally, you will play a key role in pre-sales activities, collaborating with clients to understand their needs and propose tailored solutions. Key Responsibilities: Architect and Design: Develop robust data architectures and strategies leveraging AI, cloud platforms, Databricks, and data lakes to support business goals. Data Strategy: Define and implement data strategies that align with business objectives and ensure data integrity, security, and scalability. Cloud Integration: Lead the integration of cloud services (AWS, Azure, GCP) with on-premises data systems, ensuring seamless data flow more »