51 to 75 of 162 Data Ingestion Jobs in England

Day Rate: £450 - £550 P/day Start: Immediate Start Note: Capital Markets, or at least Banking experience is a MUST! Databricks Data Engineer - Contract Day Rate: £450 - £550 P/day Location: Hybrid (Remote/2 Days a week in London) Start: Immediate Start Note: Capital Markets, or … at least Banking experience is a MUST! Key expertise and experience we're looking for: Data Engineering in Databricks: Spark programming with Scala, Python, SQL Ideally experience with Delta Lake Databricks workflows, jobs, etc. Familiarity with Azure Data Lake: experience with data ingestion and ETL/… ELT frameworks Data Governance experience - Metadata, Data Quality, Lineage, Data Access Models Good understanding of Data Modelling concepts, Data Products and Data Domains Unity Catalog experience is a key differentiator - if not then experience with a similar Catalog/Data Governance Management component More ❯

GIS Data Engineer 35k to 50k Bath Hybrid I am working with a tech for good client who is looking for a data engineer, with skills in geospatial engineering for some exciting and large scale projects that they have underway. They are looking for skills in Python, PostgreSQL … someone that really enjoys problem solving and technical challenges. Benefits: 25 days holiday Company phone Health insurance Life insurance And more Role Responsibilities: Developing data pipelines Managing databases Analysing and visualizing GIS data Developing python scripting to automate data ingestion Testing and validating data Data manipulation and cleansing Creating tiles from datasets You will be working with a cross functional team including data analysts, developers, and other key stakeholders. Experience: Degree in Computer Science, GIS, Geography or related field Experience in data engineering Geospatial related software experience is highly desirable Skills: Python More ❯

GIS Data Engineer 35k to 50k Bath Hybrid I am working with a tech for good client who is looking for a data engineer, with skills in geospatial engineering for some exciting and large scale projects that they have underway. They are looking for skills in Python, PostgreSQL … someone that really enjoys problem solving and technical challenges. Benefits: 25 days holiday Company phone Health insurance Life insurance And more Role Responsibilities: Developing data pipelines Managing databases Analysing and visualizing GIS data Developing python scripting to automate data ingestion Testing and validating data Data manipulation and cleansing Creating tiles from datasets You will be working with a cross functional team including data analysts, developers, and other key stakeholders. Experience: Degree in Computer Science, GIS, Geography or related field Experience in data engineering Geospatial related software experience is highly desirable Skills: Python More ❯

GIS Data Engineer 35k to 50k Bath Hybrid I am working with a tech for good client who is looking for a data engineer, with skills in geospatial engineering for some exciting and large scale projects that they have underway. They are looking for skills in Python, PostgreSQL … someone that really enjoys problem solving and technical challenges. Benefits: 25 days holiday Company phone Health insurance Life insurance And more Role Responsibilities: Developing data pipelines Managing databases Analysing and visualizing GIS data Developing python scripting to automate data ingestion Testing and validating data Data manipulation and cleansing Creating tiles from datasets You will be working with a cross functional team including data analysts, developers, and other key stakeholders. Experience: Degree in Computer Science, GIS, Geography or related field Experience in data engineering Geospatial related software experience is highly desirable Skills: Python More ❯

delivery, and operation of all AWS global infrastructure. In other words, we're the people who keep the cloud running. We support all AWS data centers and all of the servers, storage, networking, power, and cooling equipment that ensure our customers have continual access to the innovation they rely … own them to completion. AWS Infrastructure Supply Chain (AIS-SC) organization works to deliver innovative solutions to source, build and maintain our socially responsible data center supply chains. We are a team of highly-motivated, engaged, and responsive professionals who enable the core sustainable infrastructure of AWS. Come join … a part of history as we deliver results for the largest cloud services company on Earth! AWS is seeking a highly motivated and passionate Data Engineer who is responsible for designing, developing, testing, and deploying Supply Chain Analytical Solutions. In this role you will collaborate with business leaders, work More ❯

part of an agile team that's constantly pushing the envelope to enhance, build, and deliver top-notch technology products. As a Senior Lead Data Lake Engineer at JPMorgan Chase within the Corporate Technology, Chief Data and Analytics Office Development Team, you will play a pivotal role in … technologies (AWS or GCP) via hands on experience or certification Solid understanding of software applications and technical processes within a related technical discipline (e.g. data ingestion, data storage, data serving, APIs, etc.). Hands-on experience in data lake or data warehouse and related More ❯

AWS Data Engineer (Must hold current SC) The Company: At Amber Labs, we are a cutting-edge UK and European technology consultancy that prioritises empowering autonomy, promoting experimentation, and facilitating rapid learning to provide exceptional value to our clients. Our company culture is centred around collaboration, where all colleagues … delivery times. Our team consists of highly-skilled cross-functional consultants, analysts, and support staff. Overview: We are looking for a highly skilled AWS Data Engineer to join our team. The ideal candidate will have expertise in designing, building, and managing scalable data pipelines on AWS. You will … work closely with data scientists, analysts, and software engineers to implement efficient data solutions that support business intelligence and analytics needs. Responsibilities: Design, develop, and maintain scalable ETL/ELT pipelines using AWS services such as AWS Glue, Lambda, Step Functions, and Kinesis. Work with structured and unstructured More ❯

REQUIRED: DATA ENGINEER - SC CLEARED LOCATION: FULLY REMOTE WITH OCCASIONAL TRAVEL TO CENTRAL LONDON IR35 STATUS: INSIDE DURATION: 6 MONTH INITIAL CONTRACT Seeking a Data Engineer who has a strong understanding of data concepts - data types, data structures, schemas (both JSON and Spark), schema management … etc - Strong understanding of complex JSON manipulation - Experience working with Data Pipelines using a custom Python/PySpark frameworks - Strong understanding of the 4 core Data categories (Reference, Master, Transactional, Freeform) and the implications of each, particularly managing/handling Reference Data. - Strong understanding of Data Security … principles - data owners, access controls - row and column level, GDPR etc including experience of handling sensitive datasets - Strong problem solving and analytical skills, particularly able to demonstrate these intuitively (able to work a problem out, not follow a work instruction to resolve) - Experience working in a support role would More ❯

Job Title: Azure Data Engineer Lead - Outside IR35 Location: Remote Salary/Rate: £450 per day - Outside IR35 Start Date: 28/04/2025 Duration: 3 months We have an exciting opportunity with a major retail company. They are looking for a Lead Azure Data Engineer to … working outside IR35 and fully remote. Job Responsibilities/Objectives: Lead the design and execution of the end-to-end migration from on-premise data systems to Azure, ensuring scalability, reliability, and performance. Develop and optimise data pipelines using Azure Data Factory, Synapse Analytics, and related services … to support data ingestion, transformation, and integration. Architect and implement Azure-based data solutions, including Data Lake, SQL Database, and Databricks, aligning with business and technical requirements. Establish cloud data engineering best practices, including governance, security, monitoring, and cost optimisation. Collaborate cross-functionally with cloud More ❯

REQUIRED: DATA ENGINEER - SC CLEARED Location: Fully remote with occasional travel to Central London IR35 Status: Inside Duration: 6-month initial contract Job Description: Seeking a Data Engineer with a strong understanding of data concepts, including data types, data structures, schemas (JSON and Spark), and … schema management. Key Skills and Experience: Strong understanding of complex JSON manipulation Experience with Data Pipelines using custom Python/PySpark frameworks Knowledge of the 4 core Data categories (Reference, Master, Transactional, Freeform) and handling Reference Data Understanding of Data Security principles, access controls, GDPR, and … advanced proficiency, Pydantic bonus) SQL PySpark Delta Lake Bash Git Markdown Scala (bonus) Azure SQL Server (bonus) Technologies: Azure Databricks Apache Spark Delta Tables Data processing with Python PowerBI (Data ingestion and integration) JIRA Additional Notes: Candidates with current or past high-level security clearance are encouraged More ❯

Data Engineer 4 months - with extensions Remote Active SC clearance required £640 per day inside ir35 REQUIRED Strong understanding of data concepts - data types, data structures, schemas (both JSON and Spark), schema management etc Strong understanding of complex JSON manipulation Experience working with Data Pipelines … using a custom Python/PySpark frameworks Strong understanding of the 4 core Data categories (Reference, Master, Transactional, Freeform) and the implications of each, particularly managing/handling Reference Data. Strong understanding of Data Security principles - data owners, access controls - row and column level, GDPR etc including … both CLI usage and scripting) Git Markdown Scala DESIRABLE Azure SQL Server as a HIVE Metastore DESIRABLE TECHNOLOGIES Azure Databricks Apache Spark Delta Tables Data processing with Python PowerBI (Integration/Data Ingestion) JIRA If you meet the above requirements, please apply for the vacancy to be More ❯

concepts and advanced applied experience Extensive development experience using Python/SQL Solid understanding of software applications and technical processes within related disciplines (e.g., data ingestion, data storage, data serving, APIs) Hands-on experience with data lake or data warehouse technologies (e.g., Spark, ETL More ❯

This is an exciting contract opportunity for an SC Cleared Azure Data Engineer to join an experienced team in a new customer engagement working at the forefront of data analytics and Artificial Intelligence. Our client equips ambitious organisations with enduring control of their data and AI evolution. … This role is an opportunity to lead the build of bespoke data systems for our clients. Responsibilities: Design and implement scalable data pipelines and ETL processes using Azure and Databricks technologies including Delta Live Tables. Lead technical discussions with clients and stakeholders to gather requirements and propose solutions. … Help clients realise the potential of data science, machine learning, and scaled data processing within Azure/Databricks ecosystem. Mentor junior team members and support their personal development. Take ownership for the delivery of core solution components. Support with planning, requirements refinement and work estimation. Skills and Experiences More ❯

ECOM are pleased to be exclusively recruiting for a Senior Data Engineer here in Manchester. You'll join a team where your work reaches millions. This role is within a forward-thinking company leading, offering a dynamic environment where you can grow, make an impact, and shape the future … of technology. The Role: As a Data Engineer, you'll design, develop, and optimise data pipelines on a Databricks-based platform, Symphony. You'll collaborate with teams to create scalable data solutions, ensuring efficient data ingestion, transformation, and analysis. Responsibilities: Design and maintain scalable data pipelines and infrastructure using Databricks. Collaborate with commercial teams to define data product requirements and visualisations. Develop and optimise ETL/ELT processes for smooth data flow and accurate analysis. Ensure adherence to data architecture principles and support the development of data dashboards. Monitor and More ❯

Senior Data Engineer | London, Hybrid | £70,000-£90,000 DOE My client is growing fast and looking for a skilled Data Engineer to help shape the backbone of their data capabilities. You’ll play a key role in refining infrastructure, ensuring seamless data access, and supporting … a modern data platform built for scale. Working closely with teams across the business, you’ll help drive smarter decisions and long-term success through well-structured, reliable, and secure data systems. Key Responsibilities: Define and implement a modular data architecture designed to scale with business growth … Develop and maintain a cloud-based data warehouse using AWS to manage high-volume data ingestion Design robust, high-throughput data pipelines that scale with increased demand. Automate ETL/ELT processes and integrate multiple data sources to ensure real-time availability Promote high data More ❯

Senior Data Engineer | London, Hybrid | £70,000-£90,000 DOE My client is growing fast and looking for a skilled Data Engineer to help shape the backbone of their data capabilities. You’ll play a key role in refining infrastructure, ensuring seamless data access, and supporting … a modern data platform built for scale. Working closely with teams across the business, you’ll help drive smarter decisions and long-term success through well-structured, reliable, and secure data systems. Key Responsibilities: Define and implement a modular data architecture designed to scale with business growth … Develop and maintain a cloud-based data warehouse using AWS to manage high-volume data ingestion Design robust, high-throughput data pipelines that scale with increased demand. Automate ETL/ELT processes and integrate multiple data sources to ensure real-time availability Promote high data More ❯

foster continuous improvement, and ensure our moderation efforts align with broader business objectives. What You Will Do: Conduct in-depth analysis of UGC moderation data to identify trends, patterns, and optimization opportunities, translating complex insights into actionable recommendations. Collaborate with cross-functional teams like product, engineering and vendor management … as well as with external industry partners to stay current on emerging moderation practices and implement data-driven solutions. Champion a data-driven culture by fostering collaboration, knowledge sharing, and best practices that elevate moderation quality, efficiency, and scalability. Develop and implement new methodologies to measure the impact … of moderation decisions, using advanced analytics and big data techniques to inform strategic direction. Generate and maintain reports, dashboards, and data pipelines by consolidating multiple data sources, programming algorithms, and automating processes to produce actionable insights. Work on diverse-scope problems where analysis requires evaluating numerous factors More ❯

Data Engineer - SC Clearance Leeds (Hybrid and Flexible Working Available) 65K + Bonus **🔐 MUST HAVE ACTIVE SC CLEARANCE Join a leading digital intelligence organisation working at the forefront of data-driven transformation for the public sector. We are looking for a Data Engineer with active SC Clearance … to help evolve a successful proof-of-concept into a fully productionised and supported data platform using cutting-edge Microsoft Azure technologies. 🔍 What You’ll Be Doing: Over the past four years, our client has partnered with Mobilise to explore and validate a modern data warehouse and business … intelligence ecosystem. This journey included a full proof-of-concept, showcasing the potential of Microsoft’s data stack to unlock actionable insights and improve internal reporting capabilities. We are now entering the next phase: transforming this proof-of-concept into a robust, scalable, and secure production environment. You’ll More ❯

Data Engineer - SC Clearable Leeds (Hybrid and Flexible Working Available) 65K + Bonus **?? MUST BE ELIGIBLE FORSC CLEARANCE Join a leading digital intelligence organisation working at the forefront of data-driven transformation for the public sector. We are looking for a Data Engineer with active SC Clearance … to help evolve a successful proof-of-concept into a fully productionised and supported data platform using cutting-edge Microsoft Azure technologies. ?? What You'll Be Doing: Over the past four years, our client has partnered with Mobilise to explore and validate a modern data warehouse and business … intelligence ecosystem. This journey included a full proof-of-concept, showcasing the potential of Microsoft's data stack to unlock actionable insights and improve internal reporting capabilities. We are now entering the next phase: transforming this proof-of-concept into a robust, scalable, and secure production environment. You'll More ❯

JOB SUMMARY Senior Data Engineer DV Cleared £500 - £600 per day - Outside IR35 JOB DESCRIPTION This role requires strong expertise in building and managing data pipelines using the Elastic Stack (Elasticsearch, Logstash, Kibana) and Apache NiFi. The successful candidate will design, implement, and maintain scalable, secure data … This is a UK based onsite role with the option of compressed hours. The role will include: Design, develop, and maintain secure and scalable data pipelines using the Elastic Stack (Elasticsearch, Logstash, Kibana) and Apache NiFi. Implement data ingestion, transformation, and integration processes, ensuring data quality … and security. Collaborate with data architects and security teams to ensure compliance with security policies and data governance standards. Manage and monitor large-scale data flows in real-time, ensuring system performance, reliability, and data integrity. Develop robust data models to support analytics and reporting More ❯

Job Description: Engineering Lead - GCP Data Key Responsibilities: 12+ years of overall IT experience with 10+ years in building data warehouse/datamart solutions. Experience in implementing an end-to-end data platform for analytics on cloud from ingestion to datamart. 4+ years of experience in … Query, GCS. 4+ years of experience in Python; 2+ years of experience in PySpark. Knowledge of GCP Kubernetes would be an advantage. Knowledge of data warehousing concepts with a good understanding of dimensional models. Experience in implementing a metadata framework for data ingestion, data quality, and More ❯

What is Human Native? At Human Native, we're building an AI data marketplace that ensures creators and rights holders are fairly compensated for their work while providing AI developers with high-quality, responsibly licensed training data. We believe in building AI the right way - ensuring transparency, fairness, and … us solve it. The Opportunity As an ML Engineer, you'll help us index, benchmark, and evaluate training datasets at scale. Your expertise with data, AI and ML training methodologies and evaluation techniques will advance the state of the art for developing AI. You will work across: Designing and … developing benchmarks that allow our customers to understand their value of data for training ML (quantifying dataset quality and biases). Deploy these benchmarks by implementing end-to-end data evaluation pipelines to be run on different datasets and ML models. Tools to visualise, analyze, and understand the More ❯

Python - ETL - Systematic Trading - HFT - StatArb - SQL A $25BN AUM statarb/HFT equities focused hedge fund client are scaling their front office facing data team to support incoming portfolio manager/researcher hires. This Data/Software Engineer will be working on data ingestion, data management, application development and the extension of their state of the art research data platform. As well as working on ad hoc projects for the front office to help them use data correctly and augment their investment process. Ideally looking for 1-5 years experience, excellent STEM … degree and competency in both data engineering and software engineering. STRONG Python a MUST. Tech: Python, Pandas, ETL, SQL, Spark, AWS Please apply for more information More ❯

Python - ETL - Systematic Trading - HFT - StatArb - SQL A $25BN AUM statarb/HFT equities focused hedge fund client are scaling their front office facing data team to support incoming portfolio manager/researcher hires. This Data/Software Engineer will be working on data ingestion, data management, application development and the extension of their state of the art research data platform. As well as working on ad hoc projects for the front office to help them use data correctly and augment their investment process. Ideally looking for 1-5 years experience, excellent STEM … degree and competency in both data engineering and software engineering. STRONG Python a MUST. Tech: Python, Pandas, ETL, SQL, Spark, AWS Please apply for more information More ❯

BI Developer, an expert in Power BI and its related tools, to take ownership of our client's Power Reporting Platform during a significant data warehouse migration project. This role requires someone to leverage their hands-on Power BI experience to advise stakeholders on data ingestion and … advanced BI solutions tailored to stakeholder requirements is essential. Experience developing custom BI products using SQL, R, and Python is highly desirable. Knowledge of data integration, data warehousing, data modeling, and data presentation concepts is also important. Additional Skills Extensive report writing experience using Tableau or More ❯