8 of 8 Contract Data Ingestion Jobs in England

Systems Architecture - AWS data engineer Job Description Location: UK - Will require travel to customer site ](Belfast) Job Summary: We are seeking a skilled and experienced AWS Data Engineer to join our team. The successful candidate will be responsible for implementing, and managing data architecture solutions for our customers, with a strong emphasis on Cloud technologies (Preferably AWS … and tooling. This role requires understanding of data modelling, database design, and data integration techniques. Key Responsibilities -Data Ingestion and Extraction: Design and implement efficient data ingestion pipelines to and from databases and file storage services. -Data Transformation and Cleaning: Transform and clean raw data to ensure data quality and consistency. … Data Pipelines: Build, maintain, and optimise data pipelines to automate data flows and enable real-time data processing. -Data Quality Assurance: Monitor data quality and implement measures to ensure data accuracy and completeness. -Database Administration: Manage and maintain databases (e.g., SQL, NoSQL) to ensure optimal performance and security. -Cloud Infrastructure: Deploy and manage More ❯

We are seeking a skilled Azure DevOps Engineer with hands-on experience in Terraform , Kubernetes , and cloud-native data architecture to help build and maintain scalable infrastructure in Microsoft Azure. This role is focused on automating infrastructure provisioning, deploying containerized applications, and building a secure and efficient cloud data lake environment to support large-scale data ingestion … CI/CD pipelines using Azure DevOps. Manage and provision cloud infrastructure using Terraform (Infrastructure as Code). Deploy and manage Kubernetes clusters (AKS) and containerized workloads. Work with data engineering teams to design and implement a cloud-based data lake architecture on Azure. Automate data pipeline deployments and integrations using Azure services (e.g., Data Factory … in DevOps or Cloud Engineering roles. Strong proficiency with Terraform and managing IaC deployments. Experience managing Kubernetes (AKS preferred) in production environments. Experience designing or supporting a cloud-based data lake (Azure Data Lake Gen2, Blob Storage, etc.). Proficient in scripting languages such as PowerShell , Bash , or Python . Experience with CI/CD tools, especially Azure More ❯

DV (MOD) Cleared Data Engineer - Elastic Stack & Apache NiFi Location: Bristol | Contract Type: £430.00 pd (Outside IR35) | Working Pattern: Hybrid (3 - 4 days on-site) Are you a contract Data Engineer with a knack for designing secure, high-performance data solutions? We're on the lookout for a technical expert in the Elastic Stack and Apache NiFi … to take the lead in building robust, real-time data pipelines in a security-focused environment. This is a hands-on contract opportunity to make a real impact-ideal for professionals with a strong track record in regulated sectors. What You'll Be Doing Designing and deploying scalable, secure data pipelines using Elasticsearch, Logstash, Kibana, and Apache NiFi … Handling real-time data ingestion and transformation with an emphasis on integrity and availability Collaborating with architects and cybersecurity stakeholders to align with governance and compliance needs Monitoring and optimising high-throughput data flows across on-prem and cloud environments Building insightful Kibana dashboards to support business intelligence and operational decision-making Maintaining documentation of data More ❯

Job Title: Data Engineer (NPPV3 + SC Clearance Required)Location: Hybrid (UK-based)Contract: 3 Months (with potential extension/move to other projects)Rate: £550 per day (Outside IR35)A bout the Role:We are seeking an experienced Data Engineer with NPPV3 + SC clearance to join our dynamic team on a 3-month contract basis. This … role offering the flexibility of working both remotely and onsite. The contract has potential for extension or transition onto other exciting projects. Key Responsibilities:* Design, build, and maintain scalable data pipelines using Azure Data Factory, Databricks, and Azure Synapse.* Collaborate with cross-functional teams to ensure data quality and integration.* Support data ingestion, transformation, and … storage in cloud environments.* Troubleshoot and optimise data workflows for performance and reliability.* Maintain compliance with security protocols and clearance requirements. Essential Skills & Experience:* Must hold NPPV3 + SC clearance (this is a mandatory requirement).* Proven expertise in Azure Data Factory, Databricks, and Azure Synapse Analytics.* Strong experience in building and managing cloud-based data solutions. More ❯

Data Pipeline Development: Design and implement end-to-end data pipelines in Azure Databricks, handling ingestion from various data sources, performing complex transformations, and publishing data to Azure Data Lake or other storage services. Write efficient and standardized Spark SQL and PySpark code for data transformations, ensuring data integrity and accuracy across … the pipeline. Automate pipeline orchestration using Databricks Workflows or integration with external tools (e.g., Apache Airflow, Azure Data Factory). Data Ingestion & Transformation: Build scalable data ingestion processes to handle structured, semi-structured, and unstructured data from various sources (APIs, databases, file systems). Implement data transformation logic using Spark, ensuring data is cleaned, transformed, and enriched according to business requirements. Leverage Databricks features such as Delta Lake to manage and track changes to data, enabling better versioning and performance for incremental data loads. Data Publishing & Integration: Publish clean, transformed data to Azure Data Lake or other cloud storage solutions for consumption by analytics and reporting More ❯

Data Pipeline Development: Design and implement end-to-end data pipelines in Azure Databricks, handling ingestion from various data sources, performing complex transformations, and publishing data to Azure Data Lake or other storage services. Write efficient and standardized Spark SQL and PySpark code for data transformations, ensuring data integrity and accuracy across … the pipeline. Automate pipeline orchestration using Databricks Workflows or integration with external tools (e.g., Apache Airflow, Azure Data Factory). Data Ingestion & Transformation: Build scalable data ingestion processes to handle structured, semi-structured, and unstructured data from various sources (APIs, databases, file systems). Implement data transformation logic using Spark, ensuring data is cleaned, transformed, and enriched according to business requirements. Leverage Databricks features such as Delta Lake to manage and track changes to data, enabling better versioning and performance for incremental data loads. Data Publishing & Integration: Publish clean, transformed data to Azure Data Lake or other cloud storage solutions for consumption by analytics and reporting More ❯

you will play a key role in ensuring that critical business services have the capacity and resilience to meet regulatory and operational demands. You will focus on developing robust data pipelines and solutions that support capacity monitoring, reporting, and forecasting across complex IT environments. Role Overview: Develop ETL processes to extract and normalise capacity data from infrastructure sources. … Automate data ingestion and processing into SQL Server databases. Normalising raw infrastructure and monitoring data into structured formats suitable for graphing and trending Collaborate with DBAs and technical teams to ensure transparency and supportability. Build data flows that support capacity reporting and trending, using modern technologies. Skills & Experience Required: 3+ years' experience in ETL/data processing roles. Proficiency with ETL tools, SQL Server, RESTful APIs. Experience working with infrastructure data (e.g., Nutanix, VMware, Dell EMC desirable). Scripting experience in Python, SQL (T-SQL), Excel VBA. Knowledge of data optimisation, large datasets, JSON/XML formats. Experience of setting up SQL DTSS and Control/M batch jobs useful Excellent communication, analytical More ❯

the Braze platform, ensuring it aligns with business objectives and integrates seamlessly with our existing marketing technology stack. End-to-end implementation of Braze, including SDK integration, API connections, data ingestion, and the configuration of campaigns, Canvases, and Content Cards. Serve as the primary technical consultant for all things Braze, providing strategic advice and best practice recommendations to … marketing, product, and engineering teams. Collaborate with stakeholders to understand their needs and translate them into technical requirements and actionable use cases within Braze. Work closely with data and engineering teams to design and manage data flows, ensuring the right data is available in Braze to power personalised customer journeys. Essential: Hands-on experience with the Braze … architecting and leading at least two full-cycle Braze implementations. Proven experience with Docker, Kubernetes, Terraform, or similar IaC technologies Have experience with MongoDB, Redis, Kafka, Postgres, or similar data technologies Experience with data integration, including ETL processes and customer data platforms (CDPs). Comprehensive awareness of various marketing technology platforms like CRM, DAM, dynamic creative optimization More ❯