15 of 15 Data Pipeline Jobs in Sheffield

award-winning team at Sparta Global. About this role You'll become versatile in a wide array of tools across topics covering data focused coding, data visualisation, Cloud Services, Big Data and Machine learning. You will be equipped to work within a variety … roles including but not limited to; Data Engineer, Data Analyst, Data Scientist, Information Engineer. You'll be designing, building, maintaining, and troubleshooting the data pipelines that enable organizations to store, process, and analyse their data, and ensuring the data ...

Role: Lead Data Engineer - Azure/Databricks Location: Sheffield Working Model: Hybrid - 2 days per week in person Salary: Up to £85k depending on experience Own the data platform. Shape the future architecture. This role is for a hands-on data engineering leader … wants to build something properly, not babysit legacy pipelines. Youll take full ownership of the data engineering domain, leading the design and building of a modern, highly scalable data platform that handles complex, high-frequency industrial data. Youll work closely with data scientists, software ...

Senior Data Platform Engineer £90,000-£105,000 Remote (Quarterly meets in London) This is a standout opportunity to work on high-scale data challenges, designing the next generation of a cutting-edge analytics and AI platform. You will join a small, authentic, highly collaborative engineering … from day one. THE COMPANY This organisation operates in the digital analytics and advertising technology space, helping clients understand online behaviours and turn data into actionable insight. They build and maintain a modern data platform and the applications that sit on top of it, used ...

Lead Data Engineer Location: Hybrid (60% Office/40% Remote) Sheffield Contract: 6 Month Contract (Extension possible) Rate: £400+ per day inside IR35 Role Overview We are seeking an experienced Lead Data Engineer to design, develop, and optimise enterprise-scale data platforms for large … banking, financial services, or other regulated sectors. This role requires hands-on technical expertise, leadership, and a consulting mindset to deliver scalable, resilient data solutions while promoting best practices and operational excellence. Key Responsibilities Lead the design, development, and optimisation of enterprise data engineering platforms Build ...

Role Title: Lead Data Engineer Location: Sheffield/hybrid (3 days on site) Duration: 9 months Rate: 430 per day inside ir35 We are seeking a Lead Data Engineering Consultant with proven experience in leading and developing data engineering platforms. Experience required: Extensive enterprise … Hadoop, Spark, and Splunk. Proficiency in object-oriented and functional scripting, particularly in Python. Skilled in handling raw, structured, semi-structured, and unstructured data (SQL and NoSQL). Experience integrating large, disparate datasets using modern tools and frameworks. Strong background in building and optimizing ETL/ELT data ...

Role Title: Lead Data Engineer Location: Sheffield/hybrid (3 days on site) Duration: 9 months Rate: £430 per day inside ir35 We are seeking a Lead Data Engineering Consultant with proven experience in leading and developing data engineering platforms. Experience required: Extensive enterprise … Hadoop, Spark, and Splunk. Proficiency in object-oriented and functional scripting, particularly in Python. Skilled in handling raw, structured, semi-structured, and unstructured data (SQL and NoSQL). Experience integrating large, disparate datasets using modern tools and frameworks. Strong background in building and optimizing ETL/ELT data ...

Lead Data Platform Engineer £75,000 - £85,000 Remote You are a well-experienced Lead Data Platform Engineer, looking to take ownership of a modern lakehouse platform and shape best-in-class data engineering practices. THE COMPANY This industry-leading organisation serves specialist, people … first knowledge to its clients and is looking to provide further expertise by improving its data capabilities. THE ROLE As a Lead Data Engineer you will build a Databricks platform with ownership, shaping how data is engineered, deployed, automated, and governed in Azure. Specifically ...

Role Title: Lead Data Engineering Consultant Duration: contract to run until 21/11/2026 Location: Sheffield, Hybrid 3 days per week onsite Rate: up to 423.78 p/d Umbrella inside IR35 Role purpose/summary We are seeking a Lead Data Engineering Consultant … with proven experience in leading and developing data engineering platforms. The ideal candidate will possess hands-on expertise in the following areas: Extensive enterprise experience with Hadoop, Spark, and Splunk. Proficiency in object-oriented and functional scripting, particularly in Python. Skilled in handling raw, structured, semi-structured ...

Role Title: Lead Data Engineering Consultant Duration: contract to run until 21/11/2026 Location: Sheffield, Hybrid 3 days per week onsite Rate: up to £460 p/d Umbrella inside IR35 Role purpose/summary We are seeking a Lead Data Engineering Consultant … with proven experience in leading and developing data engineering platforms. The ideal candidate will possess hands-on expertise in the following areas: Extensive enterprise experience with Hadoop, Spark, and Splunk. Proficiency in object-oriented and functional scripting, particularly in Python. Skilled in handling raw, structured, semi-structured ...

Data Engineer (Contract) 9+ Month Contract based in Sheffield £395 - £442 per day InsideIR35 BPSS clearance required - candidates must be eligible My client is seeking a Data Engineer to design and operate large-scale telemetry and observability data pipelines within a modern OpenShift … observability, delivering high-quality metrics, logs, and traces to support platform reliability, operational insight, and automation. Responsibilities: * Design, implement and maintain scalable data pipelines to ingest and process OpenShift telemetry (metrics, logs, traces) * Stream telemetry through Kafka (producers, topics, schemas) and build resilient consumer services for enrichment ...

Title: Lead Data Engineer - Hadoop, Spark, Pytthon Location: Sheffield - 3 days per week in the office Salary/Rate: Up to £617 per day inside IR35 Start Date: 02/03/2026 Job Type: Contract until November We are seeking a Lead Data Engineering Consultant … with proven experience in leading and developing data engineering platforms. The ideal candidate will possess hands-on expertise in the following areas: Extensive enterprise experience with Hadoop, Spark, and Splunk. Proficiency in object-oriented and functional scripting, particularly in Python. Skilled in handling raw, structured, semi-structured ...

seeking a Lead Data Engineering Consultant with proven experience in leading and developing data engineering platforms. The ideal candidate will possess hands-on expertise in the following areas: Extensive enterprise experience with Hadoop, Spark , and Splunk. Proficiency in object-oriented and functional scripting, particularly in Python … . Skilled in handling raw, structured, semi-structured, and unstructured data (SQL and NoSQL). Experience integrating large, disparate datasets using modern tools and frameworks. Strong background in building and optimizing ETL/ELT data pipelines . Familiarity with source control and implementing Continuous Integration, Delivery ...

that blends deep industry expertise with modern technology to shape the future of digital services Your new role We are seeking a Lead Data Engineering Consultant with proven experience in leading and developing data engineering platforms. The ideal candidate will possess hands-on expertise … Hadoop, Spark, and Splunk. Proficiency in object-oriented and functional scripting, particularly in Python. Skilled in handling raw, structured, semi-structured, and unstructured data (SQL and NoSQL). Experience integrating large, disparate datasets using modern tools and frameworks. Strong background in building and optimizing ETL/ELT data ...

Responsibilities: Design, implement, and maintain data pipelines to ingest and process OpenShift telemetry (metrics, logs, traces) at scale. Stream OpenShift telemetry via Kafka (producers, topics, schemas) and build resilient consumer services for transformation and enrichment. Engineer data models and routing for multi-tenant observability; ensure lineage … proactive insights). Implement schema management (Avro/Protobuf), governance, and versioning for telemetry events. Build automated validation, replay, and backfill mechanisms for data reliability and recovery. Instrument services with OpenTelemetry; standardize tracing, metrics, and structured logging across platforms. Use LLMs to enhance observability capabilities (e.g., query assistance ...

extend) Location: Remote Day Rate: £550 (Outside IR35) Start Date: ASAP We’re looking for a Microsoft Fabric Architect to lead a greenfield data platform build within a growing organisation looking to modernise its data estate. You’ll take ownership of the end-to-end architecture … solution through to production. Key Experience Required Strong hands-on experience with Microsoft Fabric Experience building end-to-end Lakehouse solutions Good data modelling skills (e.g. star schema) Experience using PySpark or Spark SQL Building data pipelines from APIs, ERP or SaaS systems Working with Fabric ...