Lead Data Platform Engineer - Outside IR35

Lead Data Platform Engineer - Outside IR35 - 1 day pw London (rest remote) - 3 months Initially

We are currently recruiting for a Lead Data Platform Engineer to join a programme focused on data platforms, analytics engineering, and modern Lakehouse solutions. The client is looking for a hands-on engineering professional with strong Microsoft Fabric, Spark, PySpark and SparkSQL expertise, capable of designing, building and supporting enterprise-scale data platforms.

Start Date: ASAP
Duration: 3 months Initially
Pay Rate: Competitive
IR35 Status: Outside IR35
Location: London (1 day per week onsite, remainder remote)

The Role:

The core requirement is strong Lakehouse expertise, particularly Spark, PySpark, and SparkSQL. Hands-on experience in low-code is essential.

Data & Analytics Expertise:

• Particularly relevant to your current Data Platform background:
• Data architecture (Lakehouse, Medallion, Kimball, 3NF).
• Data ingestion and processing.
• Data quality frameworks (Great Expectations, Soda).
• Data, BI and analytics platforms:

1. Microsoft Fabric
2. Purview
3. APIs
4. Streams

• Data governance and data literacy.
• KPI and semantic model design.
• AI and innovation in analytics.

Software Engineering:

Strong knowledge of:

• Design, Build, Test, Release, Deploy using Azure Fabric, data engineering tools.
• Secure coding practices.
• API design and integration patterns.
• Integrations using APIs, ADF for managing wide variety of data from structured, unstructured to semi.

DevSecOps, Cloud & Infrastructure:

• Cloud platforms: Azure, AWS
• CI/CD pipelines and automated deployments.
• Infrastructure as Code (IaC):

1. Terraform
2. Git actions

• Containerisation:

1. Docker

• Security practices:

1. Identity and access management
2. Secrets management

• Monitoring, logging, SLAs/SLOs.

Agile Delivery:

Strong experience in:

• Agile, Scrum, and Lean methodologies.
• Backlog management and roadmap planning.
• Technical cross team dependency management.
• Continuous improvement of delivery performance.

Development Tools:

Hands-on capability with:

• Pyspark, pytest, DAX, python, Azure functions, Lamda, and SQL.
• Git source control.
• Testing frameworks like great expectation, TDD, meta data driven.
• Build and dependency management tools.
• CI platforms.
• Terraform/Ansible (or similar).

Platform-Specific Expertise (Nice to have):

The role expects deep expertise in at least one of:

• APIs & Integrations
• DevSecOps & Cloud Platforms
• Application Development (React, JavaScript/ECMAScript, PWA)
• QA Automation (Playwright, Selenium)
• Data Engineering
• Reporting & Analytics

Technical Leadership & Architecture (Nice to have):

• Lead technical direction for products, platforms, and services.
• Define and promote software development standards, patterns, and governance.
• Drive architectural consistency and manage technical debt.
• Review, establish and improve product software development life cycles.

To apply for Lead Data Platform Engineer contract job, please click the button below and submit your latest CV.

Curo Services endeavours to respond to all applications, however this may not always be possible during periods of high volume. Thank you for your patience.

Curo Services is a trading name of Curo Resourcing Ltd and acts as an Employment Business for contract and temporary recruitment as well as an Employment Agency in relation to permanent vacancies.