Job Title: Platform Engineer Databricks Modernisation
Leeds, West Yorkshire, Yorkshire, United Kingdom
Hybrid / WFH Options
Hybrid / WFH Options
Unolabs
for 8,000+ test cases, notebooks, and services. CI/CD Pipeline Development Develop and enhance GitLab CI/CD pipelines and integrate them with Jenkins where applicable. Enable Docker-in-Docker strategies for GitLab runners to support test and integration pipelines, while considering container security risks and mitigations. Modify and maintain Jenkins pipelines to deploy to both environments, ensuring … consistent test coverage across shared and core repos. Dockerization & Service Management Build and publish Docker images for: Databricks compute environments. Supporting microservices such as Broker, Scheduler, and Status Monitor. Push and manage Docker images in AWS ECR (Elastic Container Registry) and integrate them with GitLab CI/CD runners and ECS-based compute in CDP. Notebook & Pipeline Migration Migrate critical … including workspace setup, instance profiles, notebook CI/CD, and Delta Lake capabilities. Hands-on with Terraform for infrastructure provisioning and GitLab/Jenkins for pipeline orchestration. Experience with Docker, container image lifecycle, and secure CI/CD container practices (Docker-in-Docker understanding is a bonus). Proficiency in scripting (Python, Bash) and YAML-based CI/CD pipeline More ❯
Employment Type: Contract, Work From Home
Posted: