1 of 1 GitLab Jobs in Lanarkshire

teams to understand data requirements and translate them into efficient pipelines. Optimize performance and cost-efficiency of Databricks workloads. Develop and maintain CI/CD workflows for Databricks using GitLab or similar tools. Ensure data quality and reliability through robust unit testing and validation frameworks. Implement best practices for data governance, security, and access control within Databricks. Provide technical mentorship … data pipelines in cloud environments. Experience with AWS services such as S3, Glue, Lambda, Step Functions, Athena, IAM, and VPC. Proficiency in Python for data engineering tasks. Familiarity with GitLab for version control and CI/CD. Strong understanding of unit testing and data validation techniques. Preferred Qualifications: Experience with Databricks Delta Lake, Unity Catalog, and MLflow. Knowledge of CloudFormation More ❯