Artificial Intelligence Engineer

Job Title: Artificial Intelligence Engineer (Databricks)

Rate: DOE (outside IR35)

Location: Remote

Contract Length: 6 months

A consultancy client of ours have secured a project requiring a Databricks focused Artificial Intelligence Engineer. This is an exciting opportunity to work on cutting-edge machine learning projects, building scalable ML pipelines and cloud-based systems that deliver real-world impact.

Key Responsibilities:

• Lead the design, development, and optimisation of scalable machine learning workflows using Azure Databricks
• Build and deploy robust ML pipelines leveraging Delta Lake, MLflow, notebooks, and Databricks Jobs
• Apply advanced knowledge of Databricks architecture and performance tuning to support production-grade ML solutions
• Collaborate with data scientists, data engineers, and analysts to operationalise machine learning models at scale
• Champion the use of Databricks-native features (e.g., Unity Catalog, MLflow Model Registry, AutoML) to improve model lifecycle management
• Migrate legacy model training and scoring workflows into unified Databricks-based pipelines
• Ensure best practices in model reproducibility, governance, monitoring, and security within the Databricks environment
• Act as a subject matter expert on Databricks ML capabilities, advising on architecture, tools, and integrations
• Mentor peers and junior engineers on ML engineering practices, with a focus on MLOps and Databricks workflows
• Continuously improve the machine learning platform, tooling, and deployment practices to accelerate delivery

Experience and Qualifications Required:

• Deep hands-on experience with Azure Databricks, particularly in developing and deploying machine learning solutions using Delta Lake, MLflow, and Spark ML/PyTorch/TensorFlow integrations
• Strong programming skills in Python (including ML libraries like scikit-learn, pandas, PySpark) and experience using SQL for data preparation and analysis
• Experience orchestrating end-to-end ML pipelines, including data preprocessing, model training, validation, and deployment
• Solid understanding of MLOps principles, including model versioning, monitoring, and CI/CD for ML workflows
• Familiarity with Azure cloud services, including Azure Data Lake, Azure Machine Learning, and Data Factory
• Experience with feature engineering, model management, and automated retraining in production environments
• Knowledge of data governance, security, and regulatory compliance in the context of ML workflows
• Strong problem-solving skills, with the ability to debug and optimise distributed ML pipelines
• Proven track record of delivering machine learning models in production within enterprise-scale environments
• Excellent communication and collaboration skills, with experience engaging both technical and business stakeholders
• Experience mentoring others and promoting best practices in ML engineering and Databricks usage

If this sounds like an exciting opportunity please apply with your CV.