3 of 3 Luigi Jobs in England

models, schemas to support business requirements Develop and maintain data ingestion and processing systems using various tools and technologies, such as SQL, NoSQL, ETL, Luigi, Airflow, Argo, etc. Implement data storage solutions using different types of databases, such as relational, non-relational, or cloud-based. Working collaboratively with the client …/Azure SQL, PostgreSQL) You have framework experience within either Flask, Tornado or Django, Docker Experience working with ETL pipelines is desirable e.g. Luigi, Airflow or Argo Experience with big data technologies, such as Apache Spark, Hadoop, Kafka, etc. Data acquisition and development of data sets and improving data quality ...

agreement), confidence scoring, audit trail mindset Pipeline building : reproducible pre-processing and training pipelines (not necessarily Kubernetes), eg Python pipelines, Airflow/Prefect/Luigi-style orchestration, and good experiment hygiene Experiment tracking : MLflow and or Weights & Biases (runs, metrics, artefacts, model registry) Version control : Git, plus dataset/versioned ...

agreement), confidence scoring, audit trail mindset Pipeline building: reproducible pre-processing and training pipelines (not necessarily Kubernetes), eg Python pipelines, Airflow/Prefect/Luigi-style orchestration, and good experiment hygiene Experiment tracking: MLflow and or Weights & Biases (runs, metrics, artefacts, model registry) Version control: Git, plus dataset/versioned ...