Loading
Model Distillation is a technique in which a smaller "student" model is trained to reproduce the behaviour of a larger "teacher" model, transferring much of its capability into a more efficient form.
Model Distillation, a technique in which a smaller "student" model is trained to reproduce the behaviour of a larger "teacher" model, transferring much of its capability into a more efficient form.
Distillation makes models cheaper and faster to run, which is why distilled models are common in production. It carries governance and legal nuance: distilling from a third-party model may breach its terms of use or intellectual-property rights, and the student can inherit the teacher's biases and weaknesses while being harder to trace back to them.
Source: Machine-learning literature
Distillation makes models cheaper and faster to run, which is why distilled models are common in production. It carries governance and legal nuance: distilling from a third-party model may breach its terms of use or intellectual-property rights, and the student can inherit the teacher's biases and weaknesses while being harder to trace back to them.
See where you stand on AI governance
Take the free 7-question maturity assessment and get a personalised action plan.
Free assessment, 3 minutes →