Distillation will allow sophisticated models to run in production by reducing their dimensions and latency, whilst preserving most of the performance of much larger, more computationally costly models. It has been made use of to boost Google Look for and Smart Summary for Gmail, Chat, Docs, and much