distillation
5 posts tagged distillation.
- A Strong Model Can Scaffold a Weak One Without Any Retraining
- Relay-OPD and the prefix failure problem in on-policy distillation
- OPD2 tries to distill reasoning by subtracting the base model
- Reusing RL Gains Across Model Sizes: The Case for Direct-OPD
- DemoPSD Treats Teacher Disagreement as a Training Signal