distillation
34 posts tagged distillation.
- Agentic-OPD says your best RL checkpoint is not the whole model
- Grasp-TAGS picks the grasp before the robot commits
- Zyphra's tree-structured attention aims at million-token-plus context
- dSpark and the Order Problem in Diffusion Language Models
- A 0.6B Persian Embedding Model That Catches a 4B: What Yalda Teaches About Distillation
- Nanbeige4.3-33B is really a small-model bet
- ASTEROID teaches agents to explore before they act
- The Teacher That Scores Worse Than It Plays: Fixing Agentic Distillation
- Reasoning distillation works better when the trace fits the student
- SGUID says most skills are not worth distilling
- JOLT makes self-distillation follow the student’s path
- Multi-teacher distillation should copy the delta, not the whole teacher
- The Explorer and the Student: How ExpDis Splits RL Into Two Jobs
- Paradee makes single-voice TTS small enough for a laptop
- Pivot-SD trains the denoising choices that steer diffusion LMs
- GALA makes Gaussian avatars lighter by distilling animation into shared blendshapes
- AdviSD trains a small model to steer bigger frozen models
- For Linear ViTs, copy the MLP and distill the attention
- Reinforcement learning weakens reasoning distillation defenses
- UECR-GRPO treats the teacher as evidence, not an oracle
- Xiaomi's MiMo V2.6 Ships in Three Flavors, and the Split Matters
- When the Teacher Knows to Quit: RetireOPD and Distillation for Agents
- Distillation needs calibration when the teacher is biased
- OptiFlow treats offline RL policy learning as sample matching
- Garry Tan’s distillation argument is really about AI capability access
- On-policy distillation may need fewer prompts and better absorption
- On-policy distillation may be pruning tails, not teaching
- The token-budget bug hiding in multi-teacher distillation
- When the Teacher and the Verifier Disagree: Fixing On-Policy Distillation for Long Context
- A Strong Model Can Scaffold a Weak One Without Any Retraining
- Relay-OPD and the prefix failure problem in on-policy distillation
- OPD2 tries to distill reasoning by subtracting the base model
- Reusing RL Gains Across Model Sizes: The Case for Direct-OPD
- DemoPSD Treats Teacher Disagreement as a Training Signal