Post-Training

Beyond Static Interpretability: Anticipating Post-SFT Mechanisms from Pre-SFT Parameters for Better Tuning featured image

Beyond Static Interpretability: Anticipating Post-SFT Mechanisms from Pre-SFT Parameters for Better Tuning

Mechanistic Localization bridges mechanistic interpretability and post-training optimization by isolating critical parameters via interpretative approaches and then guiding …

hang-chen-jiaying-zhu-wenya-wang
•

Navigating by Old Maps: The Pitfalls of Static Mechanistic Localization in LLM Post-Training

The "Locate-then-Update" paradigm has become a predominant approach in the post-training of large language models (LLMs), identifying critical components via mechanistic …

hang-chen-jiaying-zhu-hongyang-chen-hongxu-liu-xinyu-yang-wenya-wang
•