IIRC it was observed in a training mode with weight decay. Perhaps a basin with ... | Hacker News

Hacker Newsnew | past | comments | ask | show | jobs | submit

killerstorm on June 3, 2024 | parent | context | favorite | on: Grokfast: Accelerated Grokking by Amplifying Slow ...

IIRC it was observed in a training mode with weight decay. Perhaps a basin with proper generalization is more stable.

Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact