Mastodon Feed
Okay, so, Model Collapse:
1. are there any good articles or papers on it?
2. I am vaguely aware that it is "well known" that if you *directly* Ouroboros a model to itself it degrades pretty rapidly and dramatically (although, c.f. 1, I would really love a good citation for that) but is there any research on the *creeping* model collapse that we will experience as more and more *unlabeled* extruded text makes its way into the training corpus?