silence7@slrpnk.net to Technology@lemmy.worldEnglish · 8 hours agoAI Companies Are Buying Tons of Old Books Because They're Free of AI Slopwww.404media.coexternal-linkmessage-square57fedilinkarrow-up1234
arrow-up1234external-linkAI Companies Are Buying Tons of Old Books Because They're Free of AI Slopwww.404media.cosilence7@slrpnk.net to Technology@lemmy.worldEnglish · 8 hours agomessage-square57fedilink
minus-squareDeacon@lemmy.worldlinkfedilinkEnglisharrow-up9·2 hours agoYou can only stave off model collapse so long. It’s only a matter of time before most of the data going in to LLMs came out of one too. Like I’m not even good at math or anything, much less computer science, but that seems obvious and inevitable to me. Maybe someone smarter than me can chime in.
minus-squareUairhahs@lemmy.worldlinkfedilinkEnglisharrow-up4·edit-256 minutes agoIt’s kind of sort of already happening. Emdashes are fragments of it, a token saving strategy introduced by already trained on old books in their data sources. Model collapse is inevitable.
You can only stave off model collapse so long.
It’s only a matter of time before most of the data going in to LLMs came out of one too.
Like I’m not even good at math or anything, much less computer science, but that seems obvious and inevitable to me.
Maybe someone smarter than me can chime in.
It’s kind of sort of already happening. Emdashes are fragments of it, a token saving strategy introduced by already trained on old books in their data sources. Model collapse is inevitable.