MeatPilot@sh.itjust.works to Lemmy Shitpost@lemmy.world · edit-211 hours agoA Space Odyssey 2026sh.itjust.worksvideomessage-square8fedilinkarrow-up1147file-text
arrow-up1147videoA Space Odyssey 2026sh.itjust.worksMeatPilot@sh.itjust.works to Lemmy Shitpost@lemmy.world · edit-211 hours agomessage-square8fedilinkfile-text
minus-squareyeehaw@lemmy.calinkfedilinkarrow-up3·5 hours agoMan I feel stupid. I don’t get the purpose of tokens. It’s to track your AI usage to make sure you’re using it? Or it’s to somehow limit your queries fo ensure a team uses it equally? Or…?
minus-squarewraekscadu@vargar.orglinkfedilinkarrow-up5·4 hours agoLLMs predict the next best “token”. So what’s a token? It can be a word, a simple punctuation mark, or sometimes even a phrase. So basically, here’s what an LLM works like: An An apple An apple is An apple is red Basically, it feeds in tokens to itself to predict the next best token. Then, that entire chunk of tokens is fed back in to predict the next token. As you can see, tokens kinda correlate well with energy consumption, hardware wear and tear and so on. So, a low token request will consume less resources than a high token request. Hence, it makes sense to charge per token.
Man I feel stupid. I don’t get the purpose of tokens. It’s to track your AI usage to make sure you’re using it? Or it’s to somehow limit your queries fo ensure a team uses it equally? Or…?
LLMs predict the next best “token”. So what’s a token? It can be a word, a simple punctuation mark, or sometimes even a phrase.
So basically, here’s what an LLM works like:
Basically, it feeds in tokens to itself to predict the next best token. Then, that entire chunk of tokens is fed back in to predict the next token.
As you can see, tokens kinda correlate well with energy consumption, hardware wear and tear and so on.
So, a low token request will consume less resources than a high token request. Hence, it makes sense to charge per token.