The AI which cures cancers is architecturally pretty similar to LLMs in that it’s based on transformers and GPT in particular is also based on transformers (that’s the T in GPT). It is also generative AI, you give it tokens that describe proteins and it generates folding structures. It’s like an LLM except the tokens aren’t words or syllables like in LLMs.
It’s used very differently and the energy cost is a few orders of magnitude less (you can run it on a single H200 with 141 GB VRAM for ~9000 tokens or an A100 or H100 for ~5000 according to this university and it even has a slow CPU/IO heavy first stage preparing everything so they can minimize the GPU time in stage 2).
There is yet hope that such AI applications will eventually benefit from all the research that’s been put into LLMs. Obviously not everything will carry over, but there may be architectural improvements to be made in research models too.
The AI which cures cancers is architecturally pretty similar to LLMs in that it’s based on transformers and GPT in particular is also based on transformers (that’s the T in GPT). It is also generative AI, you give it tokens that describe proteins and it generates folding structures. It’s like an LLM except the tokens aren’t words or syllables like in LLMs.
It’s used very differently and the energy cost is a few orders of magnitude less (you can run it on a single H200 with 141 GB VRAM for ~9000 tokens or an A100 or H100 for ~5000 according to this university and it even has a slow CPU/IO heavy first stage preparing everything so they can minimize the GPU time in stage 2).
There is yet hope that such AI applications will eventually benefit from all the research that’s been put into LLMs. Obviously not everything will carry over, but there may be architectural improvements to be made in research models too.