Hell, both Gemini and Claude have done well for me in throwing some code together for simple apps, but I still looked it over before running it. I try to do that for even human-made open source code, although at least in principle, if it’s been up a while, others have at least tried it and given feedback.
I’m sure this is just a joke or a proof of the issue; you’d have to look at the PDF or whatever it gives and see that something is way off.
And for the record, the best code that I’ve gotten from an LLM has been the first few runs of an idea, one that was thoroughly explained in the prompt. If you put together a vague prompt and continue to add to it, it will get worse quickly, with the LLM even changing parts that were perfectly fine. Maybe turning the temperature down, if possible, will help with that randomness, but it’s always better to keep sessions short and precise.
Hell, both Gemini and Claude have done well for me in throwing some code together for simple apps, but I still looked it over before running it. I try to do that for even human-made open source code, although at least in principle, if it’s been up a while, others have at least tried it and given feedback.
I’m sure this is just a joke or a proof of the issue; you’d have to look at the PDF or whatever it gives and see that something is way off.
And for the record, the best code that I’ve gotten from an LLM has been the first few runs of an idea, one that was thoroughly explained in the prompt. If you put together a vague prompt and continue to add to it, it will get worse quickly, with the LLM even changing parts that were perfectly fine. Maybe turning the temperature down, if possible, will help with that randomness, but it’s always better to keep sessions short and precise.