I’m referring to nondeterminism for the same prompt, since unless you start a session from scratch, it’s unlikely you’ll have the same history. If you give it a prompt, then depending on what you’ve told it previously, it may blow up in your face.
Determinism for the same prompt means you can’t give it context through a conversation, which vastly shrinks its utility.
That said, even that form of determinism can be unreliable: the example of arithmetic still works; you could have it completely deterministic, but if it only performs correctly on 80% of arithmetic problems, it’s still unreliable.
Not really… Determinism would only help if you could copy someone else’s prompt and history 100%, which you generally would not be able to.
Because maybe it always gets 1+1 correct, but fails 1+2.
I’m referring to nondeterminism for the same prompt, since unless you start a session from scratch, it’s unlikely you’ll have the same history. If you give it a prompt, then depending on what you’ve told it previously, it may blow up in your face.
Determinism for the same prompt means you can’t give it context through a conversation, which vastly shrinks its utility.
That said, even that form of determinism can be unreliable: the example of arithmetic still works; you could have it completely deterministic, but if it only performs correctly on 80% of arithmetic problems, it’s still unreliable.