“Deterministic” compilers and reproducible builds are actually fairly difficult problems. (Yes I know technically the compiler is deterministic, but AI is also technically deterministic since you can set the seed).
It’s not just about being deterministic. A compiler crucially follows clearly defined formal rules on what is a correct output given some input, which the programmer can fully rely on.
Most of the time, today. Mid 90s I was constantly checking compiler output for bugs, and finding them. Early 90s the compiler bugs were finding me, particularly in Borland Turbo C++. I turned off the linker optimizer in 1992 and couldn’t turn it back on until 2008 without getting difficult to identify bugs.
There is a clear line between deterministic code and what comes out of an LLM. One of the bigger problems there is that LLMs have a rather large “context window” they operate from, and human operators don’t readily “grok” how the content of that context window enhances and degrades the desirability of the outputs you get.
I almost exclusively use LLMs to generate, review and test deterministic code, then use that code for “the real tasks” with zero LLM involvement. The output of LLMs is still unpredictable, but once they’ve “said” something valuable, recognizing and using that value is a deterministic activity.
“Deterministic” compilers and reproducible builds are actually fairly difficult problems. (Yes I know technically the compiler is deterministic, but AI is also technically deterministic since you can set the seed).
It’s not just about being deterministic. A compiler crucially follows clearly defined formal rules on what is a correct output given some input, which the programmer can fully rely on.
Most of the time, today. Mid 90s I was constantly checking compiler output for bugs, and finding them. Early 90s the compiler bugs were finding me, particularly in Borland Turbo C++. I turned off the linker optimizer in 1992 and couldn’t turn it back on until 2008 without getting difficult to identify bugs.
There is a clear line between deterministic code and what comes out of an LLM. One of the bigger problems there is that LLMs have a rather large “context window” they operate from, and human operators don’t readily “grok” how the content of that context window enhances and degrades the desirability of the outputs you get.
I almost exclusively use LLMs to generate, review and test deterministic code, then use that code for “the real tasks” with zero LLM involvement. The output of LLMs is still unpredictable, but once they’ve “said” something valuable, recognizing and using that value is a deterministic activity.