recently I gave AI a task to configure a software and the software didn’t have any docs of how it works only the source code. the agent went through all the files and generated a summary of how it works and why it’s not working for my particular scenario and suggested edits to my docker-compose.yml file. I couldn’t believe it since the bug was very hard to find and it used headless firefox to find why it wasn’t working.
made me realize do we still need documentation of how a software work when a AI can easily explain it?
I’ll respond for the sake of argument, but to put it bluntly: there are no stupid questions, just questions phrased stupidly.
Documentation comes in many formats. There’s prose documentation split into articles and chapters, there’s video documentation, inline documentation for code, tutorials, training, and so on. Most things are documented several ways. This is because people learn things differently from each other.
To remove documentation entirely and rely on only LLMs to document removes all of the ways that people might learn and narrows it down to only one form of documentation. Even if we ignore how fallible LLMs can be, having documentation in only a single format limits learning only to those who learn effectively through that format.
In other words, relying only on LLMs for documentation discards all of the millenia of teaching strategies learned throughout the course of human history and puts it in the hands of a stochastic magic box that is incapable of recreating those strategies.
That makes sense! Different people learn in different way so we need different people’s words on the same topic to have diverse view on subject.
Good point