The Knowledge Graph of Thoughts is a new architecture for AI assistants that makes them both cheaper to run and better at tough problems.

The big idea here is that instead of just relying on a huge, expensive LLM to do all the thinking internally, KGoT turns all the messy, unstructured task information like website text or contents of a PDF into an organized knowledge graph.

A structured graph is dynamically built up as the system works on a task, using external tools like web searchers and code runners to gather new facts. Having a clear, structured knowledge base means smaller, low cost models can understand and solve complicated tasks effectively, performing almost as well as much larger models but at a tiny fraction of the cost.

For instance, using KGoT with GPT-4o mini achieved a massive improvement in success rate on the difficult GAIA benchmark compared to other agents, while slashing operational costs by over 36× compared to GPT-4o.

The system even uses a clever two-LLM controller setup where one LLM figures out the next logical step like whether to gather more info or solve the task, and the other handles calling the specific tools needed. Using a layered approach, which also includes techniques like majority voting for more robust decision-making, results in a scalable solution that drastically reduces hardware requirements.