For the longest time, I’ve been trying to figure out a way to “survive” in this new AI age without having to fork over a ton of money just to keep up. I’ve tried using local models via Ollama, and while they definitely work to a degree, they’re (unsurprisingly) not as good as the big model providers.
The local models tend to
- Forget what they’re doing
- Struggle to break larger tasks into smaller ones
- Lose focus easily
- Have weaker coding performance
- Drift over longer sessions
So to improve the reliability of fully local, smaller models (and to keep all my data local and in my own network), I created Loki.
It’s a local-first, batteries-included command line tool and runtime for building and running LLM workflows locally. It’s model agnostic and supports things like
- Agents and agent delegation
- Roles/personas
- MCP Servers
- RAG
- Custom tools
- Macros
- Workflow Scripting
A lot of the features it supports are specifically designed to compensate for weaknesses in smaller local models. For example:
- Auto continuation to keep pushing models to completion instead of stopping halfway through problems
- Parallel agent delegation so tasks can be split into smaller, focused scopes
- Workflow-based execution (“If this, do that”) for building more reliable and repeatable automations
It also supports the major cloud providers if you want them (which definitely helped while testing 😄), but my long-term goal is simple:
Get as close as possible to Claude Code-style reliability using fully local models.
I’m always open to feedback, questions, or ideas.
Very cool idea! Going with the Coyote theme maybe name it Wile E?
Certified Genius: Have Brain, Will Travel