For the longest time, I’ve been trying to figure out a way to “survive” in this new AI age without having to fork over a ton of money just to keep up. I’ve tried using local models via Ollama, and while they definitely work to a degree, they’re (unsurprisingly) not as good as the big model providers.

The local models tend to

  • Forget what they’re doing
  • Struggle to break larger tasks into smaller ones
  • Lose focus easily
  • Have weaker coding performance
  • Drift over longer sessions

So to improve the reliability of fully local, smaller models (and to keep all my data local and in my own network), I created Loki.

It’s a local-first, batteries-included command line tool and runtime for building and running LLM workflows locally. It’s model agnostic and supports things like

  • Agents and agent delegation
  • Roles/personas
  • MCP Servers
  • RAG
  • Custom tools
  • Macros
  • Workflow Scripting

A lot of the features it supports are specifically designed to compensate for weaknesses in smaller local models. For example:

  • Auto continuation to keep pushing models to completion instead of stopping halfway through problems
  • Parallel agent delegation so tasks can be split into smaller, focused scopes
  • Workflow-based execution (“If this, do that”) for building more reliable and repeatable automations

It also supports the major cloud providers if you want them (which definitely helped while testing 😄), but my long-term goal is simple:

Get as close as possible to Claude Code-style reliability using fully local models.

I’m always open to feedback, questions, or ideas.

Repo: https://github.com/Dark-Alex-17/loki

    • Dark-Alex-17@lemmy.worldOPEnglish
      8·
      3 hours ago

      So actually, this was the original purpose of it. But all the help I tried to get on it didn’t really have much interest in doing anything outside of the usual big model providers, so I tried advertising a more general use case to attract more input. I can’t deny that agnostic support for even the big providers is helpful when you’re trying to stay current with the rapid advances in LLMs.

      After that, I kind of gave up on getting feedback on local-first models. So, instead, I just dove in head-first the way I wanted;Trying new things, building new agents to try and rival Claude Code, adding features as I found them useful and necessary to improve that reliability, etc., and iterating. Then, with the most recent release on Friday, I had done so many changes and improvements specifically for local models that I thought I finally had a strong enough tool to maybe pique enough people’s interest to get some feedback and input. 🙂

      Oh, and the config example shows how to add Ollama models here