Ello Ello.
Can you recommend a program with which I can generate commit messages locally? I DO NOT want an LLM local or cloud API. What I want is a tiny or small language model. If I could easily swap or customize the model for each project, it would be better, too. So I could have smaller models for specific projects. I can do the training meself.
Thanks in advance!
What is your small lm threshold?
There are Gemma 4 models that run on phones, is that what you’re looking for?
To do this well you will need tool calls because just reading the git diff won’t be enough context to do a good job.
This is probably something you can fine tune, it is getting exponentially cheaper to do this, but without knowing why you don’t want to self host an existing LLM to solve this problem it’s hard to give good advice.
You can look at smollm3 or nanogpt and nanollama as (dated) examples of how to build such a small model yourself, but they’re still LLMs that run on LLM tools. Smollm has guides to fine tune for your task.
You could run them with PyTorch directly or something too, if you’re opposed to the full llama.cpp (vllm is even heavier, with very long startup times).