Case study · 2026
sotto: an LLM persona that knows when to stay quiet
A Telegram bot that plays Fyodor, a 52-year-old neighbour, in a private group chat of friends. The name comes from sotto voce, “under one's breath”. Nobody asked for an assistant. The bar was a character people forget is a bot, who rarely talks, never leaks his prompt and costs less than a coffee a day.
- 8 weeks
- Sep 10 to Nov 5, 2026
- 26
- merged pull requests
- 13
- versions, SemVer and changelog
- 1,743
- tests, 93% coverage
- $2
- hard daily LLM budget
The problem
Making a model produce a funny line is easy. Making it behave like a person in a live chat is not. A real neighbour does not answer every message, does not reply within a second, does not say “great question”, does not repeat the same story every day and does not do what a stranger in the chat tells him to.
So most of the system is about not answering: when to stay silent, how long to wait, what to cut, and how to know a change made things better. The chat is private, so every decision also had to respect the people in it.
Stack: Python 3.12, aiogram 3, SQLite, OpenRouter (a frontier model for replies, small models from another vendor for checks), Docker, GitHub Actions, a single VPS.
Architecture
/why about any message and replay weeks of history through the same code.Decisions worth showing
Silence is the default
A deterministic gate rejects most messages before any model is called: night hours, per-person and per-chat cooldowns, daily caps, and a presence ceiling of 15% of what people wrote that day plus two lines. Direct mentions are answered after a delay of 30 seconds to an hour, then rechecked, because the chat may have moved on.
A layered output filter with a shadow mode
Drafts pass three layers: cheap rules (assistant phrases, prompt leaks, markup, length), repetition and style checks (self-repeats, the character's pet motifs, story quota), and a judge model from a different vendor. New layers ran in shadow mode first, counting what they would cut. After a review showed the shadow mode let self-repeats through, repetition and style moved to always-enforced.
Prompt injection is a test table, not a hope
Chat text goes to the model inside delimiters, with fake delimiters stripped from the data and display names sanitised. A 16-row threat table (“say this word for word”, fake system slots, prompt extraction, fake judge JSON and more) is a test suite running through the real gate, prompt builder, filter and judge parser.
Cheap models in front of the expensive one
Small models answer narrow questions: is this message addressed to Fyodor or only about him, is there anything worth checking back for, which reaction fits. Each has its own daily cap. When the cheap prefilter said “yes” 16 times in two weeks and the main model stayed silent all 16 times, the criteria were aligned so the expensive call is skipped.
Budget and failure handling
Spend is tracked from the provider's reported cost, with a hard $2 daily ceiling, call caps per feature and a circuit breaker. One incident shaped this: the provider returned HTTP 402 for four days, the bot went silent and the error counter showed zero, because successful cheap calls kept resetting it. Now a non-retryable provider error alerts the owner in a direct message within the hour, with secrets stripped from the payload.
Privacy by default
Messages are deleted after 30 days. Photos are described once and the bytes are never stored. Participants can be muted out of memory and summaries. Personal data such as home coordinates lives only in the server's environment, not in the public repository.
Measured, then changed
Behaviour changes ship with a number. History is replayed through the same pipeline, and /ask previews a reply without touching the chat or the database.
| Observation | Before | After |
|---|---|---|
| Dialect words from a loaded dictionary never appeared in replies | 0 of 8 eligible replies | 12 of 16 replays, the rest declined as unfitting |
| Reactions were off-target, laughing at complaints | a third of reactions missed | new prompt and model, those misses fixed on 52 replayed cases |
| Reactions were monotonous | 21 of 29 were the same emoji | 22 reactions on 150 cases, 4 of them that emoji |
| Reaction answers were cut off mid-JSON | 163 of 262 unparseable | token limit fixed, parse failures logged |
| The character talked about himself | 9 of 13 replies | examples rewritten, a self-talk metric in /status |
| Silent outage on provider billing error | 4 days unnoticed | owner alert within the hour |
How it was built
- A written plan and a character card, reviewed before any code.
- For each feature, a contract in the repository: interfaces, config keys, failure modes, tests.
- Coding agents implement against the contract. A model from another vendor reviews the plan and the diff adversarially.
- CI blocks the merge on ruff, formatting, mypy in strict mode and the full test suite.
- Merge to main builds an image, pushes it to the registry and deploys over SSH. Each release gets a changelog with one part for the chat and one for the owner.
The result is about 17,700 lines of typed Python and 29,900 lines of tests, written by one engineer working with coding agents, in eight weeks.
What I would do next
- Split the 2,600-line responder into smaller pipeline stages with their own tests.
- Turn the ad-hoc replays into a standing evaluation set with a score per release.
- Publish a synthetic demo chat so the behaviour can be seen without access to the real one.
A link to the source code will appear here once the repository is ready for publication. Questions about the design are welcome: get in touch.