How a completion is made: what a 7B model sees when you pause

Autocomplete is the feature people judge a coding assistant on first, and the one where a small local model can hold its own. Not because small models are clever, but because the job is narrow: fill in the middle of a file whose surroundings the model can see. Almost everything that makes twinny’s suggestions good or bad happens before the model runs.

Six steps

1. Trigger. You pause for 300 ms (twinny.debounceWait), or press Alt+\. A request still in flight from the previous pause is cancelled, so the model only ever works on the current cursor position.
2. Prefix and suffix. twinny takes the lines before and after the cursor (twinny.contextLength, 100 lines, capped at 12,000 and 3,000 characters). The suffix is the half that most tools forget: without it the model does not know the function already has a closing brace.
3. Context. Up to 6,000 characters of extra material: your recent edits in any open file, as short diffs; matching symbols and the signature of the call the cursor is inside, from the language server; and, if you switch it on, snippets from related open files.
4. Prompt. The model’s own fill-in-the-middle tokens wrap prefix, suffix and context. Qwen2.5-Coder wants <|fim_prefix|>, CodeLlama wants <PRE>, DeepSeek wants <｜fim▁begin｜>. twinny picks the template from the model name, and you can override it.
5. Stream. The reply streams in and is shown as it arrives. twinny gives up after 30 seconds.
6. Format. Text the suffix already contains is trimmed, so completing inside foo(|) does not produce a second ). Indentation is matched. Stray template tokens are removed.

Every step is logged in the Twinny output channel. At Debug level you can read the exact prompt and the exact reply, which is the fastest way to understand why a suggestion was odd.

Knowing when to stop

Small models do not know when to stop; they keep generating until the token budget runs out. twinny decides for them. While the reply streams, it is parsed with tree-sitter and cut at the end of the enclosing block or statement, so a suggested function ends at its closing brace rather than running into the next one. twinny.maxLines (40) caps a multi-line suggestion. The model family’s stop tokens end the reply. The token budget per request is 512 (twinny.numPredictFim).

This is why a 1.5B or 7B coder model, which would ramble in a chat window, produces tidy completions: it is never asked to know where a function ends.

Recent edits are the best context

Of the context sources, recent edits pull the most weight per character. If you are halfway through renaming userId to accountId across a file, the last six changes as diffs tell the model exactly what pattern to continue. The language server adds the names and parameters that exist, so the model does not invent them. Related files help on large projects but cost latency, which is why they are off by default.

Models with repository-level FIM tokens (Qwen2.5-Coder, StarCoder2, CodeGemma, Granite, CodeGeeX) receive extra files as separate named blocks. Others get them as commented-out blocks above the prefix. Either way the model sees a shape, not a lecture.

What to run

For completion, a base coder model, not an instruct one: instruct models want to chat. Something between 1.5B and 7B is the sweet spot on a laptop GPU; the docs’ supported models page says what works where. Keep twinny.temperature low (0.2 by default) if suggestions wander.

The full reference, including every setting named above, is at Code completion.