Pooling a teammate's GPU: one gateway, no open ports
How twinny-server lets a developer share their own computer's models with the team, what the sharer consents to, and why a busy GPU queues instead of failing.
Most small teams do not have a GPU server. They have four laptops and one person with a desktop that has a 3090 in it. twinny-server was built for that team as much as for the one with a rack.
The shape of it
The gateway is one process on one machine, in front of whatever backends you give it: a local model server, a private inference endpoint, or teammates’ computers. Every developer’s VS Code talks to the gateway with a personal key. The gateway keeps usage per key, applies policy, and routes each request to a backend.
developer laptops ──HTTPS──▶ twinny-server ──▶ your model servers
▲
└──── teammates' computers (pooled)A developer who wants to contribute their machine flips Share this computer in the extension. Their local model server, the one they already run for themselves, starts serving the team through the gateway. Nothing is exposed on their side: the connection goes out from their machine to the gateway, so there is no port to open and no firewall rule to ask for.
Consent, in one line
Sharing is not silent. Only a named key can share, so the gateway always knows whose machine is serving. Before sharing starts, the sharer sees one line saying what that means: teammates’ requests will run on this computer’s local server through the gateway. Requests are streamed back through the gateway and not kept on the sharer’s machine. Usage records name the machine that served each request, so the admin page shows who is serving what, and an admin can disconnect a machine.
Team policy can say which backends a workspace may use, so a repository that must stay on the office GPU never lands on a laptop at home, even if that laptop is sharing.
A queue, not an error
One GPU serving five people hits a limit quickly. The naive answer is to refuse the sixth request, which shows up in VS Code as a failed completion and a confused developer. The gateway does something else. maxActiveRequests caps how many chat and autocomplete requests run at once; a request that finds every slot taken waits, oldest first, for up to a few seconds, and a freed slot goes straight to the head of the queue. The developer sees a short pause. Embeddings are not counted, because indexing a workspace is hundreds of tiny requests and would starve everyone.
If a request waits too long, or too many are already waiting, it is refused with rate-limited and the log says which. twinny_queued_requests on /metrics shows how often the queue is used. If it is rarely empty, the answer is a bigger cap or another backend, and now you have the numbers to justify the hardware.
Where to start
On the machine with the models: npx twinny-server quickstart. It finds the model server, writes a config, prints an admin key once and starts serving. Free for five developers, for good; more seats are $6 a month each. The teams overview has the rest, and the live demo is a real admin page with real models behind it.