Sangama → Results
Qwen3.5-397B across three US states
30 September 2026, round 13
Our earlier three-stage tests kept every machine in one datacenter. This one puts real distance between them, to measure two things we had built but not tested where they matter: connecting machines directly instead of through a relay, and QUIC as the transport instead of TCP.
Result:
- Direct connections: one request ran 17% faster, 8.9 to 10.4 tokens per second.
- QUIC: the same speed as TCP on these links, but the first token arrived in about 0.3 seconds instead of 0.55.
Setup
| Model | Qwen3.5-397B-A17B, 4-bit, 3 stages of 20 layers |
| Stages | 96 GB GPUs in Maryland, Utah and Texas |
| Relay and client | One machine in Colorado |
| Settings | 48 sessions per machine, shared steps, no guessing |
| Cost | About $4.80 an hour |
Round trips between the machines: relay to Maryland 43 ms, relay to Utah and to Texas 26 ms each, Maryland to Utah 60 ms, Utah to Texas 34 ms.
Numbers
| Mode | One request | Network per trip | First token | 16 requests | 48 requests |
| TCP through the relay | 8.8-8.9 | 98.6 ms | 0.40-0.45 s | 113 | 225 |
| QUIC through the relay | 8.7-8.8 | 96-97 ms | 0.33-0.46 s | 110 | 203 |
| TCP direct | 10.3-10.4 | 82.6 ms | 0.55 s | 127 | 247 |
| QUIC direct | 10.4-10.5 | 81.4 ms | 0.28-0.31 s | 128-129 | 243-245 |
Tokens per second. The 16 and 48 request columns count tokens over the whole run. GPU computing was 14 ms per trip in every mode. Single runs, except the two direct modes under load, which were run twice.
- Direct connections save what the map says they should. From the round trips above, a trip through the relay should spend about 95 ms on the network and a direct one about 82 ms. We measured 98.6 and 82.6.
- QUIC does not speed up a healthy link. These are datacenter links that rarely lose packets. QUIC is meant to help where packets are lost, since over TCP one lost packet holds up every request on that connection, and where home routers need hole punching to connect at all. Neither applies here.
- QUIC does start faster. The first token came in about 0.3 seconds over direct QUIC against 0.55 over direct TCP.
- Distance is the limit again. The same three stages in one datacenter ran one request at 48 to 50 tokens per second. Spread over three states they run at 9 to 10, and the network is 85% of each trip.
What the testing caught
- A slow run on direct TCP. The first run under load gave 78 and 184 where the repeat gave 127 and 247. Some machines still had a relayed connection open next to the direct one, and requests alternate between the two. We only close the relayed one when nothing is in flight, which under steady load may be never. That needs fixing.
- A machine that could not call its own public address. The client runs on the relay machine and now reaches the relay locally.
- A rented machine that never started, replaced before the test.
Next
- Close the relayed connection as soon as the direct one works, even under load.
- Test where QUIC should matter: links with packet loss, and two real home networks.
Full report and raw data: docs/test-results.
Sangama is open source under the MIT License. Contact: diljit@diljitpr.net · Discord