Sangama

Sangama (संगम) means confluence: computers coming together to run one AI model.

Sangama is an open-source program, written in Rust, that splits a language model's layers across several computers: Macs, GPUs and ordinary CPUs. A model too large for any one device can then run across several, with encrypted, invitation-only connections between them.

Status: experimental. It runs on an invite-only network of trusted computers. It has run Qwen3.5-397B across 20 GPUs with 16 GB each (results), and is tested in depth with a small model (Qwen2.5-0.5B-Instruct). It is released under the MIT License.

How it works

A language model predicts each next piece of text (a token) by passing numbers through a stack of layers. Sangama gives each computer a contiguous slice of those layers.

  +----------+   tokens    +-------------------+  activations  +-------------------+
  |  Client  | ----------> |     Worker A      | ------------> |     Worker B      |
  |          |             |  layers 0 - 11    |               |  layers 12 - 23   |
  |          |             |  (e.g. a Mac)     |               |  (e.g. a GPU)     |
  +----------+             +-------------------+               +-------------------+
       ^                                                                 |
       |                     next token, then repeat                     |
       +-----------------------------------------------------------------+

Contribute. Each worker loads only the slice of the model it serves, sized to the memory it offers.
Route. A client checks that its workers cover every layer in order, reserves them, and sends the prompt through the chain.
Generate. The last worker picks the next token and sends it back. Each worker keeps its own conversation state, so only a few kilobytes cross the network for each token.

The model's weights stay loaded on the workers; they are not sent over the network for each request.

Architecture

Sangama keeps three questions apart: who may join, how computers reach each other, and where the model runs.

                  +---------------------------------------------+
                  |                   Portal                    |
                  | invitations, signed membership, revocation  |
                  +---------------------------------------------+
                        |                   |                  |
                  signed membership list, checked every few seconds
                        v                   v                  v
                  +----------+        +----------+       +----------+
                  |  Client  | <----> |  Relay   | <---> | Workers  |
                  +----------+        +----------+       +----------+
                        |          (only when routers         ^
                        |           block a direct path)      |
                        +-------------------------------------+
                            direct encrypted connection

Part	What it does
Worker	Contributes memory and compute. Loads its share of the model and runs those layers.
Client	Finds workers, checks they cover the whole model, and runs requests through them.
Portal	Issues single-use invitations, keeps the membership list, and revokes members.
Relay	Passes encrypted traffic between computers when their routers block a direct connection. It never runs model layers and cannot read the traffic.
Discovery	Workers advertise which part of the model they hold, in signed records.

Every computer has its own Ed25519 identity, and all traffic is encrypted end to end (Noise over libp2p). Workers check every model file against a pinned SHA-256 list, so a route cannot silently use a different model.

A limit to know about: a worker can see the data it processes, so today's network is meant for trusted groups. Signatures prove who is a member, not that a computation was done correctly.

Results

Test write-ups, newest first. All results.

30 Sep 2026 — Qwen3.5-397B on 20 GPUs, through a relay
29 Sep 2026 — A small model across CPUs, GPUs and the Internet

Hardware

Hardware	How	Status
Apple Silicon Macs	Metal, using unified memory	Tested
NVIDIA GPUs	CUDA or Vulkan on Linux, one GPU per worker	Tested
Any CPU	macOS and Linux	Tested
AMD, Intel, Qualcomm GPUs	Vulkan or ROCm, through llama.cpp	Built, not yet tested
Windows	CUDA or Vulkan	Not yet tested

Roadmap

Large open models. Qwen3.5-397B runs across 20 computers with 16 GB each (results). Next: make it faster with speculative decoding and nearby relays.
Published weights. Done: the model is published in per-layer slices with checksums, so each worker downloads only the layers it serves.
More hardware. Test runs on AMD, Intel, Qualcomm and Windows, then phones.
Nearby routes. Keep each route within one city or region, since network distance sets the speed.
Two home networks. The next acceptance test: two home routers, a relay, and direct connections through NAT.

Contact

Want to contribute a GPU, join a test network, or help build Sangama? Join the Discord or write to hello@sangama.co.

Source code: github.com/devdil/sangama · Contributor guide

Sangama is open source under the MIT License.