Yuvraj Singh

Yuvraj Singh

LLM · RL · Distributed Systems

SmolTorrent

GitHub

Distributed ML Checkpoint Sharding Across a Raspberry Pi Cluster

SmolTorrent

Distributed ML checkpoint sharding across a Raspberry Pi cluster, coordinated from a macOS master. Shards .safetensors checkpoints across workers over TCP with SHA-256 verification, replication factor 2, automatic watcher sync, and zero-config device discovery over mDNS and AirDrop.

This is an educational project built to learn distributed systems concepts hands-on.

→ Full documentation & setup guide · → How it was built

Master (Mac mini / Apple Silicon)
  ├── FastAPI server   backend/api.py          ← /store-shard, /gather-shards, /discover
  ├── Watcher daemon   watcher/watch.py         ← auto-syncs new checkpoints
  ├── Discovery        discovery/               ← mDNS + AirDrop device discovery
  ├── SSH manager      networking/ssh_manager.py← writes ~/.ssh/config managed block
  └── Workers × N      algorithms/SyncPS/worker.py  ← TCP listener + mDNS advertiser on each Pi

Features

  • Sharded Checkpoint Storage: Splits .safetensors checkpoints across Raspberry Pi workers over TCP
  • SHA-256 Verification: Ensures data integrity on every shard transfer
  • Replication Factor 2: Redundant storage so no single Pi failure loses data
  • Automatic Watcher Sync: Daemon watches for new checkpoints and syncs them automatically
  • Zero-Config Discovery: Devices find each other over mDNS and AirDrop with no manual IP configuration
  • Monitoring: Prometheus-based monitoring generated from cluster config