OCS Intelligence LLM β Phase 3: Mac mini architecture
Recent architecture work on Mac mini provisioning, stable networking, and a shared package cache, with verified results and remaining rollout work.
Phase 3 β Provision the Mac mini pool
Recent architecture work: repeatable Mac mini provisioning, stable node identities, and a shared package cache, with measured progress and the next steps toward a distributed orchestrator.
Preparing the Mac mini pool
Our November 12β17 work makes the future orchestrator machines repeatable to set up: one Ansible baseline, stable node identities, and a shared download cache. The inventory now counts 33 Mac minis. Silver, node 25, is the first fully configured and checked machine; the remaining fleet still needs the setup rollout. The Minis are not yet in the student request path.
flowchart TD A[FIX stick and serial-to-number table] --> B[Local bootstrap service] B --> C[Shared Ansible baseline] D[Silver: wired download cache] -->|apt packages and locked Python wheels| C C --> E[Named Ubuntu node with NetBird and pinned Python and Ray] E --> F[Local Ray smoke test] F -. next .-> G[Multi-node Ray and Mini orchestrator]
1. Set up each Mini from the same playbook
The FIX stick restores the wired network, copies the setup kit onto the Mini, and starts a local service. That service runs the same Ansible playbook we can also run over SSH. Setup can continue after the stick is removed, without depending on an operator laptop staying connected.
- Install the common baseline: fan control, power-on after power loss, zram, Docker, NetBird, and a uv-managed Python 3.12 environment with Ray 2.60.0 and locked dependencies.
- Download Python packages and install NetBird while the first-run wired connection is still active; configure the Wi-Fi driver afterwards.
- Resume after a kernel reboot, bound retries and reboot attempts, and remove the copied setup secrets when setup succeeds.
Evidence and next check: The Ansible baseline completed on silver on November 13; a rerun changed nothing, and reboot and power-cut recovery worked over Wi-Fi alone. The current unattended stick workflow has offline checks, but its first complete run on a fresh Mini is still pending.
2. Give each node a stable identity and network path
Each Mini is named from its firmware serial and the team's numbered inventory. Discovery finds our machines again when DHCP addresses move, and router reservations give wired and Wi-Fi interfaces predictable addresses. Silver is node 25; the mastermac is a separate machine, node 0.
- Discover only machines with our SSH key, expected Mac mini model, and OCS names. Remember SSH host keys by machine name so an address change does not erase the machine's identity.
- Generate router reservations with clash checks, backup, and verification. The first reservations were applied to silver; the rest follow as machines come online.
- Prefer Ethernet on silver so cache replies leave over the cable. Other nodes currently prefer Wi-Fi after setup; wired-versus-Wi-Fi performance for Ray still needs measurement.
Evidence and next check: Silver's reserved addresses and recovery were checked on November 13. Its wired-first route was verified again on November 17. These are checks on one node, not proof that the full fleet is configured.
3. Fetch setup packages from one local cache
Every Mini downloading the same packages would compete for the lab's shared internet link. Silver now runs apt-cacher-ng and serves the pinned uv binary and all 89 locked Python wheels locally. This cache accelerates provisioning; it does not store student conversations or model context.
- Keep silver wired-first and fetch repeated packages over the LAN. HTTP apt repositories use the proxy; HTTPS repositories such as NetBird's still go directly to the internet.
- Publish the wheelhouse READY marker only after the download completes. A client uses that wheelhouse only when the marker matches the SHA-256 of its own requirements lock; otherwise it uses PyPI.
- Resume interrupted cache population by checking for missing wheels, rather than assuming a copied requirements file means the cache is finished.
Evidence and next check: On November 17, silver held all 89 wheels, its READY marker matched the lock, and both cache services were active. A 79 MB Ray wheel reached the operator's Wi-Fi laptop at 48 MB/s from silver versus 9.8 MB/s from PyPI, about five times faster. A second wired Mini and concurrent fleet setup have not yet been measured.
4. Connect the pool only after the baseline works
Ray currently passes a local smoke test on silver. The next architecture step is a head node on the mastermac and worker services on the other Minis, followed by the Mini orchestrator. Installing Ray on one machine does not establish a distributed cluster or move API traffic onto it.
- Run the current FIX stick on one fresh Mini with a monitor through reboot to completion, then expand in batches and verify naming, networking, and package consistency.
- Measure wired and Wi-Fi paths between nodes before choosing how distributed compute traffic should travel.
- Add Ray head and worker services, then demonstrate work crossing machines and recovery when a worker disappears.
- Design and test the Mini router before inserting it into the API path. The larger Redis/RDS session broker remains a future proposal; the deployed gateway still holds its wait lines in process memory.
Evidence and next check: The next milestone is two independently provisioned Minis completing a distributed task, followed by a measured routing and recovery demonstration. Fleet capacity, session recovery, and classroom reliability remain to be proven.
Serving context for this phase
The API uses one public base URL, https://ai.opencodingsociety.com/v1, and a shared student key. The EC2 gateway checks that key, manages a bounded wait line for each model, and routes requests over NetBird to the appropriate rig. Tokens stream back along the same path. Per-student keys remain planned.
One API, two rigs
flowchart TD
A[OpenAI-compatible client] -->|HTTPS| B[EC2 Nginx]
B --> C[Admission gateway: key check and per-model wait line]
C -->|NetBird| D[Rig 1 orchestrator]
C -->|NetBird and stored Rig 2 key| E[Rig 2 orchestrator]
D --> F[llama.cpp: two workers on separate 4-GPU groups]
E --> G[llama.cpp: MoE worker across 9 GPUs]
The October 29 gateway change brought Rig 2 into this API path, using a server-held Rig 2 key. The Mac mini work in this phase prepares a future routing layer; it has not replaced either rig's orchestrator.
On November 3, both Rig 1 workers received a 4 GB prompt-cache cap after repeated host-memory failures. Worker A also gained speculative decoding: copying text reached 37.2 tokens/s, while varied free generation stayed near 8.8 tokens/s. Classroom stability under the new cache cap still needs a load test.
Recorded evidence: gateway routing and Rig 1 tuning.