feat(voice): proximity voice over the existing wire protocol (sim side) #42

Merged

jeroen merged 3 commits from feat/32-proximity-voice-sim into main

2026-08-10 20:52:22 +00:00

jeroen commented

2026-08-10 20:39:40 +00:00

Owner

Closes #32.

Sim-side voice: VoiceFrame/VoiceData at kinds 20/21, wire revision 13, reusing ChatHub's fan-out and ChatScope unchanged. No SFU, no str0m — an SFU would reimplement ChatHub's job while knowing nothing about avatar positions.

Verified: 20 tests in viberfox_core, 24 in viberfox_simulator (including 6 new voice integration tests), cargo check -p viberfox clean.

Closes #32. Sim-side voice: `VoiceFrame`/`VoiceData` at kinds 20/21, wire revision 13, reusing `ChatHub`'s fan-out and `ChatScope` unchanged. No SFU, no str0m — an SFU would reimplement ChatHub's job while knowing nothing about avatar positions. Verified: 20 tests in `viberfox_core`, 24 in `viberfox_simulator` (including 6 new voice integration tests), `cargo check -p viberfox` clean.

jeroen added 3 commits

2026-08-10 20:39:40 +00:00

feat(voice): proximity voice over the existing wire protocol b60b9e0b34

Voice rides the chat fabric rather than a WebRTC stack. An SFU would
reimplement what `ChatHub` already does while knowing nothing about avatar
positions — which would then have to be fed to it to get the spatial
attenuation the sim can already compute. ADR-028's str0m note is superseded
by that reasoning; this is the sim half of rolling it ourselves.

`ChatHub` needed no generalising: it deals in already-encoded frames, so its
payload was always opaque. Proximity voice is the text path with an audio
frame in place of the string, including the per-recipient encode that
distance attenuation needs for the same reason `distance_m` already needed
it. `within_radius` extracts the selection both paths were doing inline, so
the radius is now covered by a unit test — `SimHandle` exposes no way to move
avatars apart, so it could not be reached through the socket.

Four things voice must do differently from text, each with a test:

- It never loops back to the speaker. Seeing your own message is
  confirmation; hearing your own voice is an echo.
- It is never stored. Text keeps scrollback per channel; recorded audio in a
  database is a liability, and it is ephemeral in every scope, not just Local.
- It has no global tier. Unbounded fan-out is the loudest abuse surface chat
  has, and ADR-028 already suggested launching without the widest text tiers.
- It carries `seq`. The intended transport is unreliable, unordered
  datagrams, where arrival order is not send order, so ordering has to live in
  the message rather than be inferred from the stream.

Limits are sized for media, not typing: the text intervals are seconds apart
where speech is a 20 ms frame cadence.

Out of scope, and the reason this is not yet audible: `platform::audio`,
Opus, acoustic echo cancellation, and the datagram browser transport. None of
them are verifiable in the agent container, which has no audio device.

Refs #32

docs: point the eyeball hatches at a command that runs them bfd4ff44cf

Five doc comments still named `cargo test -p viberfox --bin viberfox`. Since
the lib+shim split that selects a nine-line binary with no tests in it, so each
of these `--ignored` hatches silently matched nothing and exited 0 — the
failure mode `docs/notes/agent-verification.md` was written about.

Four of them are in `crates/geo` and named `-p viberfox`, which was already
the wrong package before the split; they take `-p viberfox_geo`.

docs(notes): measure what audio the agent container can and cannot do

CI / cargo check (push) Failing after 1m21s

Details

CI / build & test viberfox (push) Successful in 5m0s

Details

CI / cargo check (pull_request) Failing after 1m43s

Details

CI / build & test viberfox (pull_request) Successful in 5m15s

Details

10df523eec

The voice work keeps raising "can we fake an audio device in here?", and the
answer is non-obvious in both directions: the container already has libasound,
its headers and pkg-config, so cpal compiles with no install — while having no
card, no /dev/snd and no CAP_SYS_MODULE, so snd-aloop cannot be loaded from
inside it at all.

The measurement worth keeping is that `pcm.null` opens both directions at
48 kHz but does not clock: 1.000 s of audio went through in 0.000 s of wall
time. It therefore answers "does the stream open" and nothing about timing —
a jitter-buffer or underrun test written against it reports green while
exercising nothing, which is worse than no test.

viberfox-agent force-pushed feat/32-proximity-voice-sim from 10df523eec

CI / cargo check (push) Failing after 1m21s

Details

CI / build & test viberfox (push) Successful in 5m0s

Details

CI / cargo check (pull_request) Failing after 1m43s

Details

CI / build & test viberfox (pull_request) Successful in 5m15s

Details

to 8eac6330a2

CI / cargo check (push) Failing after 1m37s

Details