feat(audio): microphone seam and Settings ▸ Audio (mic test + sound test) #43

Merged

jeroen merged 3 commits from feat/audio-settings into main

2026-08-10 21:06:20 +00:00

jeroen commented

2026-08-10 20:39:50 +00:00

Owner

platform::audio adds the capture half behind the ADR-030 seam — cpal on desktop, getUserMedia + WebAudio in the browser — plus a Settings panel with an Audio tab carrying a sound test and a live mic meter.

The web arm was compiled for the first time in this branch (the dev image gained the wasm32 target), which found four defects including a leaked capture graph that would have left the browser's recording indicator lit after Stop.

Verified: 300 tests pass natively both with and without the audio feature; both wasm configurations type-check.

Not verified: no microphone has ever been opened — this container has no audio device — and a cargo check neither links nor runs. Note also that the shipped web build is --no-default-features, so it currently contains no audio at all; browser voice needs that feature enabled for wasm as a separate decision. See docs/notes/checking-the-web-build.md and docs/notes/container-audio.md.

`platform::audio` adds the capture half behind the ADR-030 seam — cpal on desktop, `getUserMedia` + WebAudio in the browser — plus a Settings panel with an Audio tab carrying a sound test and a live mic meter. The web arm was compiled for the first time in this branch (the dev image gained the wasm32 target), which found four defects including a leaked capture graph that would have left the browser's recording indicator lit after Stop. Verified: 300 tests pass natively both with and without the `audio` feature; both wasm configurations type-check. **Not verified:** no microphone has ever been opened — this container has no audio device — and a `cargo check` neither links nor runs. Note also that the shipped web build is `--no-default-features`, so it currently contains no audio at all; browser voice needs that feature enabled for wasm as a separate decision. See `docs/notes/checking-the-web-build.md` and `docs/notes/container-audio.md`.

jeroen added 2 commits

2026-08-10 20:39:50 +00:00

feat(audio): a microphone seam, and Settings ▸ Audio to test it with 76210ef787

Voice needs capture, and nothing in the client had any. `bevy_audio` wraps
rodio, which is output-only, so `platform::audio` adds the input half behind
the ADR-030 seam: cpal on the desktop, `getUserMedia` + WebAudio on the web,
one `AudioInput` API above both.

The settings panel is where you find out whether any of it works. Two tests,
because they fail independently and for different reasons: a sound test that
plays a chime through the *same* `bevy_audio` path the soundscape uses (so it
proves the application's own output path, not some other one), and a mic test
with a live meter and a peak hold.

Four things are deliberate.

The desktop arm owns a thread. `cpal::Stream` is not `Send` on every backend
and a Bevy `Resource` must be `Send + Sync`, so the stream lives on a thread
that parks for its lifetime and the ECS holds atomics and one small mutex.

The meter is dB, not linear. Ordinary speech is around -20 dBFS, which is 10%
of a linear bar — the meter would look broken until you shouted. It is also
smoothed asymmetrically: fast attack so a clap registers, slow release so it
can be read.

The backlog drops the *oldest* audio. Stale speech is worthless, and an
unbounded queue turns a slow consumer into growing latency and then an OOM.

The microphone is released when the test stops or the panel closes. Holding it
open for the process lifetime is how an app earns a permanent recording
indicator in the OS status bar.

cpal rides the existing `audio` feature rather than being unconditional: it
links the same ALSA, and a headless build that opted out of `bevy_audio` must
not get an alsa-sys dependency by the back door. The DSP stays outside the
feature gate and stays tested there — 300 tests pass in both configurations.

Not verified: nothing here has opened a real device. This container has no
audio hardware (docs/notes/container-audio.md), and the **web arm has never
been compiled** — the dev image has no wasm32 target, and `rustup target add`
fails on a broken cache dir there. Treat the browser path as written, not
working, until someone runs it.

fix(audio): compile the web capture arm, and four defects that found

CI / cargo check (push) Failing after 1m31s

Details

CI / build & test viberfox (push) Failing after 2m1s

Details

CI / cargo check (pull_request) Failing after 51s

Details

CI / build & test viberfox (pull_request) Failing after 2m7s

Details

fbbecdce9c

The wasm32 target is installed in the dev image now, so `platform::audio`'s web
arm has been through a compiler for the first time. It did not survive contact.

- Ten `web-sys` features were missing and `js-sys` was never named as a
  dependency, so that arm could not have built at all.
- `attach` leaked the WebAudio graph with `mem::forget`. The module documents
  that the microphone is released when the test stops; the code left it open,
  which in a browser means the recording indicator stays lit after Stop. The
  graph now lives in a thread-local (it is `!Send` and arrives asynchronously,
  so it can be neither a field nor part of the shared state) with a `Drop` that
  disconnects the nodes, stops every `MediaStreamTrack` and closes the context.
- `MediaStreamTrack` was not among the enabled features either — the very call
  that clears that indicator.
- `downmix` is dead on wasm, where WebAudio hands over a single channel. Not a
  web bug: a native helper that only became visibly unused once the crate was
  built for a second target.

Also swapped the deprecated web-sys builder setters for the `set_*` forms.

Two things the exercise established that are worth more than the fixes.

A bare `cargo check --target wasm32-unknown-unknown` cannot check this crate:
the default `solo` feature drags in libsqlite3-sys, which fails on a missing
`stdio.h`, and `wasm_thread` needs nightly. The real recipe is four flags, each
of whose errors names something other than its cause, so it is written down in
docs/notes/checking-the-web-build.md rather than left to be rediscovered.

And the shipped web build has no audio in it: `index.html` builds
`--no-default-features` while `audio` is a default feature, so the browser
client contains neither `bevy_audio` nor this web arm today. Browser voice needs
that turned on for wasm first. The module doc says so now instead of implying
the web path is live.

Verified: 300 tests pass natively both with and without `audio`; both wasm
configurations type-check. Still unverified, and unverifiable here — neither arm
has opened a real microphone, and a `cargo check` neither links nor runs.

viberfox-agent force-pushed feat/audio-settings from fbbecdce9c

CI / cargo check (push) Failing after 1m31s

Details

CI / build & test viberfox (push) Failing after 2m1s

Details

CI / cargo check (pull_request) Failing after 51s

Details

CI / build & test viberfox (pull_request) Failing after 2m7s

Details

to f32f07c4b1

CI / cargo check (pull_request) Failing after 2m1s

Details