jeroen/cartopolis

Watch

Fork 0

Restructure the client into layered crates: state layer, Bevy-free geo crate, and crate renames #16

New issue

Open

opened 2026-08-06 11:41:28 +00:00 by jeroen · 1 comment

jeroen commented

2026-08-06 11:41:28 +00:00

Owner

Problem

Two problems with one root cause.

Builds are slow where the incremental cache does not reach. Measured
2026-08-06 (docs/notes/client-build-cost.md): a leaf edit rebuilds in 3.8 s,
but compiling viberfox with a cold incremental cache takes 52.4 s, of which
only 7.1 s is the frontend — ~45 s is codegen and linking. Everything that
discards the cache pays that in full: CI (CARGO_INCREMENTAL=0), profile
switches, flipping solo/fast-dev/scalar-fallback, and every cargo test.
Two structural taxes make it worse: crates/viberfox/Cargo.toml declares no
[lib], so cargo test -p viberfox --bin viberfox compiles all ~52k lines a
second time under cfg(test); and dev and dev-bevy are both maintained, so
target/ measured 170 GB and alternating between the two commands in
CLAUDE.md rebuilds the other set.

systems/ has become a flat bin of 50 files holding both the logic and the
state that logic coordinates on. Measured over the module graph
(docs/notes/module-graph.md): 26 mutual dependency cycles, and the
high-in-degree modules are imported for a single state type each, not for their
behaviour — free_camera (in-degree 25) for FreeCamera
(crates/viberfox/src/systems/free_camera.rs:16) and WORLD_CELL_EDGE
(free_camera.rs:382); map_stream (24) for MapStream
(map_stream.rs:153); globe (20) for GlobeState (globe.rs:146) and
HideInSpace (globe.rs:549); data_layers (17) for DataLayers
(data_layers.rs:305); egui_manager (16) for EguiManager
(egui_manager.rs:45). resources.rs and components.rs were meant to be that
layer, but 33 of the 50 system files declare Resource/Component types too.

These are the same problem: because the shared types are filed next to their
systems, the crate graph cannot be split (Rust crates cannot be cyclic), and
7,313 Bevy-free lines carrying 69 of the crate's 290 tests are compiled and
linked inside a Bevy binary.

Approach

Four phases, in order. Each is independently landable and independently
valuable; stop after any of them.

Phase 0 — build wins with no architecture risk.

Add a [lib] target to crates/viberfox; main.rs becomes a thin bin over
it, so tests link the rlib instead of building a second full copy.
Collapse dev and dev-bevy into one dev profile; update the commands in
CLAUDE.md and docs/guides/development.md to match.
debug = "line-tables-only" for workspace crates in dev — keeps backtraces,
cuts the codegen that dominates the 45 s. CI already sets this.

Phase 1 — give systems/ a shape. Pure mod moves, no code changes:

systems/map/    map_stream tile_loader tile_source globe globe_ocean globe_weather
                osm_buildings lod22 terrain height water vegetation
                street_furniture street_labels map_geometry rendering
systems/nav/    navigation routing route_ahead driver geo_nav transit_live transit_vehicles
systems/player/ avatar collision free_camera picking gizmo prim_geo
systems/ui/     egui_manager theme ui hud dock bookmarks place_pins chat ai_assistant splash
systems/net/    network user_store login
systems/dev/    shot_harness shot_presets desktop_integration window_icon

Phase 2 — the layering. This is where the cycles die.

crates/state/ (package viberfox_state): every Resource/Component type
and shared constant, absorbing today's resources.rs and components.rs
plus the hub types listed above. It is a crate rather than a module because
that is the only thing that makes re-filing a type next to its system a
compile error — a module boundary cannot enforce it, which is how the drift
happened.
crates/geo/ (package viberfox_geo): the Bevy-free half — vector_tiles,
routing, road_texture, shot_presets, utils/sun. Fixes the inverted
arrow at vector_tiles.rs:2297, where an MVT decoder calls
vegetation::scatter_vegetation.
Rename vibe_sim → simulator. Rename the vibe_core directory to
crates/core/ with package name viberfox_core — a package literally
named core shadows the sysroot crate in every dependent
(docs/notes/module-graph.md has the verified error).

Phase 3 — only if phase 2's measurements justify it. Split the Bevy half
into map / nav / ui crates to fan codegen across cores. Deliberately last:
it is the most work and the least certain return, and phase 2 may be enough.

Acceptance criteria

cargo test -p viberfox runs the suite without compiling the crate twice
One dev profile; target/ well under the measured 170 GB after a clean rebuild
systems/ has no more than ~8 entries at its top level
crates/geo builds with no bevy dependency, and cargo test -p viberfox_geo
runs its 69 tests without building Bevy
No Resource or Component type is declared outside crates/state
Zero mutual cycles between modules (re-run the edge-list method in
docs/notes/module-graph.md)
Crates are core (pkg viberfox_core), geo, state, simulator,
viberfox, big_space
docs/notes/client-build-cost.md updated with post-change measurements by
the same method

Verification

cargo check --workspace --all-targets
cargo test -p viberfox and cargo test -p big_space --lib — the suite is
the guard here, not cargo check, which does not compile cfg(test) code
cargo shots — the README image set must be byte-comparable in content; this
is a refactor with no intended visual change, so a diff in any of the six
images means something moved
Re-time the table in docs/notes/client-build-cost.md on the same machine
(22-core workstation) so the before/after is method-comparable

Out of scope

Any behavioural change. This is a move-and-rename; if a system's logic needs
fixing, that is a separate ticket.
Cold-build time. Bevy dominates it and no rearrangement of our crates changes
that.
The vendored crates/big_space fork — untouched.
Phase 3, unless phase 2's re-measurement shows the codegen tail still hurts.
The wasm build's profile tuning (web-release), which is separately tuned and
documented in the workspace Cargo.toml.

Branch: refactor/16-client-crate-layering

## Problem Two problems with one root cause. **Builds are slow where the incremental cache does not reach.** Measured 2026-08-06 (`docs/notes/client-build-cost.md`): a leaf edit rebuilds in 3.8 s, but compiling `viberfox` with a cold incremental cache takes 52.4 s, of which only 7.1 s is the frontend — ~45 s is codegen and linking. Everything that discards the cache pays that in full: CI (`CARGO_INCREMENTAL=0`), profile switches, flipping `solo`/`fast-dev`/`scalar-fallback`, and every `cargo test`. Two structural taxes make it worse: `crates/viberfox/Cargo.toml` declares no `[lib]`, so `cargo test -p viberfox --bin viberfox` compiles all ~52k lines a second time under `cfg(test)`; and `dev` and `dev-bevy` are both maintained, so `target/` measured 170 GB and alternating between the two commands in `CLAUDE.md` rebuilds the other set. **`systems/` has become a flat bin of 50 files holding both the logic and the state that logic coordinates on.** Measured over the module graph (`docs/notes/module-graph.md`): 26 mutual dependency cycles, and the high-in-degree modules are imported for a single state type each, not for their behaviour — `free_camera` (in-degree 25) for `FreeCamera` (`crates/viberfox/src/systems/free_camera.rs:16`) and `WORLD_CELL_EDGE` (`free_camera.rs:382`); `map_stream` (24) for `MapStream` (`map_stream.rs:153`); `globe` (20) for `GlobeState` (`globe.rs:146`) and `HideInSpace` (`globe.rs:549`); `data_layers` (17) for `DataLayers` (`data_layers.rs:305`); `egui_manager` (16) for `EguiManager` (`egui_manager.rs:45`). `resources.rs` and `components.rs` were meant to be that layer, but 33 of the 50 system files declare `Resource`/`Component` types too. These are the same problem: because the shared types are filed next to their systems, the crate graph cannot be split (Rust crates cannot be cyclic), and 7,313 Bevy-free lines carrying 69 of the crate's 290 tests are compiled and linked inside a Bevy binary. ## Approach Four phases, in order. Each is independently landable and independently valuable; stop after any of them. **Phase 0 — build wins with no architecture risk.** - Add a `[lib]` target to `crates/viberfox`; `main.rs` becomes a thin bin over it, so tests link the rlib instead of building a second full copy. - Collapse `dev` and `dev-bevy` into one dev profile; update the commands in `CLAUDE.md` and `docs/guides/development.md` to match. - `debug = "line-tables-only"` for workspace crates in dev — keeps backtraces, cuts the codegen that dominates the 45 s. CI already sets this. **Phase 1 — give `systems/` a shape.** Pure `mod` moves, no code changes: ``` systems/map/ map_stream tile_loader tile_source globe globe_ocean globe_weather osm_buildings lod22 terrain height water vegetation street_furniture street_labels map_geometry rendering systems/nav/ navigation routing route_ahead driver geo_nav transit_live transit_vehicles systems/player/ avatar collision free_camera picking gizmo prim_geo systems/ui/ egui_manager theme ui hud dock bookmarks place_pins chat ai_assistant splash systems/net/ network user_store login systems/dev/ shot_harness shot_presets desktop_integration window_icon ``` **Phase 2 — the layering.** This is where the cycles die. - `crates/state/` (package `viberfox_state`): every `Resource`/`Component` type and shared constant, absorbing today's `resources.rs` and `components.rs` plus the hub types listed above. It is a *crate* rather than a module because that is the only thing that makes re-filing a type next to its system a compile error — a module boundary cannot enforce it, which is how the drift happened. - `crates/geo/` (package `viberfox_geo`): the Bevy-free half — `vector_tiles`, `routing`, `road_texture`, `shot_presets`, `utils/sun`. Fixes the inverted arrow at `vector_tiles.rs:2297`, where an MVT decoder calls `vegetation::scatter_vegetation`. - Rename `vibe_sim` → `simulator`. Rename the `vibe_core` directory to `crates/core/` with package name **`viberfox_core`** — a package literally named `core` shadows the sysroot crate in every dependent (`docs/notes/module-graph.md` has the verified error). **Phase 3 — only if phase 2's measurements justify it.** Split the Bevy half into `map` / `nav` / `ui` crates to fan codegen across cores. Deliberately last: it is the most work and the least certain return, and phase 2 may be enough. ## Acceptance criteria - [ ] `cargo test -p viberfox` runs the suite without compiling the crate twice - [ ] One dev profile; `target/` well under the measured 170 GB after a clean rebuild - [ ] `systems/` has no more than ~8 entries at its top level - [ ] `crates/geo` builds with no `bevy` dependency, and `cargo test -p viberfox_geo` runs its 69 tests without building Bevy - [ ] No `Resource` or `Component` type is declared outside `crates/state` - [ ] Zero mutual cycles between modules (re-run the edge-list method in `docs/notes/module-graph.md`) - [ ] Crates are `core` (pkg `viberfox_core`), `geo`, `state`, `simulator`, `viberfox`, `big_space` - [ ] `docs/notes/client-build-cost.md` updated with post-change measurements by the same method ## Verification - `cargo check --workspace --all-targets` - `cargo test -p viberfox` and `cargo test -p big_space --lib` — the suite is the guard here, not `cargo check`, which does not compile `cfg(test)` code - `cargo shots` — the README image set must be byte-comparable in content; this is a refactor with no intended visual change, so a diff in any of the six images means something moved - Re-time the table in `docs/notes/client-build-cost.md` on the same machine (22-core workstation) so the before/after is method-comparable ## Out of scope - Any behavioural change. This is a move-and-rename; if a system's logic needs fixing, that is a separate ticket. - Cold-build time. Bevy dominates it and no rearrangement of our crates changes that. - The vendored `crates/big_space` fork — untouched. - Phase 3, unless phase 2's re-measurement shows the codegen tail still hurts. - The wasm build's profile tuning (`web-release`), which is separately tuned and documented in the workspace `Cargo.toml`. --- Branch: `refactor/16-client-crate-layering`

jeroen added the

driven

label

2026-08-06 11:41:29 +00:00

viberfox-agent added this to the Viberfox project

2026-08-06 11:41:52 +00:00

jeroen referenced this issue from a commit

2026-08-06 11:42:15 +00:00

docs(notes): measure where the client's build time and module coupling actually go

jeroen commented

2026-08-06 13:13:01 +00:00

Author

Owner

Phase 2's state crate was attempted on 2026-08-06 and reverted. The plan above underestimated it, and the correction is load-bearing for whoever picks this up.

The hub types are not state — they are streamers wearing a Resource hat. Counting fields that carry machinery (a CellStream, a HashMap keyed to Entity, an asset Handle):

Type	Fields	Of which machinery
`GlobeState`	8	6
`DataLayers`	8	5
`MapStream`	9	5
`DriverState`	10	0
`BuildingColliders`	2	0
`TerrainField`	2	0

So the three highest-in-degree types are exactly the three that cannot move. GlobeState also calls free_camera::WORLD_CELL_EDGE and utils::sun::smoothstep from its own methods; moving it produced 33 errors.

The method mistake is worth repeating so it is not repeated: the struct was judged portable by reading its impl, which is self-contained, without reading its fields, which are not.

Revised precondition. Each hub needs a split, not a move — the state half everyone reads separated from the machinery half one system owns, exactly the operation vegetation needed (870 pure / 130 ECS). That is three non-trivial refactors ahead of the state crate itself. The genuinely portable types break only 2–3 of the 22 remaining cycles on their own, because most cycles have one of the big three on the other side.

Detail and measurements: docs/notes/module-graph.md.

Done so far (all on main, verified, tests green):

lib target + shim bin — fd72850
one dev profile, line-tables-only — 0f82945
systems/ grouped into folders — 30c3e96
crates/geo extracted; edit→60 tests in 2.2 s — 3ef118e
renames: simulator, viberfox_core — ac06d22
state crate — blocked on the three hub splits above
phase 3 — hold until the full compile is re-measured on an unloaded machine

Unrelated but measured: KDE's baloo_file indexes target/ — 591,959 files here, out of 1,088,961 in its whole index. It ignores CACHEDIR.TAG. Adding target to exclude filters in ~/.config/baloofilerc is a free win and makes build timings trustworthy again; load average was 12.7 during this session, which invalidated a full-compile measurement.

Phase 2's state crate was attempted on 2026-08-06 and reverted. The plan above underestimated it, and the correction is load-bearing for whoever picks this up. **The hub types are not state — they are streamers wearing a `Resource` hat.** Counting fields that carry machinery (a `CellStream`, a `HashMap` keyed to `Entity`, an asset `Handle`): | Type | Fields | Of which machinery | |---|---|---| | `GlobeState` | 8 | **6** | | `DataLayers` | 8 | **5** | | `MapStream` | 9 | **5** | | `DriverState` | 10 | 0 | | `BuildingColliders` | 2 | 0 | | `TerrainField` | 2 | 0 | So the three highest-in-degree types are exactly the three that cannot move. `GlobeState` also calls `free_camera::WORLD_CELL_EDGE` and `utils::sun::smoothstep` from its own methods; moving it produced 33 errors. The method mistake is worth repeating so it is not repeated: the struct was judged portable by reading its `impl`, which is self-contained, without reading its **fields**, which are not. **Revised precondition.** Each hub needs a *split*, not a move — the state half everyone reads separated from the machinery half one system owns, exactly the operation `vegetation` needed (870 pure / 130 ECS). That is three non-trivial refactors ahead of the state crate itself. The genuinely portable types break only 2–3 of the 22 remaining cycles on their own, because most cycles have one of the big three on the other side. Detail and measurements: `docs/notes/module-graph.md`. **Done so far** (all on `main`, verified, tests green): - [x] lib target + shim bin — `fd72850` - [x] one dev profile, `line-tables-only` — `0f82945` - [x] `systems/` grouped into folders — `30c3e96` - [x] `crates/geo` extracted; edit→60 tests in 2.2 s — `3ef118e` - [x] renames: `simulator`, `viberfox_core` — `ac06d22` - [ ] state crate — **blocked on the three hub splits above** - [ ] phase 3 — hold until the full compile is re-measured on an unloaded machine **Unrelated but measured:** KDE's `baloo_file` indexes `target/` — 591,959 files here, out of 1,088,961 in its whole index. It ignores `CACHEDIR.TAG`. Adding `target` to `exclude filters` in `~/.config/baloofilerc` is a free win and makes build timings trustworthy again; load average was 12.7 during this session, which invalidated a full-compile measurement.

jeroen referenced this issue from a commit

2026-08-06 13:14:36 +00:00

docs(notes): record why the hub types cannot be moved to a state layer