BGT, the remaining layers: is it worth wiring in?

viberfox-agent commented

2026-08-29 17:19:15 +00:00

Collaborator

Problem

docs/direction.md:54 lists BGT, the remaining layers as an unverified frontier candidate: "the surveyed surfaces not yet read." Nobody had confirmed the interface, the licence or the volume. That check is the first half of this ticket, and it is done below — but the row names a category, not a change: the BGT publishes 49 collections and systems::surveyed reads seven of them.

What is read today: vegetatieobject_punt (trees), paal + straatmeubilair (lighting and furniture), scheiding_lijn (fences and walls), vegetatieobject_vlak (hedges), wegdeel + ondersteunendwegdeel (paving). The wire tables are crates/cartopolis/src/systems/map/surveyed.rs:72 (PROP_KINDS), :84 (LINE_KINDS) and :88 (SURFACE_KINDS), all marked append, never renumber.

The reconnaissance (2026-08-29, this container, read-only HTTP)

Interface — confirmed unchanged: https://api.pdok.nl/lv/bgt/ogc/v1, keyless, cursor-paged, no numberMatched, every historical version returned. Exactly what the retired extractor documented (recovered at git show 8c4b766^:tools/bgt_objects.py, header lines 10–20). The termination_date filter is still mandatory and still the thing that silently doubles a layer.

Licence — confirmed CC0 1.0, stated by the API itself: the collections document carries {"rel":"license","title":"CC0 1.0","href":"…/publicdomain/zero/1.0/"}. Redistributing the cached cells is unambiguously allowed; the attribution line in surveyed.rs:65 is courtesy, not obligation.

Volume and content — one page (limit=1000) per collection over a 1 km² box of central Groningen (bbox=6.5590,53.2149,6.5740,53.2239), and a second pass over ~10×9 km (bbox=6.4800,53.1800,6.6300,53.2600) for the collections the small box returned empty. Current-version counts only.

A finding worth recording on its own: the 1 km² of central Groningen returns zero features for every point collection — paal, straatmeubilair, bak, bord, put, kast, mast, sensor_punt, vegetatieobject_punt — while the same collections are well populated over the wider city. This is not a bbox bug: a 100 m box around a known paal returns 10. It is the same municipality-level coverage hole docs/notes/surveyed-street-objects.md already records for trees, and it is wider than trees.

A stale claim in our own note. docs/notes/surveyed-street-objects.md:211 says scheiding_vlak's "type is empty on every feature seen so far, so there is nothing to say whether one is a wall, a bank or a planter." That is wrong today, and probably describes plus_type. Measured over the 1 km² box: 223 current features, type filled on all of them — muur 140, kademuur 71, transitie 12; plus_type is None on all 223.

Approach

The answer to "is it worth doing" is: yes, for five collections, no for the other thirty-seven. The bar a collection has to clear, applied uniformly:

no other source the client already reads supplies it,
crates/geo/src/furniture.rs already has an honest form for it (or one small form covers it),
it is visible at the scale the layer is drawn at,
it goes beside existing content rather than replacing it — the rule surveyed-street-objects.md was built on.

Accept (five)

Collection	Current features	Form that already exists
`scheiding_vlak`	223 / km² (`muur` 140, `kademuur` 71, `transitie` 12)	`LineKind::Wall`, `crates/geo/src/furniture.rs:872`
`vegetatieobject_lijn`	33 / city (`haag` 32)	`LineKind::Hedge`, `furniture.rs:874`
`bord`	843 / city (`verkeersbord` 629, `wegwijzer` 80, `informatiebord` 52, `straatnaambord` 17)	`FurnitureKind::Board`, `furniture.rs:122`
`kast`	162 / city (`elektrakast` 75, `openbare verlichtingkast` 29, `CAI-kast` 24)	`FurnitureKind::Cabinet`, `furniture.rs:120`
`bak`	34 / city (`afvalbak` 15, `container` 11)	`FurnitureKind::Bin` / `Recycling`, `furniture.rs:113`, `:115`

Three of the five need no new client type at all — bord, kast and bak are new entries in the extractor's type→form table and, at most, appended entries in PROP_KINDS (surveyed.rs:72) if a form is not yet in the list. vegetatieobject_lijn is a hedge already surveyed as a line, so it skips the hedge_centreline reduction the area version needs (bgt_objects.py:353) and reaches LineKind::Hedge directly. scheiding_vlak takes the same centreline reduction hedges take, and drops kademuur for the reason scheiding_lijn already drops walbescherming/damwand/kademuur (note:214) — a metre of masonry along every canal on the side the map already draws as a bank.

Reject, with the reason

Another source already reads it: pand, pand_nummeraanduiding (BAG / 3DBAG), spoor (OSM textured ribbons), waterdeel, ondersteunendwaterdeel, begroeidterreindeel, onbegroeidterreindeel (the tile's own water and land fills feed the shore-distance field, the grass mask and the globe's ocean mask — the standing reason in surveyed-street-objects.md), mast (1 current over the city) and kunstwerkdeel_punt (99 hoogspanningsmast) — crates/cartopolis/src/systems/map/tall_structures.rs:1-17 already streams masts, pylons and towers from Overpass at z11 over ±30 km, which is a wider ring than a z14 BGT cell can give them.
Administrative, changes no picture: buurt, wijk, stadsdeel, waterschap, openbareruimte, functioneelgebied.
Empty or unusable: weginrichtingselement_vlak (0 current / km²), weginrichtingselement_punt (8 over the city, all niet-bgt), weginrichtingselement_lijn (molgoot only — a gutter channel, finer than the paving layer's own 10 cm simplification), ongeclassificeerdobject and overigescheiding (both Historisch).
No honest form at this scale: put (973 over the city, 787 of them kolk — a gully grate is a texture, not geometry), sensor_punt (71, mostly cameras), installatie (23, all zonnepaneel — rooftop), kunstwerkdeel_lijn (511, 434 duiker — culverts, underground), overigbouwwerk (34, mostly lage trafo), gebouwinstallatie (16 toegangstrap/bordes). This is the rule the note already states: an unstyled thing beats a wrongly styled one.
Its own ticket, not this one: overbruggingsdeel (20 / km²) and tunneldeel (45 / city) — bridge decks and tunnel bodies. This is the one rejected item with real picture value, because the paving extractor drops every surface at relatieve_hoogteligging != 0 (bgt_objects.py:485), so a bridge gets no surveyed paving today. Drawing it needs a deck height the BGT does not carry and a rung on the depth ladder the flat paving layer does not have. That is a layer, not a collection.
Would replace rather than add: openbareruimtelabel (217 / km², points carrying a rotation — surveyed street-name placements). systems::street_labels places labels view-independently from world footprints on purpose; substituting surveyed anchors is a different argument and outside the shape this ticket names.
Belongs to a different module: begroeidterreindeel_kruinlijn, onbegroeidterreindeel_kruinlijn, ondersteunendwegdeel_kruinlijn, wegdeel_kruinlijn — crest lines, whose consumer would be systems::dikes, not this layer.

Files it touches

cartopy (jeroen/cartopy, server/pipeline/bgt.ts): five new collections in the walk, the type→form tables extended, scheiding_vlak reduced to centrelines by the same routine the hedges use, kademuur/transitie filtered out. server/pipeline/format.test.ts gains the new shapes.

cartopolis:

crates/cartopolis/src/systems/map/surveyed.rs — append to PROP_KINDS (:72) only if a form is missing; no renumbering, no new file, no new coverage slot. The parse functions (:206, :212) already skip an index past the end of the table.
crates/geo/src/furniture.rs — nothing, if the five map onto Bin, Recycling, Board, Cabinet, Wall, Hedge as the counts above suggest.
docs/notes/surveyed-street-objects.md — correct the scheiding_vlak claim at :211, record the point-collection coverage hole in central Groningen, and list what stays unread and why.
docs/direction.md:54 — replace the candidate row with the recorded verdict: interface confirmed, licence CC0, five collections wired, the rest listed as rejected with reasons, overbruggingsdeel/tunneldeel split off.

Acceptance criteria

docs/direction.md:54's row records the verdict: the OGC interface, the CC0 licence, which collections were wired and which were rejected with the reason for each.
docs/notes/surveyed-street-objects.md:211 no longer claims scheiding_vlak.type is empty; it states what was measured (muur 140, kademuur 71, transitie 12 over 1 km² of central Groningen, 2026-08-29) and that plus_type is the empty field.
The note records the point-collection coverage hole: paal, straatmeubilair, bak, bord, put, kast, mast, sensor_punt and vegetatieobject_punt all return zero current features over that same 1 km², while the wider city has them.
The extractor reads scheiding_vlak, vegetatieobject_lijn, bord, kast and bak, filters on termination_date, and drops kademuur, transitie and niet-bgt.
scheiding_vlak polygons are reduced to a centreline, not laid as boxes round their outline — the same rule hedges follow, for the same reason.
No index in PROP_KINDS, LINE_KINDS, SURFACE_KINDS (surveyed.rs:72, :84, :88) is renumbered; any addition is appended, and cartopy's tables match position for position.
A cell file produced before this change still parses on the new client, and a cell file produced after it still parses on the old one (unknown indices drop the object, they do not clamp — surveyed.rs:206).
The generated layers are unchanged where the register is silent: a cell with no bord/kast/bak/scheiding_vlak records renders exactly as it did.
cargo fmt --check is clean across the workspace.
Nothing new is added to PROTOCOL_HISTORY and no migration is added — this is neither a wire nor a schema change.

Verification

Reconnaissance is reproducible from any host with outbound HTTPS, no build:

curl -s "https://api.pdok.nl/lv/bgt/ogc/v1/collections?f=json" | \
  python3 -c "import json,sys; d=json.load(sys.stdin); print(len(d['collections'])); \
    print([l for l in d['links'] if l['rel']=='license'])"

# per-collection current-feature counts over 1 km2 of central Groningen
curl -s "https://api.pdok.nl/lv/bgt/ogc/v1/collections/scheiding_vlak/items?\
bbox=6.5590,53.2149,6.5740,53.2239&limit=1000&f=json" | \
  python3 -c "import json,sys,collections; d=json.load(sys.stdin); \
    cur=[f for f in d['features'] if not f['properties'].get('termination_date')]; \
    print(len(cur), collections.Counter(f['properties'].get('type') for f in cur))"

Client side, on a machine that may build:

cargo test -p cartopolis_geo
cargo test -p cartopolis          # surveyed.rs parse tests
cargo fmt --check

cartopy side: bun install then ./node_modules/.bin/tsc --noEmit and bun test server/pipeline/format.test.ts.

Only on a host that can render (not this container): a scene gate over a re-extracted Groningen box. --dump-state already carries surveyed_props and surveyed_lines (crates/cartopolis/src/systems/dev/shot_harness.rs:558, :560), so the check is an --expect 'surveyed_lines>…' against the counts the extractor reports, plus --expect on an existing layer's counts to prove nothing moved. Whether the picture reads right — a signboard beside a kerb, a wall along a garden — is a look, and per docs/direction.md is not a machine gate.

Not measured here, and it should be before the extractor is run at scale: the per-cell byte cost of the five collections. Paving is already the second-heaviest thing the client streams at 3.8 MB for the densest cell; these five go into the objects file, which is 60 kB for that same cell (surveyed.rs:243). The command is a full extractor run over the 1 km² box and du -b on the produced bgt/14/*/*.json.

Out of scope

The carriageways. wegdeel's rijbaan *, spoorbaan and OV-baan stay with OpenStreetMap for the reason crates/geo/src/paving.rs:7-34 gives, which this ticket does not reopen.
The terrain and water fills (begroeidterreindeel, onbegroeidterreindeel, waterdeel, ondersteunendwaterdeel).
Bridge decks and tunnels (overbruggingsdeel, tunneldeel) — worth doing, own ticket, needs a height source and a depth rung.
Surveyed street-name placements (openbareruimtelabel) — would replace systems::street_labels' placement rule.
Any new collider or interaction: the boundaries stay non-solid (surveyed-street-objects.md), and a signboard is not interactable — world_places::prop_interactable is the existing split.
Re-running the pipeline over the cities the host already serves. That is an operator action in cartopy's Data tab, not part of this branch.

Open questions

This lane cannot read the repository the extractor lives in. The FORGEJO_TOKEN in this container reads jeroen/cartopolis (HTTP 200) but not jeroen/cartopy (HTTP 404 — not visible; /user answers 403 for missing read:user). git clone over both code.garage44.eu and the internal 172.17.0.1:3000 fails with "Repository not found". Half of this change is server/pipeline/bgt.ts, so the implementing agent needs either a token that can read and push cartopy, or the ticket has to be split so the cartopolis half lands here and the extractor half is done elsewhere. Which?
An already-cached cell never sees the new content. load_surveyed_cell returns the cached copy whenever one exists (surveyed.rs:281) and the coverage manifest carries no data version — zoom, cells, attribution and nothing else (crates/cartopolis/src/systems/map/coverage.rs:167-193); CoverageSlot::revision can only ever change once per process (coverage.rs:133). So re-running the extractor reaches no client that has already visited the city. Three ways out, and the choice affects three trees, not one: (a) accept it and let eviction handle it; (b) add a version to coverage.json and mix it into the cache key, which is a change to the shared CoverageSlot; (c) put the new content behind a new path so old keys simply miss. Which?
Is the coverage hole worth reporting upstream, or just recorded? Central Groningen has no current paal, straatmeubilair, bak, bord, put or kast records at all, which means the lampposts this layer was built for are absent from precisely the densest part of the city it was measured on. Recording it in the note is cheap; deciding whether the map should say something about it is not mine to make.

Branch: feat/176-bgt-remaining-collections

Original request

From the frontier in docs/direction.md: BGT, the remaining layers — the surveyed surfaces not yet read.

The first half of this ticket is deciding whether it is worth doing at all:

Is there an interface a client or an extractor can use, and what does it cost per cell?
Does the licence allow redistributing what we would cache?
Does it change the picture, or only the data behind it?

Not worth it is a valid answer. Record it in docs/direction.md against this row
with the reason, and close this ticket — that is the result, not a failure.

If it is worth doing, follow the shape the surveyed layers already use:
an extractor under tools/, a coverage manifest, a streamer that adds to what
is drawn rather than replacing it, and no unbounded per-frame upload.

_{Filed by the autopilot.}

_{🤖 Refined by the viberfox issue agent. Reply with @agent refine and what is wrong to have this rewritten.}

## Problem `docs/direction.md:54` lists **BGT, the remaining layers** as an unverified frontier candidate: "the surveyed surfaces not yet read." Nobody had confirmed the interface, the licence or the volume. That check is the first half of this ticket, and it is done below — but the row names a *category*, not a change: the BGT publishes **49 collections** and `systems::surveyed` reads **seven** of them. What is read today: `vegetatieobject_punt` (trees), `paal` + `straatmeubilair` (lighting and furniture), `scheiding_lijn` (fences and walls), `vegetatieobject_vlak` (hedges), `wegdeel` + `ondersteunendwegdeel` (paving). The wire tables are `crates/cartopolis/src/systems/map/surveyed.rs:72` (`PROP_KINDS`), `:84` (`LINE_KINDS`) and `:88` (`SURFACE_KINDS`), all marked *append, never renumber*. ### The reconnaissance (2026-08-29, this container, read-only HTTP) **Interface** — confirmed unchanged: `https://api.pdok.nl/lv/bgt/ogc/v1`, keyless, cursor-paged, no `numberMatched`, every historical version returned. Exactly what the retired extractor documented (recovered at `git show 8c4b766^:tools/bgt_objects.py`, header lines 10–20). The `termination_date` filter is still mandatory and still the thing that silently doubles a layer. **Licence** — confirmed **CC0 1.0**, stated by the API itself: the `collections` document carries `{"rel":"license","title":"CC0 1.0","href":"…/publicdomain/zero/1.0/"}`. Redistributing the cached cells is unambiguously allowed; the attribution line in `surveyed.rs:65` is courtesy, not obligation. **Volume and content** — one page (`limit=1000`) per collection over a 1 km² box of central Groningen (`bbox=6.5590,53.2149,6.5740,53.2239`), and a second pass over ~10×9 km (`bbox=6.4800,53.1800,6.6300,53.2600`) for the collections the small box returned empty. Current-version counts only. A finding worth recording on its own: **the 1 km² of central Groningen returns zero features for every point collection** — `paal`, `straatmeubilair`, `bak`, `bord`, `put`, `kast`, `mast`, `sensor_punt`, `vegetatieobject_punt` — while the same collections are well populated over the wider city. This is not a bbox bug: a 100 m box around a known `paal` returns 10. It is the same municipality-level coverage hole `docs/notes/surveyed-street-objects.md` already records for trees, and it is wider than trees. **A stale claim in our own note.** `docs/notes/surveyed-street-objects.md:211` says `scheiding_vlak`'s "`type` is empty on every feature seen so far, so there is nothing to say whether one is a wall, a bank or a planter." That is wrong today, and probably describes `plus_type`. Measured over the 1 km² box: 223 current features, `type` filled on all of them — `muur` 140, `kademuur` 71, `transitie` 12; `plus_type` is `None` on all 223. ## Approach **The answer to "is it worth doing" is: yes, for five collections, no for the other thirty-seven.** The bar a collection has to clear, applied uniformly: 1. no other source the client already reads supplies it, 2. `crates/geo/src/furniture.rs` already has an honest form for it (or one small form covers it), 3. it is visible at the scale the layer is drawn at, 4. it goes *beside* existing content rather than replacing it — the rule `surveyed-street-objects.md` was built on. ### Accept (five) | Collection | Current features | Form that already exists | |---|---|---| | `scheiding_vlak` | 223 / km² (`muur` 140, `kademuur` 71, `transitie` 12) | `LineKind::Wall`, `crates/geo/src/furniture.rs:872` | | `vegetatieobject_lijn` | 33 / city (`haag` 32) | `LineKind::Hedge`, `furniture.rs:874` | | `bord` | 843 / city (`verkeersbord` 629, `wegwijzer` 80, `informatiebord` 52, `straatnaambord` 17) | `FurnitureKind::Board`, `furniture.rs:122` | | `kast` | 162 / city (`elektrakast` 75, `openbare verlichtingkast` 29, `CAI-kast` 24) | `FurnitureKind::Cabinet`, `furniture.rs:120` | | `bak` | 34 / city (`afvalbak` 15, `container` 11) | `FurnitureKind::Bin` / `Recycling`, `furniture.rs:113`, `:115` | Three of the five need **no new client type at all** — `bord`, `kast` and `bak` are new entries in the extractor's type→form table and, at most, appended entries in `PROP_KINDS` (`surveyed.rs:72`) if a form is not yet in the list. `vegetatieobject_lijn` is a hedge already surveyed *as a line*, so it skips the `hedge_centreline` reduction the area version needs (`bgt_objects.py:353`) and reaches `LineKind::Hedge` directly. `scheiding_vlak` takes the same centreline reduction hedges take, and drops `kademuur` for the reason `scheiding_lijn` already drops `walbescherming`/`damwand`/`kademuur` (note:214) — a metre of masonry along every canal on the side the map already draws as a bank. ### Reject, with the reason - **Another source already reads it:** `pand`, `pand_nummeraanduiding` (BAG / 3DBAG), `spoor` (OSM textured ribbons), `waterdeel`, `ondersteunendwaterdeel`, `begroeidterreindeel`, `onbegroeidterreindeel` (the tile's own water and land fills feed the shore-distance field, the grass mask and the globe's ocean mask — the standing reason in `surveyed-street-objects.md`), `mast` (1 current over the city) and `kunstwerkdeel_punt` (99 `hoogspanningsmast`) — `crates/cartopolis/src/systems/map/tall_structures.rs:1-17` already streams masts, pylons and towers from Overpass at z11 over ±30 km, which is a wider ring than a z14 BGT cell can give them. - **Administrative, changes no picture:** `buurt`, `wijk`, `stadsdeel`, `waterschap`, `openbareruimte`, `functioneelgebied`. - **Empty or unusable:** `weginrichtingselement_vlak` (0 current / km²), `weginrichtingselement_punt` (8 over the city, all `niet-bgt`), `weginrichtingselement_lijn` (`molgoot` only — a gutter channel, finer than the paving layer's own 10 cm simplification), `ongeclassificeerdobject` and `overigescheiding` (both *Historisch*). - **No honest form at this scale:** `put` (973 over the city, 787 of them `kolk` — a gully grate is a texture, not geometry), `sensor_punt` (71, mostly cameras), `installatie` (23, all `zonnepaneel` — rooftop), `kunstwerkdeel_lijn` (511, 434 `duiker` — culverts, underground), `overigbouwwerk` (34, mostly `lage trafo`), `gebouwinstallatie` (16 `toegangstrap`/`bordes`). This is the rule the note already states: an unstyled thing beats a wrongly styled one. - **Its own ticket, not this one:** `overbruggingsdeel` (20 / km²) and `tunneldeel` (45 / city) — bridge decks and tunnel bodies. This is the one rejected item with real picture value, because the paving extractor drops every surface at `relatieve_hoogteligging != 0` (`bgt_objects.py:485`), so a bridge gets no surveyed paving today. Drawing it needs a deck *height* the BGT does not carry and a rung on the depth ladder the flat paving layer does not have. That is a layer, not a collection. - **Would replace rather than add:** `openbareruimtelabel` (217 / km², points carrying a rotation — surveyed street-name placements). `systems::street_labels` places labels view-independently from world footprints on purpose; substituting surveyed anchors is a different argument and outside the shape this ticket names. - **Belongs to a different module:** `begroeidterreindeel_kruinlijn`, `onbegroeidterreindeel_kruinlijn`, `ondersteunendwegdeel_kruinlijn`, `wegdeel_kruinlijn` — crest lines, whose consumer would be `systems::dikes`, not this layer. ### Files it touches **cartopy** (`jeroen/cartopy`, `server/pipeline/bgt.ts`): five new collections in the walk, the type→form tables extended, `scheiding_vlak` reduced to centrelines by the same routine the hedges use, `kademuur`/`transitie` filtered out. `server/pipeline/format.test.ts` gains the new shapes. **cartopolis:** - `crates/cartopolis/src/systems/map/surveyed.rs` — append to `PROP_KINDS` (`:72`) only if a form is missing; no renumbering, no new file, no new coverage slot. The parse functions (`:206`, `:212`) already skip an index past the end of the table. - `crates/geo/src/furniture.rs` — nothing, if the five map onto `Bin`, `Recycling`, `Board`, `Cabinet`, `Wall`, `Hedge` as the counts above suggest. - `docs/notes/surveyed-street-objects.md` — correct the `scheiding_vlak` claim at `:211`, record the point-collection coverage hole in central Groningen, and list what stays unread and why. - `docs/direction.md:54` — replace the candidate row with the recorded verdict: interface confirmed, licence CC0, five collections wired, the rest listed as rejected with reasons, `overbruggingsdeel`/`tunneldeel` split off. ## Acceptance criteria - [ ] `docs/direction.md:54`'s row records the verdict: the OGC interface, the CC0 licence, which collections were wired and which were rejected with the reason for each. - [ ] `docs/notes/surveyed-street-objects.md:211` no longer claims `scheiding_vlak.type` is empty; it states what was measured (`muur` 140, `kademuur` 71, `transitie` 12 over 1 km² of central Groningen, 2026-08-29) and that `plus_type` is the empty field. - [ ] The note records the point-collection coverage hole: `paal`, `straatmeubilair`, `bak`, `bord`, `put`, `kast`, `mast`, `sensor_punt` and `vegetatieobject_punt` all return zero current features over that same 1 km², while the wider city has them. - [ ] The extractor reads `scheiding_vlak`, `vegetatieobject_lijn`, `bord`, `kast` and `bak`, filters on `termination_date`, and drops `kademuur`, `transitie` and `niet-bgt`. - [ ] `scheiding_vlak` polygons are reduced to a centreline, not laid as boxes round their outline — the same rule hedges follow, for the same reason. - [ ] No index in `PROP_KINDS`, `LINE_KINDS`, `SURFACE_KINDS` (`surveyed.rs:72`, `:84`, `:88`) is renumbered; any addition is appended, and cartopy's tables match position for position. - [ ] A cell file produced before this change still parses on the new client, and a cell file produced after it still parses on the old one (unknown indices drop the object, they do not clamp — `surveyed.rs:206`). - [ ] The generated layers are unchanged where the register is silent: a cell with no `bord`/`kast`/`bak`/`scheiding_vlak` records renders exactly as it did. - [ ] `cargo fmt --check` is clean across the workspace. - [ ] Nothing new is added to `PROTOCOL_HISTORY` and no migration is added — this is neither a wire nor a schema change. ## Verification Reconnaissance is reproducible from any host with outbound HTTPS, no build: ```bash curl -s "https://api.pdok.nl/lv/bgt/ogc/v1/collections?f=json" | \ python3 -c "import json,sys; d=json.load(sys.stdin); print(len(d['collections'])); \ print([l for l in d['links'] if l['rel']=='license'])" # per-collection current-feature counts over 1 km2 of central Groningen curl -s "https://api.pdok.nl/lv/bgt/ogc/v1/collections/scheiding_vlak/items?\ bbox=6.5590,53.2149,6.5740,53.2239&limit=1000&f=json" | \ python3 -c "import json,sys,collections; d=json.load(sys.stdin); \ cur=[f for f in d['features'] if not f['properties'].get('termination_date')]; \ print(len(cur), collections.Counter(f['properties'].get('type') for f in cur))" ``` Client side, on a machine that may build: ```bash cargo test -p cartopolis_geo cargo test -p cartopolis # surveyed.rs parse tests cargo fmt --check ``` cartopy side: `bun install` then `./node_modules/.bin/tsc --noEmit` and `bun test server/pipeline/format.test.ts`. **Only on a host that can render** (not this container): a scene gate over a re-extracted Groningen box. `--dump-state` already carries `surveyed_props` and `surveyed_lines` (`crates/cartopolis/src/systems/dev/shot_harness.rs:558`, `:560`), so the check is an `--expect 'surveyed_lines>…'` against the counts the extractor reports, plus `--expect` on an existing layer's counts to prove nothing moved. Whether the picture reads right — a signboard beside a kerb, a wall along a garden — is a look, and per `docs/direction.md` is not a machine gate. **Not measured here, and it should be before the extractor is run at scale:** the per-cell byte cost of the five collections. Paving is already the second-heaviest thing the client streams at 3.8 MB for the densest cell; these five go into the *objects* file, which is 60 kB for that same cell (`surveyed.rs:243`). The command is a full extractor run over the 1 km² box and `du -b` on the produced `bgt/14/*/*.json`. ## Out of scope - The carriageways. `wegdeel`'s `rijbaan *`, `spoorbaan` and `OV-baan` stay with OpenStreetMap for the reason `crates/geo/src/paving.rs:7-34` gives, which this ticket does not reopen. - The terrain and water fills (`begroeidterreindeel`, `onbegroeidterreindeel`, `waterdeel`, `ondersteunendwaterdeel`). - Bridge decks and tunnels (`overbruggingsdeel`, `tunneldeel`) — worth doing, own ticket, needs a height source and a depth rung. - Surveyed street-name placements (`openbareruimtelabel`) — would replace `systems::street_labels`' placement rule. - Any new collider or interaction: the boundaries stay non-solid (`surveyed-street-objects.md`), and a signboard is not interactable — `world_places::prop_interactable` is the existing split. - Re-running the pipeline over the cities the host already serves. That is an operator action in cartopy's Data tab, not part of this branch. ## Open questions 1. **This lane cannot read the repository the extractor lives in.** The `FORGEJO_TOKEN` in this container reads `jeroen/cartopolis` (HTTP 200) but not `jeroen/cartopy` (HTTP 404 — not visible; `/user` answers 403 for missing `read:user`). `git clone` over both `code.garage44.eu` and the internal `172.17.0.1:3000` fails with "Repository not found". Half of this change is `server/pipeline/bgt.ts`, so the implementing agent needs either a token that can read and push cartopy, or the ticket has to be split so the cartopolis half lands here and the extractor half is done elsewhere. **Which?** 2. **An already-cached cell never sees the new content.** `load_surveyed_cell` returns the cached copy whenever one exists (`surveyed.rs:281`) and the coverage manifest carries no data version — `zoom`, `cells`, `attribution` and nothing else (`crates/cartopolis/src/systems/map/coverage.rs:167-193`); `CoverageSlot::revision` can only ever change once per process (`coverage.rs:133`). So re-running the extractor reaches no client that has already visited the city. Three ways out, and the choice affects three trees, not one: (a) accept it and let eviction handle it; (b) add a `version` to `coverage.json` and mix it into the cache key, which is a change to the shared `CoverageSlot`; (c) put the new content behind a new path so old keys simply miss. **Which?** 3. **Is the coverage hole worth reporting upstream, or just recorded?** Central Groningen has no current `paal`, `straatmeubilair`, `bak`, `bord`, `put` or `kast` records at all, which means the lampposts this layer was built for are absent from precisely the densest part of the city it was measured on. Recording it in the note is cheap; deciding whether the map should say something about it is not mine to make. --- Branch: `feat/176-bgt-remaining-collections` <details><summary>Original request</summary> From the frontier in [`docs/direction.md`](../src/branch/main/docs/direction.md): **BGT, the remaining layers** — the surveyed surfaces not yet read. The first half of this ticket is deciding whether it is worth doing at all: - Is there an interface a client or an extractor can use, and what does it cost per cell? - Does the licence allow redistributing what we would cache? - Does it change the picture, or only the data behind it? **Not worth it is a valid answer.** Record it in `docs/direction.md` against this row with the reason, and close this ticket — that is the result, not a failure. If it is worth doing, follow the shape the surveyed layers already use: an extractor under `tools/`, a coverage manifest, a streamer that adds to what is drawn rather than replacing it, and no unbounded per-frame upload. Filed by the autopilot.  </details> 🤖 Refined by the viberfox issue agent. Reply with **@agent refine** and what is wrong to have this rewritten.

viberfox-agent added the

labels

2026-08-29 17:19:15 +00:00

viberfox-agent added this to the Viberfox project

2026-08-29 17:19:26 +00:00

viberfox-agent commented

2026-08-29 17:19:47 +00:00

Author

Collaborator

🤖 Refining this before anything gets built — reading the code, no branch, no commits. If it turns out I'd have to guess at the requirements I'll stop and ask instead.

viberfox-agent added

agent:needs-input

and removed

agent:refining

labels

2026-08-29 17:28:42 +00:00

viberfox-agent commented

2026-08-29 17:28:43 +00:00

Author

Collaborator

🤖 This needs a decision before I build anything. The open questions are in the description above.

Answer them and reply with @agent refine to have the spec rewritten, or @agent to make me implement it as it stands.

_{🤖 viberfox issue agent on the VPS — 36 turns · $3.30 · session 420e631c-a0ae-4653-9586-72f4a32c1de1. Reply with @agent to continue this session.}

🤖 **This needs a decision before I build anything.** The open questions are in the description above. Answer them and reply with **@agent refine** to have the spec rewritten, or **@agent** to make me implement it as it stands. --- 🤖 viberfox issue agent on the VPS — 36 turns · $3.30 · session `420e631c-a0ae-4653-9586-72f4a32c1de1`. Reply with **@agent** to continue this session.

viberfox-agent commented

2026-08-29 17:45:56 +00:00

Author

Collaborator

🤖 Nobody is going to answer this, so decide it yourself.

The seven values at the top of docs/direction.md are there to settle exactly this: measured over plausible, official sources first, degrade rather than break, add rather than replace, keep the frame budget, ship the smallest thing a machine can judge, and write down what you decided.

Pick the reading those support, record which one you applied in the commit body, and build it. Stop again only for something no one can derive from the repository — a credential, a licence somebody must accept, or a question about what the project is for.

🤖 **Nobody is going to answer this, so decide it yourself.** The seven values at the top of [`docs/direction.md`](../src/branch/main/docs/direction.md) are there to settle exactly this: measured over plausible, official sources first, degrade rather than break, add rather than replace, keep the frame budget, ship the smallest thing a machine can judge, and write down what you decided. Pick the reading those support, record which one you applied in the commit body, and build it. Stop again only for something no one can derive from the repository — a credential, a licence somebody must accept, or a question about what the project is for.

viberfox-agent added

agent

agent:running

and removed

agent:needs-input

labels

2026-08-29 17:45:56 +00:00

viberfox-agent commented

2026-08-29 17:46:07 +00:00

Author

Collaborator

🤖 Picked this up — working on branch feat/176-bgt-remaining-collections in a fresh worktree on the VPS. I'll report back here when I'm done.

_{Label an issue agent:skip to keep me off it, or driven to keep it in a session you are driving.}

🤖 Picked this up — working on branch `feat/176-bgt-remaining-collections` in a fresh worktree on the VPS. I'll report back here when I'm done. Label an issue `agent:skip` to keep me off it, or `driven` to keep it in a session you are driving.

viberfox-agent referenced this issue from a commit

2026-08-29 17:46:45 +00:00

feat(direction): seven values to decide by, and a lane that stops stopping

Rows
Columns

BGT, the remaining layers: is it worth wiring in? #176

Problem

The reconnaissance (2026-08-29, this container, read-only HTTP)

Approach

Accept (five)

Reject, with the reason

Files it touches

Acceptance criteria

Verification

Out of scope

Open questions

What changed

What the sweep found that the ticket did not

How I verified it

Not verified

For you to decide