Three branches were merged in 63 minutes while `main` was already red, and every merge commit was red #282

New issue

Closed

opened 2026-09-03 00:59:01 +00:00 by viberfox-agent · 0 comments

viberfox-agent commented

2026-09-03 00:59:01 +00:00

Collaborator

Found by the 7-day retrospective on #279, reading the Actions task list against
main's history.

What happened on 2 September

Three tickets were merged into main inside 63 minutes. Every branch head was
green. Every merge commit was red. Nothing stopped between them.

	commit	`test cartopolis`
last green `main`	`a76cbbe1`	success (30 Aug)
#231 branch head	`0a9836d7`	success 07:10
merged as PR #273	`f9e457b7`	failure 07:25
#233 branch head	`b2b92339`	success 07:25
merged as PR #274	`e0e4de69`	failure 08:13 and 08:15
#267 branch head	`16091087`	success 08:06
merged as PR #275	`9e5ed895`	failure 08:28
repair (#276) merged	`0b63cd7d`	success 09:41

Each merge was followed, on its own ticket, by:

🤖 Merged, but not deployed. The deploy step exited 1:
refusing: CI is not green on …

So the lane was told in plain words at 08:01 that main was red, and merged again
at 08:06 and at 08:28.

#276 established the cause of the last one: #273 and #274 were both cut from
a76cbbe, each was green on its own, and they change opposite sides of one
function — CrossingKind::spec() went from a 3-tuple to a 2-tuple in one while
the other added two callers of the 3-tuple. git merge joins them cleanly and the
result does not compile. (Three commits were also pushed to main directly in the
same window, 2094b35…b395686, so f9e457b7's failure is not necessarily the
same fault; the pattern is what matters, not which change it was.)

Why the existing guards did not catch it

Two guards exist and both sit on the wrong side of the merge:

tools/deploy-main gates on a green main. It refused, correctly, three
times — after each merge. It protects the public services, not main.
generate() refuses to file while main is red, and says why: "every branch
is cut from main, so while it is broken each new ticket inherits the failure and
stalls at its own CI." That reasoning applies at least as strongly to merging,
and shipReady does not make the check.

Note the autopilot's own landing path already avoids half of this by design —
landOnMain fast-forwards to the exact commit CI passed on, with the comment "a
merge commit is a commit nobody built" — but these three landed as pull-request
merges, which is a merge commit nobody built, on top of a main nobody re-checked.

What would fix it

Two changes, independent:

Do not land while main is red. shipReady already has jobsFor; one call
against main's head, and a red answer parks the tick with a line saying so.
That alone turns three bad merges into one.
A branch verdict expires when main moves. A green run on a commit cut from
an older main says nothing about the merge. Either re-dispatch CI on the merge
result before landing, or require the branch to be fast-forwardable onto the
current main — which is what landOnMain already enforces, and which would
have refused #274 outright and asked for a rebase.

Where the seam is

tools/autopilot.ts — shipReady (:281), and whichever path merged these as
pull requests rather than through landOnMain (:434).

_{Filed by the 7-day retrospective (#279). Not labelled autonomous — it edits

tools/autopilot.ts, which the fence refuses to land, and three finished branches

are already waiting for review.}

Found by the 7-day retrospective on #279, reading the Actions task list against `main`'s history. ## What happened on 2 September Three tickets were merged into `main` inside 63 minutes. Every branch head was green. Every merge commit was red. Nothing stopped between them. | | commit | `test cartopolis` | |---|---|---| | last green `main` | `a76cbbe1` | **success** (30 Aug) | | #231 branch head | `0a9836d7` | **success** 07:10 | | merged as PR #273 | `f9e457b7` | **failure** 07:25 | | #233 branch head | `b2b92339` | **success** 07:25 | | merged as PR #274 | `e0e4de69` | **failure** 08:13 *and* 08:15 | | #267 branch head | `16091087` | **success** 08:06 | | merged as PR #275 | `9e5ed895` | **failure** 08:28 | | repair (#276) merged | `0b63cd7d` | **success** 09:41 | Each merge was followed, on its own ticket, by: > 🤖 **Merged, but not deployed.** The deploy step exited 1: > `refusing: CI is not green on …` So the lane was told in plain words at 08:01 that `main` was red, and merged again at 08:06 and at 08:28. #276 established the cause of the last one: #273 and #274 were both cut from `a76cbbe`, each was green on its own, and they change opposite sides of one function — `CrossingKind::spec()` went from a 3-tuple to a 2-tuple in one while the other added two callers of the 3-tuple. `git merge` joins them cleanly and the result does not compile. (Three commits were also pushed to `main` directly in the same window, `2094b35`…`b395686`, so `f9e457b7`'s failure is not necessarily the same fault; the pattern is what matters, not which change it was.) ## Why the existing guards did not catch it Two guards exist and both sit on the wrong side of the merge: - **`tools/deploy-main` gates on a green `main`.** It refused, correctly, three times — *after* each merge. It protects the public services, not `main`. - **`generate()` refuses to file while `main` is red**, and says why: *"every branch is cut from `main`, so while it is broken each new ticket inherits the failure and stalls at its own CI."* That reasoning applies at least as strongly to merging, and `shipReady` does not make the check. Note the autopilot's own landing path already avoids half of this by design — `landOnMain` fast-forwards to the exact commit CI passed on, with the comment *"a merge commit is a commit nobody built"* — but these three landed as pull-request merges, which is a merge commit nobody built, on top of a `main` nobody re-checked. ## What would fix it Two changes, independent: 1. **Do not land while `main` is red.** `shipReady` already has `jobsFor`; one call against `main`'s head, and a red answer parks the tick with a line saying so. That alone turns three bad merges into one. 2. **A branch verdict expires when `main` moves.** A green run on a commit cut from an older `main` says nothing about the merge. Either re-dispatch CI on the merge result before landing, or require the branch to be fast-forwardable onto the current `main` — which is what `landOnMain` already enforces, and which would have refused #274 outright and asked for a rebase. ## Where the seam is `tools/autopilot.ts` — `shipReady` (`:281`), and whichever path merged these as pull requests rather than through `landOnMain` (`:434`). <sub>Filed by the 7-day retrospective (#279). Not labelled `autonomous` — it edits `tools/autopilot.ts`, which the fence refuses to land, and three finished branches are already waiting for review.</sub>

viberfox-agent added the

autopilot

label

2026-09-03 00:59:01 +00:00

viberfox-agent added this to the Viberfox project

2026-09-03 00:59:23 +00:00

viberfox-agent referenced this issue

2026-09-03 01:00:28 +00:00

Retrospective: the last 7 day(s), and what to do differently #279

viberfox-agent referenced this issue

2026-09-04 00:30:53 +00:00

alreadyFiled only ever sees 50 issues, so every autopilot dedupe key silently expires — #206 was QA'd twice #281