ci: run conformance on both wires from the automatic triggers

yuris commented

2026-08-21 09:54:18 +00:00

Owner

Gate 1 of #89. Touches .forgejo/workflows/conformance.yml only.

v2 was reachable only by workflow_dispatch, so the binary wire had no automatic gate — it cannot become the default while that is true. The conformance job is now a matrix over protocol: [v1, v2], with fail-fast: false so one wire failing still reports the other, which is the whole point of comparing them. The workflow_dispatch input is repurposed from "opt into v2" to "narrow the matrix to one wire".

Matrix narrowing uses fromJSON(format(...)) on inputs.protocol. That was verified against the parser this instance actually runs rather than against GitHub's docs: the host reports Forgejo 15.0.1, whose go.mod pins forgejo/runner/v12 v12.10.2, where act/jobparser/jobparser.go:102-103 evaluates strategy.matrix with inputs in scope and act/exprparser/interpreter.go registers both format and fromjson. On push and PR there are no inputs, so the expression short-circuits to ["v1","v2"]. If that reading were still wrong the failure is bounded and loud: automatic triggers keep expanding to both wires, only manual narrowing would degrade to running both.

The HTML5 step loses continue-on-error: true and now gates both legs. The issue-#81 self-report step stays — this Forgejo serves no job logs over the API, so that comment is the only readable trace of the tier. Two legs mean two #81 comments per run; accepted, no dedup built. If the tier flakes during the observation window the rollback is pre-decided and named in the step comment: restore the line, evidence on #81.

&conformance_paths is unchanged (verified byte-identical after explode) and already lists this workflow file, so this PR triggers the new matrix on itself.

Gate 1 of #89. Touches `.forgejo/workflows/conformance.yml` only. v2 was reachable only by `workflow_dispatch`, so the binary wire had no automatic gate — it cannot become the default while that is true. The conformance job is now a matrix over `protocol: [v1, v2]`, with `fail-fast: false` so one wire failing still reports the other, which is the whole point of comparing them. The `workflow_dispatch` input is repurposed from "opt into v2" to "narrow the matrix to one wire". Matrix narrowing uses `fromJSON(format(...))` on `inputs.protocol`. That was verified against the parser this instance actually runs rather than against GitHub's docs: the host reports Forgejo 15.0.1, whose go.mod pins `forgejo/runner/v12 v12.10.2`, where `act/jobparser/jobparser.go:102-103` evaluates `strategy.matrix` with `inputs` in scope and `act/exprparser/interpreter.go` registers both `format` and `fromjson`. On push and PR there are no inputs, so the expression short-circuits to `["v1","v2"]`. If that reading were still wrong the failure is bounded and loud: automatic triggers keep expanding to both wires, only manual narrowing would degrade to running both. The HTML5 step loses `continue-on-error: true` and now gates both legs. The issue-#81 self-report step stays — this Forgejo serves no job logs over the API, so that comment is the only readable trace of the tier. Two legs mean two #81 comments per run; accepted, no dedup built. If the tier flakes during the observation window the rollback is pre-decided and named in the step comment: restore the line, evidence on #81. `&conformance_paths` is unchanged (verified byte-identical after explode) and already lists this workflow file, so this PR triggers the new matrix on itself.

yuris added 1 commit

2026-08-21 09:54:18 +00:00

ci: run conformance over both wires on every trigger

ci / ci (pull_request) Successful in 20s

Details

conformance / conformance (v1) (pull_request) Successful in 2m23s

Details

conformance / conformance (v2) (pull_request) Successful in 2m47s

Details

8e715275b3

Gate 1 of #89. The v2 wire was reachable only by `workflow_dispatch`, so the
protocol that is about to become the default had never been held to a gate on
a commit that changed it. This makes the conformance job a matrix over
`protocol: [v1, v2]`, so both wires run on every push to `main` and every pull
request matching `&conformance_paths`.

Only `.forgejo/workflows/conformance.yml` changes. The path filter and
`conformance.sh` are untouched — the harness already keys off `STDB_PROTOCOL`,
which now comes from `matrix.protocol` instead of `inputs.protocol || 'v1'`.
No default protocol moves in this commit; `stdb/init.lua` and the harness
scripts still resolve to v1.

## The dispatch input changed meaning

`protocol` no longer opts *into* v2. It narrows the matrix to a single wire, so
a dispatch can re-run one leg against the HTML5 flake without paying for the
other. Automatic triggers carry no inputs, so `inputs.protocol` is null there
and the expression falls through to both wires.

That fall-through is load-bearing, and expressions in `strategy.matrix` are not
something to assume: if the value were taken literally, `STDB_PROTOCOL` would
be a `${{ … }}` string and both legs would run against nothing. It was verified
against the parser this host actually runs, not against GitHub's docs. This
Forgejo reports `15.0.1+gitea-1.22.0`, and `v15.0`'s `go.mod` pins
`code.forgejo.org/forgejo/runner/v12 v12.10.2`, where:

- `act/jobparser/jobparser.go:102` builds the matrix evaluator with `pc.inputs`
  in scope and evaluates `job.Strategy.RawMatrix` through it, so `inputs` is a
  valid context in a matrix value.
- `act/jobparser/evaluator.go` walks the matrix mapping and re-encodes each
  evaluated value, turning the scalar expression into a sequence node — which
  is what makes a `fromJSON` array expand into matrix legs.
- `act/exprparser/interpreter.go` registers both `format` and `fromjson`.
- the same file resolves a missing `inputs` key to nil rather than an error, and
  `IsTruthy(nil)` is false, so `null && … || …` short-circuits to the two-wire
  array on push and pull request.

Dynamic matrices shipped in Forgejo v14.0, and v15.0's reference lists `inputs`
among the contexts available to `jobs.<job-id>.strategy`.

## The HTML5 tier gates now

`continue-on-error: true` is gone from the HTML5 step. It was there because run
716 failed and run 717 passed on the same commit (#81), and a red X nobody can
explain is worse than a footnote. The trade has changed: HTML5 is the only place
the wasm build of the binary wire runs at all, so it has to gate before v2 can
be the default, and a flake that blocks a merge gets diagnosed where one that
prints a footnote never does. If it flakes twice, restore the line and say so
on #81.

The self-report step stays. This host serves no job logs over the API, so
without that comment a failure leaves nothing a script can read. Two matrix
legs post two comments per run, which is cheaper than teaching the step to
deduplicate them.

`fail-fast: false`, because when one wire breaks the other's verdict is the
most useful thing on the page: it says whether the break is protocol-specific
or took the whole harness down.

## Verification

- `yq 'explode(.) | (.on.pull_request.paths | to_json) == (.on.push.paths |
  to_json)'` is `true` — the `&conformance_paths` anchor still resolves, and
  the exploded list is byte-identical to `main`'s. Note that yq's `==` never
  deep-compares sequences (`yq -n '[1,2] == [1,2]'` is `false`), so the naive
  form of that query reports `false` on an unmodified `main` too.
- `.jobs.conformance.env.STDB_PROTOCOL` reads `${{ matrix.protocol }}`; no step
  carries `continue-on-error`; the six steps including the #81 report are all
  still present.
- `lua dev/test/runner.lua`: 11 passed, 0 failed. `stylua --check stdb dev/test`
  clean. Neither should have moved, and neither did.

The matrix itself is proved by the pull request's own run, since this file is in
the path filter: both legs must be green before it merges.

yuris added 1 commit

2026-08-21 10:04:13 +00:00

ci: name the wire in the HTML5 self-report

ci / ci (pull_request) Successful in 20s

Details

conformance / conformance (v1) (pull_request) Successful in 3m21s

Details

conformance / conformance (v2) (pull_request) Successful in 2m41s

Details

e50fffede6

The matrix runs this job twice per commit, and the report body was built from
run_number, outcome and sha -- all three identical across legs. So the two
comments on #81 were distinguishable only by their outcome word, and only when
the log tail happened to name a protocol: html5-conformance.mjs prints it on
the PASS line and two late assertions, so an early failure (wasm build, nix
develop, Chromium launch) posted a tail with no wire in it at all. Those are
the failures the comment exists to triage, and this API serves no job logs, so
"which wire was this?" had no answer.

Also corrects the header's cost figure, which still priced a triggering commit
at one leg and so contradicted the `on:` comment added alongside the matrix.

yuris referenced this pull request

2026-08-21 10:04:26 +00:00

Flip the default protocol to v2 (2.0.0) #89

yuris merged commit 68476250a1 into main

2026-08-21 10:11:06 +00:00

yuris deleted branch flip-ci

2026-08-21 10:11:06 +00:00

yuris referenced this pull request

2026-08-21 10:11:06 +00:00

Flip the default protocol to v2 (2.0.0) #89

yuris referenced this pull request from a commit

2026-08-21 10:11:06 +00:00