yuris/spacetimedb-defold
1
0
Fork 0
Code Issues 1 Pull requests Projects Releases 9 Packages Wiki Activity Actions

Flip the default protocol to v2 (2.0.0) #89

New issue
Closed
opened 2026-08-21 09:42:42 +00:00 by yuris · 6 comments
yuris commented 2026-08-21 09:42:42 +00:00
Owner
Copy link

The flip decision memo lives in the closing comments of #9; this issue tracks executing it.

Shipping straight to 2.0.0 (no 1.2.0 deprecation release). v1 JSON stays a first-class, CI-covered opt-out via connect({ protocol = "v1" }).

  • Gate 1 — CI: v2 in the automatic triggers, both wires per PR (PR: flip-ci)
  • Gate 2 — confirmed_reads: match peer SDKs (nil omits the param; server default applies), pinned by tests (PR: flip-confirmed)
  • Gate 3 — v2 smoke parity: one-off query, sliced apply over BSATN, inline-result stand-in for the light assertion (PR: flip-smoke-parity)
  • Gate 4 — observation: automatic conformance green on both wires for the PRs above
  • Gate 5 — the flip itself: default v2, version 2.0.0, docs in the same commit (PR: flip-default)
  • Post-merge: wiki sweep, tag v2.0.0 (annotation = changelog), close this issue

HTML5 tier gating is attempted in Gate 1 (dropping continue-on-error on the HTML5 step for both matrix legs). If it flakes during the observation window the fallback is pre-decided: restore continue-on-error: true, post the flake evidence to #81, record the exemption here, and proceed — the flip does not wait on #81's flake having a cause.

The flip decision memo lives in the closing comments of #9; this issue tracks executing it. Shipping straight to `2.0.0` (no 1.2.0 deprecation release). v1 JSON stays a first-class, CI-covered opt-out via `connect({ protocol = "v1" })`. - [x] Gate 1 — CI: v2 in the automatic triggers, both wires per PR (PR: flip-ci) - [x] Gate 2 — confirmed_reads: match peer SDKs (nil omits the param; server default applies), pinned by tests (PR: flip-confirmed) - [x] Gate 3 — v2 smoke parity: one-off query, sliced apply over BSATN, inline-result stand-in for the light assertion (PR: flip-smoke-parity) - [x] Gate 4 — observation: automatic conformance green on both wires for the PRs above - [x] Gate 5 — the flip itself: default v2, version 2.0.0, docs in the same commit (PR: flip-default) - [x] Post-merge: wiki sweep, tag v2.0.0 (annotation = changelog), close this issue HTML5 tier gating is attempted in Gate 1 (dropping `continue-on-error` on the HTML5 step for both matrix legs). If it flakes during the observation window the fallback is pre-decided: restore `continue-on-error: true`, post the flake evidence to #81, record the exemption here, and proceed — the flip does not wait on #81's flake having a cause.
yuris commented 2026-08-21 09:52:31 +00:00
Author
Owner
Copy link

Gate 2 done — #90 merged (squash).

confirmed_reads matches the official SDKs and is now pinned: nil omits the parameter (server default: false on v1, true on v2), false is the documented opt-out, true is sent as asked. No production change was needed — subscribe_url already behaved this way; what landed is the test that fails if anyone writes the default out as confirmed=false.

Review caught one real defect in the first cut: the confirmed=true assertion sat behind a composition case that already pinned the same URL, so it was unreachable and a dropped confirmed reported as a parameter-ordering failure. Reordered, then verified by swallowing a v2 confirmed_reads = true in subscribe_url and confirming the tri-state assertion's own message is what reports.

**Gate 2 done** — #90 merged (squash). `confirmed_reads` matches the official SDKs and is now pinned: `nil` omits the parameter (server default: false on v1, true on v2), `false` is the documented opt-out, `true` is sent as asked. No production change was needed — `subscribe_url` already behaved this way; what landed is the test that fails if anyone writes the default out as `confirmed=false`. Review caught one real defect in the first cut: the `confirmed=true` assertion sat behind a composition case that already pinned the same URL, so it was unreachable and a dropped `confirmed` reported as a parameter-ordering failure. Reordered, then verified by swallowing a v2 `confirmed_reads = true` in `subscribe_url` and confirming the tri-state assertion's own message is what reports.
yuris commented 2026-08-21 10:04:26 +00:00
Author
Owner
Copy link

Two optional nits from #91's review, recorded rather than actioned (both are pre-existing or capacity-dependent, neither blocks the flip):

  1. No max-parallel on the conformance matrix. /nix is a persisted volume shared across runs, and the install step falls through to curl | sh when it is cold. Before the matrix a single run could not produce two concurrent jobs; now it can, so on the first run after a volume reset two unsynchronised installers could race it. The window is narrow (the warm-store branch short-circuits every subsequent run) and #91's own run confirmed the legs do run concurrently, which is what buys back the wall time — ~2.5 min rather than ~5. Left concurrent deliberately; max-parallel: 1 is the fix if that race is ever actually observed.

  2. A dispatch can no longer run both wires. required: false paired with a non-empty default: v1 means Forgejo substitutes the default for an empty submission, so inputs.protocol is always truthy on dispatch and the matrix always narrows to one leg. Main behaved the same way, so nothing regressed. Dropping default: would fall through to both legs, but that collides with Gate 5's step of flipping this default to v2. Recorded so the interaction is not rediscovered later.

Also worth carrying forward from #92: STDB_BULK_BUDGET is in seconds, and at the default 10000-row sweep a v2 sliced run does not slice at all — BSATN decode finishes the apply in one frame, so the PHASE6 gate fails. Pre-existing on main. The correct sliced proof is STDB_PROTOCOL=v2 STDB_BULK_BUDGET=0.008 STDB_BULK_SWEEP=50000; Gate 5's verification list uses that, superseding the plan's STDB_BULK_BUDGET=8.

Two optional nits from #91's review, recorded rather than actioned (both are pre-existing or capacity-dependent, neither blocks the flip): 1. **No `max-parallel` on the conformance matrix.** `/nix` is a persisted volume shared across runs, and the install step falls through to `curl | sh` when it is cold. Before the matrix a single run could not produce two concurrent jobs; now it can, so on the first run after a volume reset two unsynchronised installers could race it. The window is narrow (the warm-store branch short-circuits every subsequent run) and #91's own run confirmed the legs do run concurrently, which is what buys back the wall time — ~2.5 min rather than ~5. Left concurrent deliberately; `max-parallel: 1` is the fix if that race is ever actually observed. 2. **A dispatch can no longer run both wires.** `required: false` paired with a non-empty `default: v1` means Forgejo substitutes the default for an empty submission, so `inputs.protocol` is always truthy on dispatch and the matrix always narrows to one leg. Main behaved the same way, so nothing regressed. Dropping `default:` would fall through to both legs, but that collides with Gate 5's step of flipping this default to `v2`. Recorded so the interaction is not rediscovered later. Also worth carrying forward from #92: `STDB_BULK_BUDGET` is in **seconds**, and at the default 10000-row sweep a v2 sliced run does not slice at all — BSATN decode finishes the apply in one frame, so the PHASE6 gate fails. Pre-existing on main. The correct sliced proof is `STDB_PROTOCOL=v2 STDB_BULK_BUDGET=0.008 STDB_BULK_SWEEP=50000`; Gate 5's verification list uses that, superseding the plan's `STDB_BULK_BUDGET=8`.
yuris commented 2026-08-21 10:11:06 +00:00
Author
Owner
Copy link

Gate 1 done — #91 merged (squash).

Conformance is now a matrix over protocol: [v1, v2] with fail-fast: false, so both wires gate every push to main and every PR matching &conformance_paths. v2 is no longer dispatch-only, which was the precondition for making it the default.

The fromJSON/format matrix fall-through is not a guess — #91's own run expanded into two legs, conformance (v1) and conformance (v2), both success on e50fffe. That also settles Gate 4's open question: this Forgejo does publish per-leg commit statuses, queryable at GET /commits/{sha}/status, so the pre-decided local-re-run fallback is not needed.

The HTML5 step no longer has continue-on-error, so the tier gates both legs. Review caught that the #81 self-report body was built only from run_number/outcome/sha — identical across legs — so the two comments were indistinguishable exactly on the early failures (wasm build, nix develop, Chromium launch) whose tails never name a wire. It now prints matrix.protocol.

**Gate 1 done** — #91 merged (squash). Conformance is now a matrix over `protocol: [v1, v2]` with `fail-fast: false`, so both wires gate every push to `main` and every PR matching `&conformance_paths`. v2 is no longer dispatch-only, which was the precondition for making it the default. The `fromJSON`/`format` matrix fall-through is not a guess — #91's own run expanded into two legs, `conformance (v1)` and `conformance (v2)`, both success on `e50fffe`. That also settles Gate 4's open question: **this Forgejo does publish per-leg commit statuses**, queryable at `GET /commits/{sha}/status`, so the pre-decided local-re-run fallback is not needed. The HTML5 step no longer has `continue-on-error`, so the tier gates both legs. Review caught that the #81 self-report body was built only from run_number/outcome/sha — identical across legs — so the two comments were indistinguishable exactly on the early failures (wasm build, nix develop, Chromium launch) whose tails never name a wire. It now prints `matrix.protocol`.
yuris commented 2026-08-21 10:44:57 +00:00
Author
Owner
Copy link

Gate 3 done — #92 merged (squash).

The v2 smoke now asserts everything the v1 one does: a one-off query on both wires (neither had it), a sliced BSATN apply at unit level (test_slicing.lua never touched row_decoder), and an inline-delivery assertion standing in for v1's light = true check.

Review caught that the first cut of that stand-in was a tautology: connection.lua hands the reducer callback and the transaction event the same table unconditionally, so rawequal compared a value to itself, and a commit arriving in its own frame is a TransactionUpdate with no request_id, so the guard was skipped entirely. Proven by driving the real connection with an OkEmpty ReducerResult. Under a real regression the run would have died on a generic 90s timeout, saying nothing about the claim it was there to defend — in the one PR whose entire purpose is proving v2 parity.

Fixed by reading the settling frame's own counts.events for the insert, which is frame-local; find_row would have been satisfied by a row left behind by an earlier run. Proven to bite: patching the outcome to the OkEmpty shape now fails immediately with the send_message result settled the call but its own frame announced no insert. The v2 one-off row check was likewise strengthened from "non-empty string" (which an error string would satisfy) to a real BSATN decode with a field assertion.

Gate 4 done as well. Both legs green on three separate PR-triggered runs — e50fffe (#91), 1b044ba (#93), a743295 (#92) — read from GET /commits/{sha}/status, which this Forgejo does publish per matrix leg. No exemption or local-re-run fallback needed.

Unplanned but necessary: #93 landed in between, fixing the #81 HTML5 flake at its cause (chromium's child processes outliving a parent-only kill, so rm raced live profile writers). Gating the tier in Gate 1 is what made it diagnosable — it turned an invisible footnote into a failure with a captured tail on its second run.

**Gate 3 done** — #92 merged (squash). The v2 smoke now asserts everything the v1 one does: a one-off query on both wires (neither had it), a sliced BSATN apply at unit level (`test_slicing.lua` never touched `row_decoder`), and an inline-delivery assertion standing in for v1's `light = true` check. Review caught that the first cut of that stand-in was a **tautology**: `connection.lua` hands the reducer callback and the transaction event the same table unconditionally, so `rawequal` compared a value to itself, and a commit arriving in its own frame is a `TransactionUpdate` with no `request_id`, so the guard was skipped entirely. Proven by driving the real connection with an `OkEmpty` `ReducerResult`. Under a real regression the run would have died on a generic 90s timeout, saying nothing about the claim it was there to defend — in the one PR whose entire purpose is proving v2 parity. Fixed by reading the settling frame's own `counts.events` for the insert, which is frame-local; `find_row` would have been satisfied by a row left behind by an earlier run. Proven to bite: patching the outcome to the `OkEmpty` shape now fails immediately with `the send_message result settled the call but its own frame announced no insert`. The v2 one-off row check was likewise strengthened from "non-empty string" (which an error string would satisfy) to a real BSATN decode with a field assertion. **Gate 4 done** as well. Both legs green on three separate PR-triggered runs — `e50fffe` (#91), `1b044ba` (#93), `a743295` (#92) — read from `GET /commits/{sha}/status`, which this Forgejo does publish per matrix leg. No exemption or local-re-run fallback needed. Unplanned but necessary: #93 landed in between, fixing the #81 HTML5 flake at its cause (chromium's child processes outliving a parent-only kill, so `rm` raced live profile writers). Gating the tier in Gate 1 is what made it diagnosable — it turned an invisible footnote into a failure with a captured tail on its second run.
yuris commented 2026-08-21 11:18:40 +00:00
Author
Owner
Copy link

Gate 5 done — #94 merged (squash). connect() now defaults to protocol = "v2", game.project reads 2.0.0.

The flip exposed a real latent bug, which is the best argument for having done it behind gates. subscribe_url composed its query string from opts.protocol rather than from the wire connect had resolved. Harmless while the default was v1 and the two always agreed; under a v2 default a defaulted caller left opts.protocol nil, so the socket negotiated BSATN with no compression=None — which the codec then refuses — and light = true slipped past the v2 guard onto the wire. It is how the new default assertion and test_slicing first failed. subscribe_url now takes the resolved protocol; both halves are pinned by tests.

The test sweep was also wider than the plan assumed: native_url drives connect, so every assertion built on it inherited the default, including a block explicitly asserting "the pre-1.0 URLs, byte for byte". That block now names its wire on every call, so those assertions keep testing composition instead of silently becoming default-tracking tests.

Review caught two docs defects. The README claimed all four v2 breaks "find out at connect" — only light does; the reducers/procedures and subscribe raises land at the callsite, and an undescribed table is not refused at all but skipped and reported, which the same paragraph's own last sentence already said. And connection.lua's module header still called v2 opt-in, contradicting two other comments rewritten in the same commit.

Verified on the merge candidate: unit suite 11/11, stylua clean, and all five engine runs — native default names v2, native STDB_PROTOCOL=v1 names v1, sliced 50000 rows over 6 frames at 8.0ms, HTML5 default asserts the v2.bsatn.spacetimedb handshake, HTML5 v1 passes. Both CI legs green on da95c8e.

**Gate 5 done** — #94 merged (squash). `connect()` now defaults to `protocol = "v2"`, `game.project` reads `2.0.0`. **The flip exposed a real latent bug**, which is the best argument for having done it behind gates. `subscribe_url` composed its query string from `opts.protocol` rather than from the wire `connect` had resolved. Harmless while the default was v1 and the two always agreed; under a v2 default a defaulted caller left `opts.protocol` nil, so the socket negotiated BSATN with **no** `compression=None` — which the codec then refuses — and `light = true` slipped past the v2 guard onto the wire. It is how the new default assertion and `test_slicing` first failed. `subscribe_url` now takes the resolved protocol; both halves are pinned by tests. The test sweep was also wider than the plan assumed: `native_url` drives `connect`, so every assertion built on it inherited the default, including a block explicitly asserting "the pre-1.0 URLs, byte for byte". That block now names its wire on every call, so those assertions keep testing composition instead of silently becoming default-tracking tests. Review caught two docs defects. The README claimed all four v2 breaks "find out at `connect`" — only `light` does; the reducers/procedures and `subscribe` raises land at the callsite, and an undescribed table is not refused at all but skipped and reported, which the same paragraph's own last sentence already said. And `connection.lua`'s module header still called v2 opt-in, contradicting two other comments rewritten in the same commit. Verified on the merge candidate: unit suite 11/11, stylua clean, and all five engine runs — native default names v2, native `STDB_PROTOCOL=v1` names v1, sliced 50000 rows over 6 frames at 8.0ms, HTML5 default asserts the `v2.bsatn.spacetimedb` handshake, HTML5 v1 passes. Both CI legs green on `da95c8e`.
yuris commented 2026-08-21 11:33:48 +00:00
Author
Owner
Copy link

Done. Shipped as v2.0.0.

PR Gate Merge SHA
#91 flip-ci 1 — both wires in the automatic triggers 6847625
#90 flip-confirmed 2 — confirmed_reads pinned to peer parity b71a6a0
#93 fix-html5-teardown unplanned — #81's flake, fixed at its cause c46cd42
#92 flip-smoke-parity 3 — v2 smoke at parity with v1's 8b0be6c
#94 flip-default 5 — the flip, 2.0.0, docs 11f8817
#95 post-tag — the facade's own example 4a35e5f

Tag v2.0.0 is annotated (the annotation is the changelog) and the release is published. Wiki swept in one commit, eaafcb8.

End-state verification from merged main: unit suite 11/11, stylua clean, and four engine proofs — bare conformance.sh PASSes naming v2 (the gate is protocol-specific, so a silent v1 fallback would fail it), STDB_PROTOCOL=v1 PASSes naming v1, the sliced run does 50000 rows over 6 frames at an 8.0ms budget, and the bare HTML5 tier asserts the v2.bsatn.spacetimedb handshake. Both CI legs green on every merge commit above.

Three things the gates caught that a direct flip would have shipped:

  1. A latent bug in subscribe_url (#94). It composed the query string from opts.protocol instead of the wire connect had resolved — harmless while the default was v1 and the two always agreed. Under a v2 default a defaulted caller negotiated BSATN with no compression=None, which the codec refuses, and light = true bypassed the v2 guard onto the wire. This would have broken the release on first connect.
  2. A tautological parity assertion (#92). The first cut of the v2 stand-in for v1's light check compared a value to itself, and the separate-frame case skipped the guard entirely — so the one PR whose purpose was proving v2 parity proved nothing, and a real regression would have surfaced as a generic 90s timeout.
  3. #81's flake had a cause, not a curse (#93). Gating the HTML5 tier turned an invisible footnote into a diagnosable failure on its second run: chromium's child processes outlive a parent-only kill, so rm raced live profile writers. Group-kill fixed it, 0 failures in 8 races against 7 before.

Also worth recording: the plan's sliced-run invocation was wrong (STDB_BULK_BUDGET is seconds, not ms) and v2's decode is fast enough that the default 10000-row sweep does not slice at all. The working proof is STDB_BULK_BUDGET=0.008 STDB_BULK_SWEEP=50000.

Two nits recorded rather than actioned, both above: no max-parallel on the conformance matrix, and a dispatch no longer being able to run both wires. #81 stays open until the observation window adds a few more green runs on both wires; when it closes, the self-report step, the issues: write grant and both tees go with it.

**Done. Shipped as `v2.0.0`.** | PR | Gate | Merge SHA | | --- | --- | --- | | #91 `flip-ci` | 1 — both wires in the automatic triggers | `6847625` | | #90 `flip-confirmed` | 2 — confirmed_reads pinned to peer parity | `b71a6a0` | | #93 `fix-html5-teardown` | unplanned — #81's flake, fixed at its cause | `c46cd42` | | #92 `flip-smoke-parity` | 3 — v2 smoke at parity with v1's | `8b0be6c` | | #94 `flip-default` | 5 — the flip, 2.0.0, docs | `11f8817` | | #95 | post-tag — the facade's own example | `4a35e5f` | Tag `v2.0.0` is annotated (the annotation is the changelog) and the release is published. Wiki swept in one commit, `eaafcb8`. End-state verification from merged `main`: unit suite 11/11, stylua clean, and four engine proofs — bare `conformance.sh` PASSes naming **v2** (the gate is protocol-specific, so a silent v1 fallback would fail it), `STDB_PROTOCOL=v1` PASSes naming v1, the sliced run does 50000 rows over 6 frames at an 8.0ms budget, and the bare HTML5 tier asserts the `v2.bsatn.spacetimedb` handshake. Both CI legs green on every merge commit above. **Three things the gates caught that a direct flip would have shipped:** 1. **A latent bug in `subscribe_url`** (#94). It composed the query string from `opts.protocol` instead of the wire `connect` had resolved — harmless while the default was v1 and the two always agreed. Under a v2 default a defaulted caller negotiated BSATN with no `compression=None`, which the codec refuses, and `light = true` bypassed the v2 guard onto the wire. This would have broken the release on first connect. 2. **A tautological parity assertion** (#92). The first cut of the v2 stand-in for v1's `light` check compared a value to itself, and the separate-frame case skipped the guard entirely — so the one PR whose purpose was proving v2 parity proved nothing, and a real regression would have surfaced as a generic 90s timeout. 3. **#81's flake had a cause, not a curse** (#93). Gating the HTML5 tier turned an invisible footnote into a diagnosable failure on its second run: chromium's child processes outlive a parent-only kill, so `rm` raced live profile writers. Group-kill fixed it, 0 failures in 8 races against 7 before. Also worth recording: the plan's sliced-run invocation was wrong (`STDB_BULK_BUDGET` is seconds, not ms) and v2's decode is fast enough that the default 10000-row sweep does not slice at all. The working proof is `STDB_BULK_BUDGET=0.008 STDB_BULK_SWEEP=50000`. Two nits recorded rather than actioned, both above: no `max-parallel` on the conformance matrix, and a dispatch no longer being able to run both wires. #81 stays open until the observation window adds a few more green runs on both wires; when it closes, the self-report step, the `issues: write` grant and both tees go with it.
yuris closed this issue 2026-08-21 11:33:48 +00:00
Sign in to join this conversation.
No Branch/Tag specified
Branches Tags
main
release/v4.0.1
fix/hold-pre-open-frames
v4.0.2
v4.0.1
v4.0.0
v3.1.0
v3.0.0
v2.0.0
v1.1.0
v1.0.0
v0.1.0
Labels
Clear labels   
area:defold
Defold engine integration

  
area:docs
Documentation and examples

  
area:infra
Flake, CI, secretspec

  
area:protocol
Wire protocol conformance

  
area:testing
Test harness and fixtures

  
kind:bug
Something broken

  
kind:chore
Maintenance

  
phase:breadth
UnsubscribeMulti, procedures, BSATN

  
phase:core
Core SDK layers (protocol/cache/connection)

  
phase:hardening
Reconnect, token persistence, errors

  
phase:slice
Vertical slice: first playable end-to-end

No labels
area:defold
area:docs
area:infra
area:protocol
area:testing
kind:bug
kind:chore
phase:breadth
phase:core
phase:hardening
phase:slice
Milestone
Clear milestone
No items
No milestone
Projects
Clear projects
No items
No project
Assignees
Clear assignees
No assignees
1 participant
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference
yuris/spacetimedb-defold#89
No description provided.