archive: fix the broken HTML5 install path and ship a consumer-only game.project #53
No reviewers
Labels
No labels
area:defold
area:docs
area:infra
area:protocol
area:testing
kind:bug
kind:chore
phase:breadth
phase:core
phase:hardening
phase:slice
No milestone
No project
No assignees
1 participant
Notifications
Due date
No due date set.
Dependencies
No dependencies set.
Reference
yuris/spacetimedb-defold!53
Loading…
Add table
Add a link
Reference in a new issue
No description provided.
Delete branch "feat/archive-fixes"
Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?
Release-gate work for #24: the consumer-archive checklist item, plus the version and install-URL bumps it depends on. Ten findings from an archive audit, all documentation or configuration — no
stdb/logic touched.The one that mattered: the HTML5 install path was broken
README.md:95was the only consumer-facing instruction for the only HTML5 path, and it did not work:Defold does not name its dependency-cache entries after the dependency. The one entry in this repo's own cache is
14fb91dae55b356692919e5cdfec9d2f27beae6c-ImNjM2I4Y2FmZTY3OGY2ZjMzYmEwZjEx…i.zip— sha1 plus base64-of-URL-hash, withwebsocketappearing nowhere. Run verbatim:The repo's own harness already knew:
dev/tools/html5-conformance.sh:44-52opens every cached zip and identifies the extension by looking inside forwebsocket/ext.manifest, with a comment saying why filename matching is unreliable. SoREADME.md:91's claim that the snippet was "exactly whatdev/tools/html5-conformance.shdoes" was false precisely at the step that breaks. That claim is deleted.The install section now downloads 4.2.3 from upstream directly, sidestepping the Defold cache. Verified by extracting the instructions programmatically from the shipped README and running them as a script: exit 0, pristine
md5 833ad14e…confirmed pre-patch, andCreateConnectionEmscripten(conn)relocated intoLuaConnectat line 514 behind#if defined(__EMSCRIPTEN__). No.rej, no.orig.This also fixes the quieter half:
[library] include_dirs = stdbmeans Defold mounts onlystdb/, sopatches/shipped inside the zip but never appeared in a consumer's project tree — the oldpath/to/patches/…was unresolvable without manually locating that same hash-named zip. The patch now has a direct raw URL.The patch trailer contradicted the README
patches/extension-websocket-html5-subprotocol.patchis the only content ofpatches/and its own only documentation. Its trailer said "apply to a fork … and pointgame.projectat the fork" — the opposite of what the README says, and exactly what #15 frames as the thing to avoid. The archive was shipping the superseded instruction as the more discoverable one.Trailer rewritten to the vendoring flow. The patch body is untouched: the audit found it genuinely self-describing and verified both hunks apply to pristine 4.2.3 with zero fuzz and zero offset. Re-verified after editing —
patch -p1 --dry-runclean, hunks at 424 and 508,git apply --numstat8/3.game.project: split into a library manifest and a harness overlayIt declared
[bootstrap] main_collection = /dev/example/main/main.collectioncand[input] game_binding = /dev/example/input/game.input_bindingc— both under export-ignored/dev, so both targets were absent from the archive. Plus three sections of harness-only display and script settings.The tension: this same file is the harness's own project file, so the conformance runs need those keys. Resolved with Defold's own mechanism rather than an invented one —
bob --settings, which layers settings files left to right.dev/example/harness.projectcarries the harness keys and both harness scripts pass the flag.Verified the layering actually merges before relying on it: a settings file carrying a bogus
dependencies#0madebob resolvetry to fetch it, in all four path spellings.Tradeoff, stated plainly. A consumer who extracts the archive and opens it as a Defold project still cannot build it —
bob resolveOK,bob buildexit 1 withthe '/logic/main.collection' can't be found. That is Defold's defaultmain_collection, not a dangling reference. Judged acceptable: the honest state of this artifact is "a library with no application in it", and Defold's generic default says that, whereas the old value actively sent readers hunting for adev/example/directory deliberately stripped from their download. Making the build succeed would mean shipping a main collection to consumers purely to satisfy a project-open they have no reason to perform.harness.projectcarries no comments because bob's settings parser rejects#both before the first section and inside one — it logsCould not parseat SEVERE and then silently ignores the file, which would have dropped the settings without failing the build. All three cases verified; the rationale lives indev/tools/conformance.shnext to the flag instead.Version and URL bumps
game.projectversion0.1.0→1.0.0v1.0.0. Shape left alone — Forgejo serves/archive/<tag>.zip, and the token-as-query-param variant matches Forgejo's behaviour since basic auth 404s.grepfor0.1.0across the repo afterwards: zero matches.The
raw/tag/<tag>/<path>shape was verified against the tag that exists (raw/tag/v0.1.0/README.md→ 200);v1.0.0404s until the tag is cut, which is correct for a release-gate branch since the README ships inside the v1.0.0 archive.Smaller findings
.gitignoreshipped to consumers — noexport-ignorerule, while every other repo-only dotfile had one. It ignores/dev/test/.deps/,/dev/server/node_modules/,/result-*,/.direnv/— paths absent from the archive. Rule added; its duplicate/.internal/entry fixed..gitattributessize comment claimed "69 KB before, 23 KB after". Real numbers now, in both places it appeared (the audit only named one): 54 entries / ~220 KB before, 11 entries / 87,646 B after; payload 232,696 B,README.md76,502 B of it.dev/paths a consumer never receives. The## Devsection now leads with the fact thatdev/is stripped from the release archive, so none of those commands exist in a downloaded copy.Verified against the live forge, not a local simulation
The decisive check, because
git archivelocally is not byte-equivalent to the published artifact — the forge adds aspacetimedb-defold/prefix:.gitignoreabsent — confirmed fixed in production. Shippedgame.projectreadsversion = 1.0.0withinclude_dirs = stdband nothing else. Shipped patch trailer confirmed to be the vendoring text.Suite
6 passed, 0 failed; stylua clean. Native conformance PASS both runs through PHASE5, identity persisted across a process restart, schema fixture matched. HTML5 conformance PASS — run rather than assumed, because this PR changes that script'sbobinvocation: handshakesubprotocol=v1.json.spacetimedb→101, phases 1–5, reconnect after restart, identity survived a reload.Ten findings from a verified audit of the v1.0.0 candidate archive. Every one is documentation or configuration; no stdb/ logic changed. A4 (HIGH) was a proven break, not a style fix. README:95 was the only consumer-facing instruction for the only HTML5 path, and it did not work: unzip -q .internal/lib/*websocket*.zip -d /tmp/ws Defold names dependency-cache entries by hash, not after the dependency. Run verbatim against a real populated .internal/lib it exits 9 with "No zipfiles found" -- the substring "websocket" appears nowhere in the filename. This repo's own harness already knew: html5-conformance.sh iterates every *.zip and identifies the extension by looking inside it for websocket/ext.manifest, with a comment explaining why filename matching is unreliable. So the README's claim that the snippet was "exactly what dev/tools/html5-conformance.sh does" was false precisely at the step that broke. Replaced with a download of the 4.2.3 tag from upstream, which sidesteps the cache entirely. A5 falls out of the same fix. [library] include_dirs = stdb means Defold mounts only stdb/, so patches/ ships inside the zip but never appears in a consumer's project tree, which made the old `path/to/patches/...` unresolvable without locating that same hash-named cache zip. The patch now has a direct raw URL on the tag, so patches/ being invisible to Defold stops mattering. A6: the patch's own trailer told readers to fork extension-websocket and point game.project at the fork -- the opposite of what the README says, and the exact thing #15 exists to avoid. patches/ is that file's only documentation, so the archive was shipping the superseded instruction as the more discoverable one. Trailer prose only: both hunks are untouched and still apply to pristine 4.2.3 with zero fuzz and zero offset. A2: game.project declared [bootstrap] main_collection and [input] game_binding under /dev, which .gitattributes strips, so both targets were absent from every archive a consumer downloaded. It also shipped the harness's [script], [display] and [graphics] settings. Moved all of them to dev/example/harness.project and layered them back with bob --settings, which is Defold's own mechanism for exactly this. game.project is now only what a consumer needs: [project] title/version and [library] include_dirs. Both conformance tiers still pass. A7 and A8: version 0.1.0 -> 1.0.0, and both install URLs from /archive/v0.1.0.zip to v1.0.0. The URL shape was already correct -- Forgejo serves /archive/<tag>.zip and refuses basic auth -- only the tag was stale. A1: .gitignore had no export-ignore rule, so it landed in every consumer archive despite being unambiguously dev-only: it ignores /dev/test/.deps/, /dev/server/node_modules/, /result-* and /.direnv/, none of which exist in the archive. An omitted rule, not a deliberate exception. A3: the .gitattributes size comment claimed "69 KB before, 23 KB after", which was true when the rules were added and is now stale by 2.6x. Measured on this tree: 54 entries / 224,894 B of zip before, 11 entries / 87,583 B after. A9: the README cited dev/ paths a consumer never receives. Both the html5-conformance.sh citation and the Dev section now say so explicitly. Also dropped a redundant .gitignore block: /dist/headless/ was already covered by /dist/, and /.internal/ was listed twice. Verified: lua suite 6/6, stylua clean, native conformance PASS, html5 conformance PASS (101 Switching Protocols with the subprotocol negotiated), the README command block runs clean when extracted verbatim from the file, and the edited patch still applies to pristine 4.2.3 with no .rej and no .orig. Refs #24, #15Review — automated reviewer, posted for the record
Verdict: READY TO TAG. All five review items verified by running them; no blocking defects. Five findings, all non-blocking; B2/B3/B4 have since been actioned in
48a2cfcand a root cause the review surfaced in3379c51.Install instructions — run, they work
The shipped README from the live forge zip contains exactly one
shfenced block in 1369 lines, so extraction is unambiguous. Run in a clean empty dir with no repo access: exit 0, patch applied, no.rej/.orig, pristinemd5 833ad14e…→8eb2ab3e…, hunks at 424 and 508. A second run using the branch's own shipped patch fetched by URL produced a byte-identical result. The old exit-9 failure is gone and nothing in the flow touches Defold's cache.raw/tag/<tag>/<path>confirmed a real Forgejo route via the tag that exists (raw/tag/v0.1.0/README.md→ 200).v1.0.0404s until cut, as expected.game.project split
Live-forge copy is 175 B / 7 lines:
[project]title +version = 1.0.0+dependencies#0, and[library] include_dirs = stdb. Every/devpath and all three harness-only sections gone.Both conformance tiers PASS on a fresh clone with an isolated port — native through PHASE5 with identity persisted across a process restart, HTML5 handshake → 101 with PHASE1-5 on both loads and identity surviving a reload.
B1 — premise right, mechanism wrong. The PR body said bob "silently ignores" a settings file containing comments. Measured: bob exits 1 and prints
Could not parse: <file>twice, aborting before it resolves libraries. All four comment placements (#/;, before and inside a section) fail the same way. So the decision stands — no comment syntax parses, the rationale must live elsewhere — but the real behaviour is safer than described.Layering was proven to genuinely merge by using a bogus
dependencies#0as the observable: no--settings→ exit 0; with it →Failed to fetch library … HTTP 404.The tradeoff is the right call. A dependency's
game.projectis read by Defold only forinclude_dirs; no consumer path performs the project-open that fails. Makingbob buildsucceed would mean shipping a main collection plus a third top-level directory thatinclude_dirs = stdbwould not even expose. And the failure a consumer now sees is Defold's own generic default, where the old value actively sent readers hunting for adev/example/directory deliberately stripped from their download.Archive contents — zero accidental exclusions
Decisive check:
diffofunzip -Z1on/archive/main.zipagainst the branch's archive shows exactly one line,-spacetimedb-defold/.gitignore, and nothing in either direction besides it. 13 → 12 entries. No module, patch or README lost.The
.gitattributesnumbers reproduce to the byte: 54 entries / 225,054 B before, 11 / 87,646 B after; payload 232,696 B, README 76,502 B (32.9%),stdb/153,302 B — the parts sum to the whole.Version and URLs
grep -rn '0\.1\.0'over the whole checkout: zero matches, shipped or not.v1.0.0at exactly three places, all README. The upstream 4.2.3 URL occurs 5× and none was disturbed;release.ymlbuilds its dependency URL from\${TAG}, so nothing to bump there.Patch trailer
Body byte-unchanged —
diff -uyields a single hunk located entirely above the--- a/line. The wordforknow appears only as "no fork is needed". Still applies cleanly, proven by the install run that fetched this exact shipped file by URL.Findings
flake.nix:180.bob buildat the repo root without--settingsnow fails, so the editor advertised in the shell banner no longer bootsdev/example. Actioned in48a2cfc.README.md:105,108.curl -sLon a 404 pipesNot found.intopatch→Only garbage was found in the patch input, exit 2, cause never named. Actioned:curl -fsSL.README.md:105-106. Hardcoded/tmp/wsxmade a rerun stop at a replace prompt and exit 1 under a non-tty. Actioned:unzip -qo.Also fixed, found while actioning
vendorJsoninflake.nixused relative paths, so entering the dev shell from any subdirectory minted a phantomdev/test/.depstree there — an untrackeddev/server/dev/test/.deps/json.luawas sitting in the main checkout, and.gitignoreonly covers the root one, so anygit add -Awould have committed it. Anchored to the repo root in3379c51and verified fromdev/server.