yuris/spacetimedb-defold
1
0
Fork 0
Code Issues 1 Pull requests Projects Releases 9 Packages Wiki Activity Actions

HTML5: upstream the extension-websocket subprotocol fix #15

New issue
Closed
opened 2026-08-19 12:06:45 +00:00 by yuris · 3 comments
yuris commented 2026-08-19 12:06:45 +00:00
Owner
Copy link

HTML5 is blocked on a bug in extension-websocket 4.2.3, diagnosed and patched in #8 but not yet fixed at the source. Until it is, every consumer who wants a browser build has to carry a fork, which is a bad thing to ask of a library's users.

Upstream: https://github.com/defold/extension-websocket/issues/67 (filed 2026-08-21).

The report deliberately gives only the symptom and the two handshakes, not the cause, so a maintainer has room to reach the ordering independently. The analysis below, the patch, and the patched-build handshake are held back for that thread. The ordering is still present on master, so this is not a fixed-but-unreleased case.

The bug

websocket/src/websocket.cpp opens the browser socket before recording the subprotocol:

  • line 427 — CreateConnection() calls CreateConnectionEmscripten(conn)
  • line 510 — LuaConnect() then assigns conn->m_Protocol = strdup(protocol)

conn is memset to zero, so emscripten_websocket_new reads a null protocols pointer and the glue takes its no-subprotocol branch. Sec-WebSocket-Protocol never leaves the browser. Native builds are unaffected — wslay writes the handshake later, from the update loop.

Any server that requires a subprotocol is unreachable from HTML5. SpacetimeDB answers 400 no valid protocol selected.

The fix is six lines, moving socket creation to the end of LuaConnect: patches/extension-websocket-html5-subprotocol.patch. Verified against a live 2.8.2 server with a wasm-web bundle in headless Chromium — handshake goes from subprotocol=<ABSENT> / 400 to subprotocol=v1.json.spacetimedb / 101, and the full smoke client passes.

Custom headers are not part of this

Re-tested 2026-08-21 in a scratch project against a raw TCP listener, so the handshake is read off the wire rather than inferred. Native sends both Sec-WebSocket-Protocol and the custom header. Unpatched HTML5 sends neither. Patched HTML5 sends the subprotocol but still no custom header, because emscripten's WebSocket API cannot set arbitrary request headers at all.

So the two failures have different causes and only one is upstream's to fix. The ?token= workaround stays permanently, and the upstream report covers the subprotocol only.

Work

  • Open the upstream issue — defold/extension-websocket#67
  • Open the PR with the patch and the handshake evidence, linked to #67 — overtaken by events: upstream fixed it themselves from the report alone, so no PR from us was needed.
  • Publish an interim fork (tagged, so consumers can pin it) and document the dependency URL in the README's Install section — never done, and no longer needed. See "Worth deciding" below: the harness vendored and patched the extension itself, so CI never needed a fork.
  • Point dev/tools/html5-conformance.sh at the fork by default — moot. It now builds against the plain 4.2.4 dependency, unpatched, and passes.
  • Drop the fork and the patch once upstream ships a release containing the fix — 4.2.4, released 2026-08-22. Patch deleted in #112.

Worth deciding

Whether the interim fork lives under this account or the README simply tells people to fork it themselves. Hosting it is friendlier but becomes a thing to maintain and re-tag against upstream releases.

Worth reconsidering now that dev/tools/html5-conformance.sh vendors and patches the extension by itself: no fork is strictly needed for CI, only for downstream consumers of this SDK.

HTML5 is blocked on a bug in extension-websocket 4.2.3, diagnosed and patched in #8 but not yet fixed at the source. Until it is, every consumer who wants a browser build has to carry a fork, which is a bad thing to ask of a library's users. **Upstream: https://github.com/defold/extension-websocket/issues/67** (filed 2026-08-21). The report deliberately gives only the symptom and the two handshakes, not the cause, so a maintainer has room to reach the ordering independently. The analysis below, the patch, and the patched-build handshake are held back for that thread. The ordering is still present on `master`, so this is not a fixed-but-unreleased case. ## The bug `websocket/src/websocket.cpp` opens the browser socket before recording the subprotocol: - line 427 — `CreateConnection()` calls `CreateConnectionEmscripten(conn)` - line 510 — `LuaConnect()` then assigns `conn->m_Protocol = strdup(protocol)` `conn` is `memset` to zero, so `emscripten_websocket_new` reads a null `protocols` pointer and the glue takes its no-subprotocol branch. `Sec-WebSocket-Protocol` never leaves the browser. Native builds are unaffected — wslay writes the handshake later, from the update loop. Any server that requires a subprotocol is unreachable from HTML5. SpacetimeDB answers `400 no valid protocol selected`. The fix is six lines, moving socket creation to the end of `LuaConnect`: `patches/extension-websocket-html5-subprotocol.patch`. Verified against a live 2.8.2 server with a `wasm-web` bundle in headless Chromium — handshake goes from `subprotocol=<ABSENT>` / 400 to `subprotocol=v1.json.spacetimedb` / 101, and the full smoke client passes. ## Custom headers are not part of this Re-tested 2026-08-21 in a scratch project against a raw TCP listener, so the handshake is read off the wire rather than inferred. Native sends both `Sec-WebSocket-Protocol` and the custom header. Unpatched HTML5 sends neither. Patched HTML5 sends the subprotocol but still no custom header, because emscripten's WebSocket API cannot set arbitrary request headers at all. So the two failures have different causes and only one is upstream's to fix. The `?token=` workaround stays permanently, and the upstream report covers the subprotocol only. ## Work - [x] Open the upstream issue — defold/extension-websocket#67 - ~~Open the PR with the patch and the handshake evidence, linked to #67~~ — overtaken by events: upstream fixed it themselves from the report alone, so no PR from us was needed. - ~~Publish an interim fork (tagged, so consumers can pin it) and document the dependency URL in the README's Install section~~ — never done, and no longer needed. See "Worth deciding" below: the harness vendored and patched the extension itself, so CI never needed a fork. - ~~Point `dev/tools/html5-conformance.sh` at the fork by default~~ — moot. It now builds against the plain 4.2.4 dependency, unpatched, and passes. - [x] Drop the fork and the patch once upstream ships a release containing the fix — 4.2.4, released 2026-08-22. Patch deleted in #112. ## Worth deciding Whether the interim fork lives under this account or the README simply tells people to fork it themselves. Hosting it is friendlier but becomes a thing to maintain and re-tag against upstream releases. Worth reconsidering now that `dev/tools/html5-conformance.sh` vendors and patches the extension by itself: no fork is strictly needed for CI, only for downstream consumers of this SDK.
yuris commented 2026-08-19 14:27:11 +00:00
Author
Owner
Copy link

Scoped down to one item. Two of the four checkboxes were solved by a better mechanism, and the open question answers itself.

The bug is still there upstream, and nobody has reported it

Checked defold/extension-websocket master (websocket.cpp, last touched 2026-06-01, c40a1d4): identical to 4.2.3, at the same line numbers.

427:#if defined(__EMSCRIPTEN__)
428:    CreateConnectionEmscripten(conn);
429:#endif
...
510:    conn->m_Protocol = protocol ? strdup(protocol) : 0;

4.2.3 is still the latest release, and its only change was an unrelated use-after-free fix (upstream #66). A search of every issue and PR in that repo for "subprotocol" returns 0 results — so this is unreported, not merely unfixed. The diagnosis in the description holds unchanged.

Two checkboxes are already obsolete

  • Point the harness at the fork by default, so HTML5 conformance runs without manual setup — the harness does not need a fork. dev/tools/html5-conformance.sh:44-61 unzips whatever extension archive bob resolved, applies patches/extension-websocket-html5-subprotocol.patch to it, and builds a scratch project carrying it as a local extension with the remote dependency stripped. It already runs with zero manual setup, and it re-applies the patch on every run, so it fails loudly the moment upstream moves the code.
  • Publish an interim fork, tagged so consumers can pin it — see below.

"Worth deciding": neither. No fork.

The framing assumed a consumer's only options are pin a hosted fork or fork it themselves. There is a third, which is what the harness proves out on every run: vendor the extension into your own project and patch it there. Defold cannot patch a dependency in place, but a local extension directory is not a dependency.

README now documents that directly instead of telling people to fork:

unzip -q .internal/lib/*websocket*.zip -d /tmp/ws
mv /tmp/ws/*/websocket .            # now a local extension dir
patch -p1 <path/to/patches/extension-websocket-html5-subprotocol.patch

…plus the trap that matters: drop the remote extension-websocket URL from game.project afterwards, or the build carries two websocket extensions.

Hosting a fork would mean an account-level repo to re-tag against every upstream release, a second URL in the Install section, and a consumer's build silently pinned to my mirror rather than Defold's. Four lines of unzip-and-patch beats all of that, and it is the same four lines CI already runs.

What is actually left

  • Open the PR against defold/extension-websocket with the patch and the handshake evidence

That is the whole issue now. It needs a GitHub account pushing a branch, so it is a human action — the patch, the commit message, and the before/after handshake evidence are all in patches/extension-websocket-html5-subprotocol.patch, ready to paste.

  • Drop patches/ and the harness's vendoring step once upstream ships a release containing the fix

Not blocking #16

#16 listed this issue as a blocker. It is not, and never was — the harness patches the extension itself, so HTML5 conformance runs green today against an unpatched upstream. The HTML5 CI step has landed; details there.

Scoped down to one item. Two of the four checkboxes were solved by a better mechanism, and the open question answers itself. ## The bug is still there upstream, and nobody has reported it Checked `defold/extension-websocket` master (`websocket.cpp`, last touched 2026-06-01, `c40a1d4`): identical to 4.2.3, at the same line numbers. ``` 427:#if defined(__EMSCRIPTEN__) 428: CreateConnectionEmscripten(conn); 429:#endif ... 510: conn->m_Protocol = protocol ? strdup(protocol) : 0; ``` 4.2.3 is still the latest release, and its only change was an unrelated use-after-free fix (upstream #66). A search of every issue and PR in that repo for "subprotocol" returns **0 results** — so this is unreported, not merely unfixed. The diagnosis in the description holds unchanged. ## Two checkboxes are already obsolete - ~~Point the harness at the fork by default, so HTML5 conformance runs without manual setup~~ — the harness does not need a fork. `dev/tools/html5-conformance.sh:44-61` unzips whatever extension archive bob resolved, applies `patches/extension-websocket-html5-subprotocol.patch` to it, and builds a scratch project carrying it as a local extension with the remote dependency stripped. It already runs with zero manual setup, and it re-applies the patch on every run, so it fails loudly the moment upstream moves the code. - ~~Publish an interim fork, tagged so consumers can pin it~~ — see below. ## "Worth deciding": neither. No fork. The framing assumed a consumer's only options are *pin a hosted fork* or *fork it themselves*. There is a third, which is what the harness proves out on every run: **vendor the extension into your own project and patch it there.** Defold cannot patch a dependency in place, but a local extension directory is not a dependency. README now documents that directly instead of telling people to fork: ```sh unzip -q .internal/lib/*websocket*.zip -d /tmp/ws mv /tmp/ws/*/websocket . # now a local extension dir patch -p1 <path/to/patches/extension-websocket-html5-subprotocol.patch ``` …plus the trap that matters: drop the remote `extension-websocket` URL from `game.project` afterwards, or the build carries two `websocket` extensions. Hosting a fork would mean an account-level repo to re-tag against every upstream release, a second URL in the Install section, and a consumer's build silently pinned to my mirror rather than Defold's. Four lines of unzip-and-patch beats all of that, and it is the same four lines CI already runs. ## What is actually left - [x] Open the PR against `defold/extension-websocket` with the patch and the handshake evidence That is the whole issue now. It needs a GitHub account pushing a branch, so it is a human action — the patch, the commit message, and the before/after handshake evidence are all in `patches/extension-websocket-html5-subprotocol.patch`, ready to paste. - [x] Drop `patches/` and the harness's vendoring step once upstream ships a release containing the fix ## Not blocking #16 #16 listed this issue as a blocker. It is not, and never was — the harness patches the extension itself, so HTML5 conformance runs green today against an unpatched upstream. The HTML5 CI step has landed; details there.
yuris commented 2026-08-22 03:10:13 +00:00
Author
Owner
Copy link

Upstream has a pull request: defold/extension-websocket#68, opened by a Defold maintainer against #67.

It reaches the ordering independently of anything we said — the report deliberately carried the two handshakes and not the cause. The fix separates connection allocation from platform-specific startup, so the Emscripten WebSocket constructor and the native worker never observe a half-initialised connection, and starts the connection only once its protocol, headers, timeout, callback and registration are set. Verified upstream on wasm-web, wasm_pthread-web and both macOS targets, with Sec-WebSocket-Protocol confirmed present in Chromium and native.

Nothing here is waiting on us. This issue closes when that merges and a release carries it, at which point patches/extension-websocket-html5-subprotocol.patch and the harness step that applies it can both go, along with the "remove the remote dependency" instruction in the install docs.

Worth restating what it does not fix, since it would be easy to expect: custom headers. Emscripten's WebSocket API cannot set arbitrary request headers at all, so a patched build sends the subprotocol and still no Authorization. Passing the token as ?token= stays permanent on HTML5 rather than being a stopgap.

Upstream has a pull request: [defold/extension-websocket#68](https://github.com/defold/extension-websocket/pull/68), opened by a Defold maintainer against [#67](https://github.com/defold/extension-websocket/issues/67). It reaches the ordering independently of anything we said — the report deliberately carried the two handshakes and not the cause. The fix separates connection allocation from platform-specific startup, so the Emscripten WebSocket constructor and the native worker never observe a half-initialised connection, and starts the connection only once its protocol, headers, timeout, callback and registration are set. Verified upstream on `wasm-web`, `wasm_pthread-web` and both macOS targets, with `Sec-WebSocket-Protocol` confirmed present in Chromium and native. Nothing here is waiting on us. This issue closes when that merges and a release carries it, at which point `patches/extension-websocket-html5-subprotocol.patch` and the harness step that applies it can both go, along with the "remove the remote dependency" instruction in the install docs. Worth restating what it does **not** fix, since it would be easy to expect: custom headers. Emscripten's WebSocket API cannot set arbitrary request headers at all, so a patched build sends the subprotocol and still no `Authorization`. Passing the token as `?token=` stays permanent on HTML5 rather than being a stopgap.
yuris commented 2026-08-22 10:22:24 +00:00
Author
Owner
Copy link

Resolved upstream. extension-websocket 4.2.4 shipped on 2026-08-22 with the
fix, so this SDK carries no patch and no fork.

The release notes name the issue directly: "The protocol option is now honored
on HTML5, ensuring the Sec-WebSocket-Protocol header is included in the
WebSocket handshake (#67)."

How the work items landed

  • Upstream issue opened, defold/extension-websocket#67. Fixed and released.
  • No PR from us was needed. Upstream took the report and fixed it themselves, so
    the analysis, the patch and the patched-build handshake stayed held back as the
    original plan intended.
  • No interim fork was ever published, which settles the open question at the
    bottom of this issue. Hosting one would have bought a few weeks of convenience
    and left a thing to re-tag against every upstream release. The harness vendored
    and patched the extension itself in the meantime, so CI never needed a fork,
    and consumers who could not wait had the patch in-tree.
  • dev/tools/html5-conformance.sh no longer vendors or patches anything. It
    builds against the plain dependency.
  • The patch is deleted.

Verified against a real browser, not assumed

The HTML5 tier passes unpatched, and the handshake assertion in
dev/tools/html5-conformance.mjs is what proves the upstream fix rather than a
workaround:

[handshake] -> subprotocol=v2.bsatn.spacetimedb
[handshake] <- 101 Switching Protocols

That is the exact transition this issue was opened about, from
subprotocol=<ABSENT> and a 400 to a named subprotocol and a 101.

What did not change

Custom headers are still unreachable on HTML5. Emscripten's WebSocket API cannot
set arbitrary request headers at all, so that was never upstream's to fix and the
?token= query parameter stays permanently. stdb.connect picks it
automatically on that target, so a consumer does nothing.

What a consumer does now

Two plain dependency URLs, on every target including HTML5. The three manual
steps a browser build used to require, unpacking the fetched extension, applying
the patch, and removing the remote dependency, are gone. Documented in the
README's Install section and in the wiki's Installation page.

Closed by #112.

Resolved upstream. extension-websocket **4.2.4** shipped on 2026-08-22 with the fix, so this SDK carries no patch and no fork. The release notes name the issue directly: "The `protocol` option is now honored on HTML5, ensuring the `Sec-WebSocket-Protocol` header is included in the WebSocket handshake ([#67](https://github.com/defold/extension-websocket/issues/67))." ## How the work items landed - Upstream issue opened, defold/extension-websocket#67. Fixed and released. - No PR from us was needed. Upstream took the report and fixed it themselves, so the analysis, the patch and the patched-build handshake stayed held back as the original plan intended. - **No interim fork was ever published**, which settles the open question at the bottom of this issue. Hosting one would have bought a few weeks of convenience and left a thing to re-tag against every upstream release. The harness vendored and patched the extension itself in the meantime, so CI never needed a fork, and consumers who could not wait had the patch in-tree. - `dev/tools/html5-conformance.sh` no longer vendors or patches anything. It builds against the plain dependency. - The patch is deleted. ## Verified against a real browser, not assumed The HTML5 tier passes unpatched, and the handshake assertion in `dev/tools/html5-conformance.mjs` is what proves the upstream fix rather than a workaround: ``` [handshake] -> subprotocol=v2.bsatn.spacetimedb [handshake] <- 101 Switching Protocols ``` That is the exact transition this issue was opened about, from `subprotocol=<ABSENT>` and a 400 to a named subprotocol and a 101. ## What did not change Custom headers are still unreachable on HTML5. Emscripten's WebSocket API cannot set arbitrary request headers at all, so that was never upstream's to fix and the `?token=` query parameter stays permanently. `stdb.connect` picks it automatically on that target, so a consumer does nothing. ## What a consumer does now Two plain dependency URLs, on every target including HTML5. The three manual steps a browser build used to require, unpacking the fetched extension, applying the patch, and removing the remote dependency, are gone. Documented in the README's Install section and in the wiki's Installation page. Closed by #112.
yuris closed this issue 2026-08-22 10:22:24 +00:00
Sign in to join this conversation.
No Branch/Tag specified
Branches Tags
main
release/v4.0.1
fix/hold-pre-open-frames
v4.0.2
v4.0.1
v4.0.0
v3.1.0
v3.0.0
v2.0.0
v1.1.0
v1.0.0
v0.1.0
Labels
Clear labels   
area:defold
Defold engine integration

  
area:docs
Documentation and examples

  
area:infra
Flake, CI, secretspec

  
area:protocol
Wire protocol conformance

  
area:testing
Test harness and fixtures

  
kind:bug
Something broken

  
kind:chore
Maintenance

  
phase:breadth
UnsubscribeMulti, procedures, BSATN

  
phase:core
Core SDK layers (protocol/cache/connection)

  
phase:hardening
Reconnect, token persistence, errors

  
phase:slice
Vertical slice: first playable end-to-end

No labels
area:defold
area:docs
area:infra
area:protocol
area:testing
kind:bug
kind:chore
phase:breadth
phase:core
phase:hardening
phase:slice
Milestone
Clear milestone
No items
No milestone
Projects
Clear projects
No items
No project
Assignees
Clear assignees
No assignees
1 participant
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference
yuris/spacetimedb-defold#15
No description provided.