Special types: Timestamp/TimeDuration/ScheduleAt units are a silent footgun #23

New issue

Closed

opened 2026-08-19 15:19:39 +00:00 by yuris · 0 comments

yuris commented

2026-08-19 15:19:39 +00:00

Owner

A module column of type Timestamp, TimeDuration, ConnectionId, ScheduleAt or Uuid arrives in the cache as whatever JSON the server wrote. The SDK decodes rows positionally and hands the raw value through, so consumers get a bare number or string and have to know the encoding to do anything with it.

Identity is the exception and shows the pattern already exists: handlers.IdentityToken runs it through protocol.unwrap (stdb/connection.lua:222-223), so db.identity is usable. Nothing else gets that treatment.

The one that actually misleads people

Timestamp is microseconds since the Unix epoch, not seconds and not milliseconds (SpacetimeDB v2.8.2, sdks/typescript/src/lib/timestamp.ts:18-29). Every other timestamp a Defold developer touches is seconds (socket.gettime, os.time) or milliseconds if they came from JavaScript. So the natural thing to write is:

os.date("%H:%M", row.created_at)     -- silently wrong by a factor of 1e6

That does not error. It produces a date roughly 50,000 years out, or a number Lua formats without complaint. Worse, in Lua 5.1 a microsecond epoch value exceeds the exact-integer range of a 32-bit int build and starts losing precision in arithmetic, which fails differently on different targets.

TimeDuration has the same unit and the same trap. ScheduleAt is a sum type — { Interval = <TimeDuration> } or { Time = <Timestamp> } — so it needs tag inspection before the unit conversion even applies.

Proposed change

A small set of pure conversion helpers, in stdb/protocol.lua beside unwrap and decode_payload where the other wire-encoding knowledge lives:

Helper	Returns
`protocol.timestamp_to_seconds(v)`	float seconds, suitable for `os.date` / `os.time`
`protocol.duration_to_seconds(v)`	float seconds
`protocol.schedule_at(v)`	`"interval"` or `"time"`, plus the converted value
`protocol.connection_id(v)`	hex string, matching `unwrap`'s treatment of identity

Deliberately not proposing wrapper objects or automatic per-column coercion. Automatic coercion would need the generated schema to carry column types, which it currently does not — it carries names, order and pk (dev/tools/gen-schema.sh). Adding types to codegen to enable transparent conversion is a much larger change and a separate decision; explicit helpers cost four small pure functions and are trivially testable.

Acceptance

Unit tests in dev/test/test_protocol.lua with a real captured value, asserting the seconds conversion round-trips against a known date, plus both ScheduleAt variants. Precision needs one explicit test: a current-era microsecond timestamp must survive the conversion without losing seconds resolution on a 32-bit Lua build.

README gains a short table of the special types and their units, in the API reference next to the cache section. The unit is the whole point — a helper nobody knows about does not prevent the os.date bug.

Priority

Medium, rising to high the moment any module in use has a timestamp column. It is a silent-wrong-answer bug rather than a crash, which is the category this SDK has been careful about elsewhere (#8's dropped header, #17's silent new identity).

A module column of type `Timestamp`, `TimeDuration`, `ConnectionId`, `ScheduleAt` or `Uuid` arrives in the cache as whatever JSON the server wrote. The SDK decodes rows positionally and hands the raw value through, so consumers get a bare number or string and have to know the encoding to do anything with it. Identity is the exception and shows the pattern already exists: `handlers.IdentityToken` runs it through `protocol.unwrap` (`stdb/connection.lua:222-223`), so `db.identity` is usable. Nothing else gets that treatment. ## The one that actually misleads people **`Timestamp` is microseconds since the Unix epoch**, not seconds and not milliseconds (SpacetimeDB v2.8.2, `sdks/typescript/src/lib/timestamp.ts:18-29`). Every other timestamp a Defold developer touches is seconds (`socket.gettime`, `os.time`) or milliseconds if they came from JavaScript. So the natural thing to write is: ```lua os.date("%H:%M", row.created_at) -- silently wrong by a factor of 1e6 ``` That does not error. It produces a date roughly 50,000 years out, or a number Lua formats without complaint. Worse, in Lua 5.1 a microsecond epoch value exceeds the exact-integer range of a 32-bit `int` build and starts losing precision in arithmetic, which fails differently on different targets. `TimeDuration` has the same unit and the same trap. `ScheduleAt` is a sum type — `{ Interval = <TimeDuration> }` or `{ Time = <Timestamp> }` — so it needs tag inspection before the unit conversion even applies. ## Proposed change A small set of pure conversion helpers, in `stdb/protocol.lua` beside `unwrap` and `decode_payload` where the other wire-encoding knowledge lives: | Helper | Returns | |---|---| | `protocol.timestamp_to_seconds(v)` | float seconds, suitable for `os.date` / `os.time` | | `protocol.duration_to_seconds(v)` | float seconds | | `protocol.schedule_at(v)` | `"interval"` or `"time"`, plus the converted value | | `protocol.connection_id(v)` | hex string, matching `unwrap`'s treatment of identity | Deliberately **not** proposing wrapper objects or automatic per-column coercion. Automatic coercion would need the generated schema to carry column *types*, which it currently does not — it carries names, order and pk (`dev/tools/gen-schema.sh`). Adding types to codegen to enable transparent conversion is a much larger change and a separate decision; explicit helpers cost four small pure functions and are trivially testable. ## Acceptance Unit tests in `dev/test/test_protocol.lua` with a real captured value, asserting the seconds conversion round-trips against a known date, plus both `ScheduleAt` variants. Precision needs one explicit test: a current-era microsecond timestamp must survive the conversion without losing seconds resolution on a 32-bit Lua build. README gains a short table of the special types and their units, in the API reference next to the cache section. The unit is the whole point — a helper nobody knows about does not prevent the `os.date` bug. ## Priority Medium, rising to high the moment any module in use has a timestamp column. It is a silent-wrong-answer bug rather than a crash, which is the category this SDK has been careful about elsewhere (#8's dropped header, #17's silent new identity).

yuris referenced this issue

2026-08-19 15:20:43 +00:00

v1.0.0 release checklist #24

yuris referenced this issue

2026-08-19 15:42:13 +00:00

Codegen: emit column types, indexes and reducer signatures (foundation for 1.0) #25

yuris referenced this issue

2026-08-19 15:42:13 +00:00

Decode every AlgebraicType, with a fixture module that exercises all of them #26

yuris referenced this issue

2026-08-19 16:11:51 +00:00

Identity-keyed tables are broken in the cache (pk decodes to a table) #35