management: config presence authorises a production store deploy; 56 dispatches of one commit observed #79

Closed
opened 2026-09-02 10:52:06 +00:00 by yuris · 0 comments
Owner

Observed while checking whether the new checks workflow fired on a pull request. Filed with the attribution unresolved — the evidence is solid, the cause is not, and guessing would be worse than saying so.

What was measured

/actions/tasks shows 57 deploy runs for the single commit d0b58259 — 56 triggered by workflow_dispatch, 1 by the push that merged it. Of those, 36 cancelled and 21 succeeded.

They arrive in bursts:

1045..1052   2026-09-01T06:21:53Z .. 06:34:20Z    (8 runs)
1053..1056   2026-09-01T09:59:25Z .. 10:09:02Z    (4 runs)

Each success is a real wrangler pages deploy apps/store/out — a production release of the storefront. So the live store was rebuilt and redeployed at least 21 times for one unchanged commit.

The cancel-in-progress concurrency group is working as designed and is why most were cancelled; it collapses a burst rather than preventing one. The concurrency group is the mitigation, not the fix.

Why this is a real defect and not just noise

lib/store/deploy.ts#requestStoreDeploy is called by every inventory write path, by design: the store is a static export whose catalog is read at build time, so a published change is invisible until a rebuild. That reasoning is sound and is documented in the file.

What is missing is any notion of which environment is allowed to dispatch:

  • The only guard is if (!url || !token) return;. Configuration presence is the authorisation.
  • So any process holding STORE_DEPLOY_WORKFLOW_URL and STORE_DEPLOY_TOKEN — a developer's shell, a dev server started from a worktree that inherited the environment, a test that forgot to point at a sink — dispatches a production deploy on ref: "main", from a write to a non-production database.
  • The dispatch is fire-and-forget and failures only console.error, so nothing surfaces to the person who caused it.

The content risk is low: the deploy builds from the production catalog API, so a dev-database row cannot reach the live site through this path. The risk is that a non-production action causes a production release, invisibly, and that nothing in the code distinguishes the two.

Attribution, stated honestly

Not established. At the time of filing, no apps/management/.env in the default checkout or in any of the four live worktrees contains STORE_DEPLOY_WORKFLOW_URL, so the obvious explanation does not hold. Candidates not ruled out:

  • the production management container servicing real operator writes — in which case 21 releases of one commit is legitimate behaviour and the burst pattern is the interesting part, not the count
  • a dev server or MCP process started earlier with the variable set, since a worktree .env is gitignored and several were created and destroyed during recent work
  • one of the write-matrix verifications run against a real server rather than a sink

Do not treat the count as proven waste until this is resolved. It is proven exposure either way.

Scope

  • Require an explicit opt-in for dispatching, not merely the presence of config. Something a developer cannot inherit by accident — an environment marker checked alongside the URL, refusing to dispatch unless the process believes it is production, and logging loudly when it declines. The current silent no-op is right for "unset"; it is wrong for "set but this is not production".
  • Do not dispatch for a write that changed nothing. Confirm whether a rejected or no-op write reaches requestStoreDeploy. PR #68 and PR #78 both added rejections; if a rejection still queues a build, that is a burst source with no cause.
  • Make a declined dispatch visible. console.error on failure and silence on skip means nobody learns they nearly released. At minimum log the reason and the environment.
  • Decide whether ref: "main" is right. A dispatch from a non-main context still deploys main, which is the surprising part of the current shape.

Not this issue

  • The cancel-in-progress concurrency group. It is correct and should stay.
  • Whether stock changes should rebuild the store at all — that trade is recorded in deploy.ts and was deliberate when price and availability became public.
  • #73 — the missing test trigger. Related only in that both are workflow-level gaps.

Verification

  • Set the URL and token in a non-production environment, perform an inventory write, and assert no dispatch leaves the process — then flip the opt-in marker and assert one does.
  • Perform a rejected write (a 10-digit buy_price, per PR #68's matrix) and assert no dispatch is queued.
  • Perform a no-op write (a PATCH that changes nothing) and assert no dispatch is queued.
  • Point the URL at a local sink and count dispatches per write, to establish the real ratio — one write should be one dispatch, and the burst pattern above suggests it may not be.
  • Re-read /actions/tasks afterwards and confirm the dispatch rate for an unchanged commit falls to zero.

Acceptance

A non-production process cannot dispatch a production deploy even with the URL and token present; a rejected or no-op write queues no build; a declined dispatch is logged with its reason; and the dispatch-per-write ratio is measured rather than assumed.

Observed while checking whether the new checks workflow fired on a pull request. Filed with the attribution **unresolved** — the evidence is solid, the cause is not, and guessing would be worse than saying so. ## What was measured `/actions/tasks` shows **57 `deploy` runs for the single commit `d0b58259`** — 56 triggered by `workflow_dispatch`, 1 by the `push` that merged it. Of those, **36 cancelled and 21 succeeded**. They arrive in bursts: ``` 1045..1052 2026-09-01T06:21:53Z .. 06:34:20Z (8 runs) 1053..1056 2026-09-01T09:59:25Z .. 10:09:02Z (4 runs) ``` Each success is a real `wrangler pages deploy apps/store/out` — a production release of the storefront. So the live store was rebuilt and redeployed at least 21 times for one unchanged commit. The `cancel-in-progress` concurrency group is working as designed and is why most were cancelled; it collapses a burst rather than preventing one. **The concurrency group is the mitigation, not the fix.** ## Why this is a real defect and not just noise `lib/store/deploy.ts#requestStoreDeploy` is called by **every inventory write path**, by design: the store is a static export whose catalog is read at build time, so a published change is invisible until a rebuild. That reasoning is sound and is documented in the file. What is missing is any notion of **which environment is allowed to dispatch**: - The only guard is `if (!url || !token) return;`. Configuration presence *is* the authorisation. - So any process holding `STORE_DEPLOY_WORKFLOW_URL` and `STORE_DEPLOY_TOKEN` — a developer's shell, a dev server started from a worktree that inherited the environment, a test that forgot to point at a sink — dispatches a **production** deploy on `ref: "main"`, from a write to a **non-production** database. - The dispatch is fire-and-forget and failures only `console.error`, so nothing surfaces to the person who caused it. The content risk is low: the deploy builds from the production catalog API, so a dev-database row cannot reach the live site through this path. **The risk is that a non-production action causes a production release**, invisibly, and that nothing in the code distinguishes the two. ## Attribution, stated honestly Not established. At the time of filing, no `apps/management/.env` in the default checkout or in any of the four live worktrees contains `STORE_DEPLOY_WORKFLOW_URL`, so the obvious explanation does not hold. Candidates not ruled out: - the production management container servicing real operator writes — in which case 21 releases of one commit is legitimate behaviour and the burst pattern is the interesting part, not the count - a dev server or MCP process started earlier with the variable set, since a worktree `.env` is gitignored and several were created and destroyed during recent work - one of the write-matrix verifications run against a real server rather than a sink **Do not treat the count as proven waste until this is resolved.** It is proven exposure either way. ## Scope - **Require an explicit opt-in for dispatching**, not merely the presence of config. Something a developer cannot inherit by accident — an environment marker checked alongside the URL, refusing to dispatch unless the process believes it is production, and logging loudly when it declines. The current silent no-op is right for "unset"; it is wrong for "set but this is not production". - **Do not dispatch for a write that changed nothing.** Confirm whether a rejected or no-op write reaches `requestStoreDeploy`. PR #68 and PR #78 both added rejections; if a rejection still queues a build, that is a burst source with no cause. - **Make a declined dispatch visible.** `console.error` on failure and silence on skip means nobody learns they nearly released. At minimum log the reason and the environment. - **Decide whether `ref: "main"` is right.** A dispatch from a non-main context still deploys `main`, which is the surprising part of the current shape. ## Not this issue - The `cancel-in-progress` concurrency group. It is correct and should stay. - Whether stock changes should rebuild the store at all — that trade is recorded in `deploy.ts` and was deliberate when price and availability became public. - **#73** — the missing test trigger. Related only in that both are workflow-level gaps. ## Verification - Set the URL and token in a non-production environment, perform an inventory write, and assert **no dispatch leaves the process** — then flip the opt-in marker and assert one does. - Perform a **rejected** write (a 10-digit `buy_price`, per PR #68's matrix) and assert no dispatch is queued. - Perform a no-op write (a `PATCH` that changes nothing) and assert no dispatch is queued. - Point the URL at a local sink and count dispatches per write, to establish the real ratio — one write should be one dispatch, and the burst pattern above suggests it may not be. - Re-read `/actions/tasks` afterwards and confirm the dispatch rate for an unchanged commit falls to zero. ## Acceptance A non-production process cannot dispatch a production deploy even with the URL and token present; a rejected or no-op write queues no build; a declined dispatch is logged with its reason; and the dispatch-per-write ratio is measured rather than assumed.
yuris closed this issue 2026-09-28 20:53:12 +00:00
Sign in to join this conversation.
No labels
No milestone
No project
No assignees
1 participant
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference
yuris/garage#79
No description provided.