yuris/life-service
1
0
Fork 0
Code Issues Pull requests Projects Releases 6 Packages 1 Wiki Activity Actions

Scoped Bearer Token Auth & PII Diagnostic Redaction #35

New issue
Closed
opened 2026-09-22 17:58:59 +00:00 by yuris · 0 comments
yuris commented 2026-09-22 17:58:59 +00:00
Owner
Copy link

Summary

Implement fine-grained scoped bearer token authentication (read, write, admin) with constant-time token comparison, stable actor identity mapping across token rotations, and comprehensive PII/credential redaction across all structured logs, tracing spans, and error diagnostic output.

Background & Technical Rationale

Principle of Least Privilege: Token Scopes

The life service exposes HTTPS endpoints consumed by automated agents, background workers, and human operators. In accordance with the principle of least privilege:

  • read Scope: Allows querying agendas, listing tasks, fetching items, resolving references, searching context, and viewing timeline history. It permits read-only analytical POST requests (e.g. /v1/reviews, /v1/next), but strictly rejects all state mutations.
  • write Scope: Allows adding/updating/completing tasks and events, capturing outcomes, adding context, and generating or applying reviewed proposals. It never grants administrative or maintenance operations.
  • admin Scope: Required for system-level operations such as backend sync rebaselining (POST /v1/sync) or diagnostic inspections requiring privileged access.

An agent operating in an advisory or read-only capacity must be provisioned with a read token. A write token attempting an administrative action must be rejected with HTTP 403 Forbidden.

Constant-Time Verification & Secret Protection

To prevent timing attacks that could reveal token characters based on comparison latency:

  1. Bearer tokens must be high-entropy random secrets (minimum 256 bits).
  2. The service never stores plaintext tokens. Tokens are configured via external secret files containing SHA-256 verification digests or hashed tokens.
  3. Incoming bearer tokens are hashed with SHA-256, and compared against configured valid digests using fixed-length, constant-time comparison (subtle::ConstantTimeEq).
  4. Malformed tokens (e.g. non-ASCII or incorrect lengths) are rejected before database queries without logging their contents.

Token Rotation & Stable Actor Identity (Scenario D04)

Bearer tokens must be rotatable without disrupting in-flight operations or breaking idempotency:

  • The service configuration maps secret token files to a canonical actor identity (e.g., actor_id: "usr_alice").
  • During rotation, a new token is provisioned with the same actor identity before the old token is revoked.
  • When an operation is replayed or queried (e.g. life operation get <OP_ID>), the request succeeds under the new token because idempotency and operation ownership are bound to the stable actor identity, not the transient bearer token credential.
  • Once revoked, the old token is rejected immediately.

Diagnostic Redaction & PII Safeguards (Scenario D03)

Diagnostic logs and tracing systems are a frequent source of accidental data exfiltration. The service and CLI must enforce strict diagnostic redaction:

  • Never Logged:
    • Authorization header values, bearer tokens, or password strings.
    • User-generated titles, notes, or descriptions of tasks and calendar events.
    • Recalled memory text or context snippets from Hindsight.
    • Raw iCalendar (.ics) VEVENT/VTODO strings or raw XML/JSON request bodies.
  • Permitted Diagnostic Fields:
    • Request ID (request_id).
    • Operation ID (operation_id).
    • Backend subsystem class (radicale, sqlite, hindsight, jev).
    • Request duration (in milliseconds or microseconds).
    • Sanitized HTTP route and status code.
    • Normalized error codes (invalid_input, forbidden, etag_conflict).

Exact Technical References & Specifications

1. Operations Security: Authentication & Redaction

From docs/technical/operations-security.md (lines 35–42):

"Life bearer tokens are high-entropy random secrets with separate read/write/admin scopes; a write token does not automatically grant admin. Store verification digests or secret files protected at rest and compare fixed-length derived digests in constant time. Reject malformed tokens before lookup without logging them. Token identity maps to the single user and actor audit record; no user-supplied actor may spoof a different principal. Rate-limit failed authentication and expensive queries.

The CLI/API table owns route scopes. Reads include read-only POST searches/judgments. Every mutation additionally needs operation identity; proposal apply needs reviewed approval fields and ETag preflight. The service cannot prove a human consented merely from an agent assertion. Agent permissions and review UX are part of the security boundary; the skill itself is not enforcement.

Enforce request/body/time/range limits before expensive parsing or model calls. JSON/XML/ICS parsers apply depth/size limits; no XML external entities. Limit expansion to the calendar range/instance contract. Redact Authorization, secret paths where sensitive, titles/descriptions, recalled content, raw ICS and model state from diagnostics. Log request ID, operation ID, backend class, duration and sanitized error code. Recalled/calendar/model text is untrusted data; never execute embedded instructions or use it to change API destinations/policies."

2. Route Scope Ownership Table

From docs/technical/cli-api.md (lines 27–56):

Command Method and route Required Scope
life agenda GET /v1/agenda read
life tasks list GET /v1/tasks read
life item get ID GET /v1/items/{id} read
life resolve POST /v1/resolve read
life next POST /v1/next read
life review POST /v1/reviews read
life context search POST /v1/context/search read
life timeline GET /v1/timeline read
life proposal get ID GET /v1/proposals/{id} read
life operation get ID GET /v1/operations/{id} read
life doctor GET /health/live, /health/ready read (readiness) / public (liveness)
life task add POST /v1/tasks write
life task update ID PATCH /v1/tasks/{id} write
life task complete/reopen/cancel ID POST /v1/tasks/{id}/... write
life event add POST /v1/events write
life event update/cancel ID PATCH/POST /v1/events/{id}... write
life capture POST /v1/captures write
life plan POST /v1/plans write
life replan POST /v1/replans write
life context add POST /v1/context write
life proposal apply ID POST /v1/proposals/{id}/apply write
life proposal reject ID POST /v1/proposals/{id}/reject write
life sync POST /v1/sync admin

3. Verification Scenarios D03 & D04

From docs/technical/verification.md (lines 136–137):

ID Input / operation Observable acceptance
D03 Read token attempts write/admin; write token attempts admin forbidden; token/payload redacted; no unauthorized action
D04 Token rotation during operation replay New token for same identity can retrieve/replay operation; revoked token rejected; no duplicate write

Formal Citations

  • docs/technical/operations-security.md:
    • Section "Authentication and untrusted inputs" (lines 35–42)
    • Section "Configuration contract", Table of token_files (lines 17–20, 33–34)
  • docs/technical/cli-api.md:
    • Section "Transport and common contract" (lines 5–10)
    • Table "Command-to-route table" (lines 27–56)
    • Table "Error identifiers", unauthenticated, forbidden (lines 108–111)
  • docs/technical/verification.md:
    • Table "Deployment scenarios", Scenarios D03, D04 (lines 132, 136–137)

What to Build

1. Token Verification & Scope Middleware (src/api/auth.rs)

Implement the Axum authentication layer:

  • Define Scope enum: Read, Write, Admin with hierarchy checks: 
    #[derive(Debug, Clone, Copy, PartialEq, Eq)]
pub enum Scope {
    Read,
    Write,
    Admin,
}
    Note: Admin does not automatically imply Write unless explicitly configured; Write does never grant Admin.
  • Define AuthenticatedActor: 
    #[derive(Debug, Clone)]
pub struct AuthenticatedActor {
    pub actor_id: String,
    pub scopes: HashSet<Scope>,
}
  • Implement token authentication extractor:
    • Extracts Authorization: Bearer <TOKEN> header.
    • Computes SHA-256 digest of <TOKEN>.
    • Performs constant-time lookup against configured token hashes using subtle::ConstantTimeEq.
    • Rejects missing or invalid tokens with HTTP 401 Unauthorized and JSON error code unauthenticated.
    • Rejects valid tokens lacking the route's required scope with HTTP 403 Forbidden and JSON error code forbidden.

2. Diagnostic Tracing & PII Redaction Layer (src/logging/redaction.rs)

Implement custom tracing subscribers / formatters using tracing-subscriber:

  • Redact HTTP request headers:
    • Replace authorization, cookie, proxy-authorization with "[REDACTED]".
  • Request/Response body scrubbing:
    • Do not attach raw JSON strings, query text, or XML/ICS payloads to tracing spans.
    • Format log lines with metadata only: 
      INFO request_id="req_01HPX" actor_id="usr_alice" method=POST path=/v1/tasks status=201 duration_ms=14.2
  • Sanitized error emission:
    • Ensure error logs report only the sanitized enum code, request ID, and operation ID, completely omitting user-provided task descriptions or titles.

3. Token Rotation & Idempotency Storage Integration (src/storage/operations.rs)

  • In the operations table, associate the journaled record with actor_id rather than the raw bearer token or token digest.
  • When an operation replay request arrives:
    • Verify that the caller possesses a valid token mapped to the same actor_id.
    • If the actor matches and the request fingerprint is identical, return the stored outcome.
    • If a revoked token is used, reject with HTTP 401 Unauthorized without touching the operation record (Scenario D04).

4. Integration Test Suite (tests/auth_redaction.rs)

  • Test Case D03 (Scope Enforcement & Redaction):
    • Provision a read token and attempt POST /v1/tasks. Assert HTTP 403 Forbidden with error code forbidden.
    • Provision a write token and attempt POST /v1/sync. Assert HTTP 403 Forbidden with error code forbidden.
    • Capture all stdout/stderr logs produced during these calls and verify that zero bearer tokens, task titles, or body contents are present.
  • Test Case D04 (Token Rotation during Operation Replay):
    • Issue POST /v1/tasks using Token A (mapped to usr_alice), successfully creating a task under operation_id: op_rot_1.
    • Rotate tokens: add Token B (mapped to usr_alice) and remove Token A.
    • Send the identical POST /v1/tasks request with Token B and operation_id: op_rot_1.
    • Assert HTTP 200/201 with the identical created task returned from the journal, and assert that exactly zero additional calendar writes occurred.
    • Attempt request with old Token A; assert immediate HTTP 401 rejection.

Acceptance Criteria

Observable Assertions & Verification

  • Incoming bearer tokens are compared using SHA-256 digests and constant-time equality checks (subtle::ConstantTimeEq).
  • Requests with a read token attempting to execute POST /v1/tasks return HTTP 403 Forbidden with error.code: "forbidden" (Scenario D03).
  • Requests with a write token attempting to execute POST /v1/sync return HTTP 403 Forbidden with error.code: "forbidden" (Scenario D03).
  • Standard and diagnostic logs inspected during task creation and search queries contain zero bearer tokens, authorization headers, calendar event titles, task descriptions, or raw ICS text (Scenario D03).
  • Rotating a token configuration allows the newly provisioned token for the same actor_id to replay and retrieve past operations without triggering duplicate writes (Scenario D04).
  • Requests presenting a revoked or deleted token fail immediately with HTTP 401 Unauthorized (Scenario D04).
  • Automated tests in tests/auth_redaction.rs verify scenarios D03 and D04.

Notes

  • Route scopes live in one table (auth::route_access) and the layer is applied to the whole router including the fallback; unlisted /v1 routes and all non-/v1 paths fail closed at admin. Only GET /health/live is public.
  • This PR also landed the deferred Cargo.toml authors chore (Yuris Cakranegara <hello@yuris.dev>) in the same commit that added the subtle dependency, with Cargo.lock committed.
  • Stale brief corrected: the deprecated ring::constant_time::verify_slices_are_equal was not present in the tree; src/api/pagination.rs was extended to ConstantTimeEq as a straight improvement.

Blocked by

  • #034: Hardened Production Container & Exclusive Directory Lock
## Summary Implement fine-grained scoped bearer token authentication (`read`, `write`, `admin`) with constant-time token comparison, stable actor identity mapping across token rotations, and comprehensive PII/credential redaction across all structured logs, tracing spans, and error diagnostic output. --- ## Background & Technical Rationale ### Principle of Least Privilege: Token Scopes The `life` service exposes HTTPS endpoints consumed by automated agents, background workers, and human operators. In accordance with the principle of least privilege: - **`read` Scope:** Allows querying agendas, listing tasks, fetching items, resolving references, searching context, and viewing timeline history. It permits read-only analytical POST requests (e.g. `/v1/reviews`, `/v1/next`), but strictly rejects all state mutations. - **`write` Scope:** Allows adding/updating/completing tasks and events, capturing outcomes, adding context, and generating or applying reviewed proposals. It **never** grants administrative or maintenance operations. - **`admin` Scope:** Required for system-level operations such as backend sync rebaselining (`POST /v1/sync`) or diagnostic inspections requiring privileged access. An agent operating in an advisory or read-only capacity must be provisioned with a `read` token. A write token attempting an administrative action must be rejected with HTTP 403 Forbidden. ### Constant-Time Verification & Secret Protection To prevent timing attacks that could reveal token characters based on comparison latency: 1. Bearer tokens must be high-entropy random secrets (minimum 256 bits). 2. The service never stores plaintext tokens. Tokens are configured via external secret files containing SHA-256 verification digests or hashed tokens. 3. Incoming bearer tokens are hashed with SHA-256, and compared against configured valid digests using fixed-length, constant-time comparison (`subtle::ConstantTimeEq`). 4. Malformed tokens (e.g. non-ASCII or incorrect lengths) are rejected before database queries without logging their contents. ### Token Rotation & Stable Actor Identity (Scenario D04) Bearer tokens must be rotatable without disrupting in-flight operations or breaking idempotency: - The service configuration maps secret token files to a canonical **actor identity** (e.g., `actor_id: "usr_alice"`). - During rotation, a new token is provisioned with the same actor identity before the old token is revoked. - When an operation is replayed or queried (e.g. `life operation get <OP_ID>`), the request succeeds under the new token because idempotency and operation ownership are bound to the stable actor identity, not the transient bearer token credential. - Once revoked, the old token is rejected immediately. ### Diagnostic Redaction & PII Safeguards (Scenario D03) Diagnostic logs and tracing systems are a frequent source of accidental data exfiltration. The service and CLI must enforce strict diagnostic redaction: - **Never Logged:** - `Authorization` header values, bearer tokens, or password strings. - User-generated titles, notes, or descriptions of tasks and calendar events. - Recalled memory text or context snippets from Hindsight. - Raw iCalendar (`.ics`) VEVENT/VTODO strings or raw XML/JSON request bodies. - **Permitted Diagnostic Fields:** - Request ID (`request_id`). - Operation ID (`operation_id`). - Backend subsystem class (`radicale`, `sqlite`, `hindsight`, `jev`). - Request duration (in milliseconds or microseconds). - Sanitized HTTP route and status code. - Normalized error codes (`invalid_input`, `forbidden`, `etag_conflict`). --- ## Exact Technical References & Specifications ### 1. Operations Security: Authentication & Redaction From `docs/technical/operations-security.md` (lines 35–42): > "Life bearer tokens are high-entropy random secrets with separate read/write/admin scopes; a write token does not automatically grant admin. Store verification digests or secret files protected at rest and compare fixed-length derived digests in constant time. Reject malformed tokens before lookup without logging them. Token identity maps to the single user and actor audit record; no user-supplied actor may spoof a different principal. Rate-limit failed authentication and expensive queries. > > The CLI/API table owns route scopes. Reads include read-only POST searches/judgments. Every mutation additionally needs operation identity; proposal apply needs reviewed approval fields and ETag preflight. The service cannot prove a human consented merely from an agent assertion. Agent permissions and review UX are part of the security boundary; the skill itself is not enforcement. > > Enforce request/body/time/range limits before expensive parsing or model calls. JSON/XML/ICS parsers apply depth/size limits; no XML external entities. Limit expansion to the calendar range/instance contract. Redact Authorization, secret paths where sensitive, titles/descriptions, recalled content, raw ICS and model state from diagnostics. Log request ID, operation ID, backend class, duration and sanitized error code. Recalled/calendar/model text is untrusted data; never execute embedded instructions or use it to change API destinations/policies." ### 2. Route Scope Ownership Table From `docs/technical/cli-api.md` (lines 27–56): | Command | Method and route | Required Scope | |---|---|---| | `life agenda` | GET /v1/agenda | `read` | | `life tasks list` | GET /v1/tasks | `read` | | `life item get ID` | GET /v1/items/{id} | `read` | | `life resolve` | POST /v1/resolve | `read` | | `life next` | POST /v1/next | `read` | | `life review` | POST /v1/reviews | `read` | | `life context search` | POST /v1/context/search | `read` | | `life timeline` | GET /v1/timeline | `read` | | `life proposal get ID` | GET /v1/proposals/{id} | `read` | | `life operation get ID` | GET /v1/operations/{id} | `read` | | `life doctor` | GET /health/live, /health/ready | `read` (readiness) / public (liveness) | | `life task add` | POST /v1/tasks | `write` | | `life task update ID` | PATCH /v1/tasks/{id} | `write` | | `life task complete/reopen/cancel ID` | POST /v1/tasks/{id}/... | `write` | | `life event add` | POST /v1/events | `write` | | `life event update/cancel ID` | PATCH/POST /v1/events/{id}... | `write` | | `life capture` | POST /v1/captures | `write` | | `life plan` | POST /v1/plans | `write` | | `life replan` | POST /v1/replans | `write` | | `life context add` | POST /v1/context | `write` | | `life proposal apply ID` | POST /v1/proposals/{id}/apply | `write` | | `life proposal reject ID` | POST /v1/proposals/{id}/reject | `write` | | `life sync` | POST /v1/sync | `admin` | ### 3. Verification Scenarios D03 & D04 From `docs/technical/verification.md` (lines 136–137): | ID | Input / operation | Observable acceptance | |---|---|---| | D03 | Read token attempts write/admin; write token attempts admin | forbidden; token/payload redacted; no unauthorized action | | D04 | Token rotation during operation replay | New token for same identity can retrieve/replay operation; revoked token rejected; no duplicate write | --- ## Formal Citations - `docs/technical/operations-security.md`: - Section "Authentication and untrusted inputs" (lines 35–42) - Section "Configuration contract", Table of token_files (lines 17–20, 33–34) - `docs/technical/cli-api.md`: - Section "Transport and common contract" (lines 5–10) - Table "Command-to-route table" (lines 27–56) - Table "Error identifiers", `unauthenticated`, `forbidden` (lines 108–111) - `docs/technical/verification.md`: - Table "Deployment scenarios", Scenarios D03, D04 (lines 132, 136–137) --- ## What to Build ### 1. Token Verification & Scope Middleware (`src/api/auth.rs`) Implement the Axum authentication layer: - Define `Scope` enum: `Read`, `Write`, `Admin` with hierarchy checks: ```rust #[derive(Debug, Clone, Copy, PartialEq, Eq)] pub enum Scope { Read, Write, Admin, } ``` Note: `Admin` does not automatically imply `Write` unless explicitly configured; `Write` does **never** grant `Admin`. - Define `AuthenticatedActor`: ```rust #[derive(Debug, Clone)] pub struct AuthenticatedActor { pub actor_id: String, pub scopes: HashSet<Scope>, } ``` - Implement token authentication extractor: - Extracts `Authorization: Bearer <TOKEN>` header. - Computes SHA-256 digest of `<TOKEN>`. - Performs constant-time lookup against configured token hashes using `subtle::ConstantTimeEq`. - Rejects missing or invalid tokens with HTTP 401 Unauthorized and JSON error code `unauthenticated`. - Rejects valid tokens lacking the route's required scope with HTTP 403 Forbidden and JSON error code `forbidden`. ### 2. Diagnostic Tracing & PII Redaction Layer (`src/logging/redaction.rs`) Implement custom tracing subscribers / formatters using `tracing-subscriber`: - Redact HTTP request headers: - Replace `authorization`, `cookie`, `proxy-authorization` with `"[REDACTED]"`. - Request/Response body scrubbing: - Do not attach raw JSON strings, query text, or XML/ICS payloads to tracing spans. - Format log lines with metadata only: ``` INFO request_id="req_01HPX" actor_id="usr_alice" method=POST path=/v1/tasks status=201 duration_ms=14.2 ``` - Sanitized error emission: - Ensure error logs report only the sanitized enum code, request ID, and operation ID, completely omitting user-provided task descriptions or titles. ### 3. Token Rotation & Idempotency Storage Integration (`src/storage/operations.rs`) - In the `operations` table, associate the journaled record with `actor_id` rather than the raw bearer token or token digest. - When an operation replay request arrives: - Verify that the caller possesses a valid token mapped to the same `actor_id`. - If the actor matches and the request fingerprint is identical, return the stored outcome. - If a revoked token is used, reject with HTTP 401 Unauthorized without touching the operation record (Scenario D04). ### 4. Integration Test Suite (`tests/auth_redaction.rs`) - **Test Case D03 (Scope Enforcement & Redaction):** - Provision a `read` token and attempt `POST /v1/tasks`. Assert HTTP 403 Forbidden with error code `forbidden`. - Provision a `write` token and attempt `POST /v1/sync`. Assert HTTP 403 Forbidden with error code `forbidden`. - Capture all stdout/stderr logs produced during these calls and verify that zero bearer tokens, task titles, or body contents are present. - **Test Case D04 (Token Rotation during Operation Replay):** - Issue `POST /v1/tasks` using Token A (mapped to `usr_alice`), successfully creating a task under `operation_id: op_rot_1`. - Rotate tokens: add Token B (mapped to `usr_alice`) and remove Token A. - Send the identical `POST /v1/tasks` request with Token B and `operation_id: op_rot_1`. - Assert HTTP 200/201 with the identical created task returned from the journal, and assert that exactly zero additional calendar writes occurred. - Attempt request with old Token A; assert immediate HTTP 401 rejection. --- ## Acceptance Criteria ### Observable Assertions & Verification - [x] Incoming bearer tokens are compared using SHA-256 digests and constant-time equality checks (`subtle::ConstantTimeEq`). - [x] Requests with a `read` token attempting to execute `POST /v1/tasks` return HTTP 403 Forbidden with `error.code: "forbidden"` (Scenario D03). - [x] Requests with a `write` token attempting to execute `POST /v1/sync` return HTTP 403 Forbidden with `error.code: "forbidden"` (Scenario D03). - [x] Standard and diagnostic logs inspected during task creation and search queries contain zero bearer tokens, authorization headers, calendar event titles, task descriptions, or raw ICS text (Scenario D03). - [x] Rotating a token configuration allows the newly provisioned token for the same `actor_id` to replay and retrieve past operations without triggering duplicate writes (Scenario D04). - [x] Requests presenting a revoked or deleted token fail immediately with HTTP 401 Unauthorized (Scenario D04). - [x] Automated tests in `tests/auth_redaction.rs` verify scenarios D03 and D04. --- ## Notes - Route scopes live in one table (`auth::route_access`) and the layer is applied to the whole router including the fallback; unlisted `/v1` routes and all non-`/v1` paths fail closed at `admin`. Only `GET /health/live` is public. - **This PR also landed the deferred `Cargo.toml` authors chore** (`Yuris Cakranegara <hello@yuris.dev>`) in the same commit that added the `subtle` dependency, with `Cargo.lock` committed. - Stale brief corrected: the deprecated `ring::constant_time::verify_slices_are_equal` was not present in the tree; `src/api/pagination.rs` was extended to `ConstantTimeEq` as a straight improvement. ## Blocked by - #034: Hardened Production Container & Exclusive Directory Lock
yuris changed title from Issue #035: Scoped Bearer Token Auth & PII Diagnostic Redaction to Scoped Bearer Token Auth & PII Diagnostic Redaction 2026-09-22 20:02:33 +00:00
yuris closed this issue 2026-09-23 00:38:58 +00:00
Sign in to join this conversation.
No Branch/Tag specified
Branches Tags
main
v0.2.4
v0.2.3
v0.2.2
v0.2.1
v0.2.0
v0.1.0
Labels
Clear labels
No items
No labels
Milestone
Clear milestone
No items
No milestone
Projects
Clear projects
No items
No project
Assignees
Clear assignees
No assignees
1 participant
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference
yuris/life-service#35
No description provided.