Configuration Loader, Secret File Access & Health Checks #3

New issue

Closed

opened 2026-09-22 17:58:53 +00:00 by yuris · 0 comments

yuris commented 2026-09-22 17:58:53 +00:00

Owner

Copy link

Summary

Implement the typed TOML configuration loader for life serve and life CLI, supporting mounted secret files, strict path validation, fail-fast configuration checks, and the foundational /health/live and /health/ready diagnostic endpoints and life doctor CLI command.

Background & Technical Rationale

The Life service requires strict configuration hygiene to prevent deployment accidents, credential leakage, and silent semantic divergence:

1. Secret File Invariant: Credentials (HTTP basic auth passwords, API bearer tokens) must never be passed as plain text strings in TOML literals, command-line arguments (argv), container environment variables, or image layers. Instead, configuration must reference explicit mounted secret file paths (*_file), reading the secret value at runtime and trimming trailing whitespace/newlines. Permissions must be validated.
2. Fail-Fast Validation: Startup must halt immediately with exit code 2 if unknown configuration keys are encountered, required fields are missing, IANA timezone strings are invalid, collection paths are malformed, or secret files are unreadable.
3. Timezone Equivalence: The planning.timezone must strictly match the radicale.timezone. A silent divergence between planning and calendar timezones causes corrupted scheduling boundaries.
4. Health Check Separation:
   • /health/live checks purely that the Axum HTTP process is running and responding.
   • /health/ready verifies that configuration is valid, schema migrations have run, and the local SQLite journal storage is writable. Degradation of optional upstream backends (e.g. Hindsight or Jev) is reported as a capability warning, not an unhealthy process restart trigger.
5. Sanitization: Public health probes and diagnostic CLI outputs (life doctor) must never leak secret tokens, passwords, raw host paths, or personal collection contents.

Exact Technical References & Specifications

Configuration Contract & Tables

From docs/technical/operations-security.md#configuration-contract:

"Future service/client configuration is TOML. Unknown keys, missing timezone/collection mapping, unreadable secrets, invalid TLS URLs and contradictory limits fail startup. No actual config file is created here. Paths are explicit; local development data is outside the repository under the configured data directory. Endpoint URLs, bank/collection identifiers, IANA timezone, Swarm node and backup destinations are deployment inputs, not inferred from memory."

Service section	Required fields	Optional proposed settings
server	listen, token_files mapping token identity to secret file and scopes, trusted_proxy policy	request_body_max_bytes=1048576, request_timeout_seconds=30, shutdown_timeout_seconds=30
radicale	base_url, username_file, password_file, timezone, collections mapping logical ID to approved discovered URL	connect_timeout_seconds=5, request_timeout_seconds=20, reconcile_attempts=3
hindsight	enabled; when enabled base_url, bank_id, authorization_file	request_timeout_seconds=20, recall_max_tokens=1024
jev	enabled; when enabled base_url, bearer_token_file, model=jev-1.13.0	request_timeout_seconds=10; evaluation-approved thresholds only
storage	data_directory, database_path	SQLite settings fixed by timeline contract; backup staging directory outside source
planning	timezone, working_windows, collection_ids	weekday_career_floor=1, daily_outcome_max=3, active_discretionary_projects=1, unscheduled_fraction=0.40, deferral_review_count=2

Client section	Fields
service	url (HTTPS), token_file, timeout_seconds; optional explicit CA bundle
display	timezone (must agree with requested service temporal interpretation), default_json; no silent host timezone fallback

"Planning timezone must equal the configured calendar timezone; a future multi-zone policy needs a reviewed contract rather than silent divergence. working_windows defines local weekday start/end pairs, with no implicit business hours. Empty windows mean unknown planning capacity, not an empty agenda. collection_ids must reference configured collections. Hindsight/Jev can be deliberately disabled; dependent commands return named unavailable capability while deterministic functions remain usable.

Secrets contain only the required credential value/Authorization string, with file permissions limited to the service UID or intended agent. No credentials in TOML literals, argv, image layers, URLs, example production hostnames or logs. Rotation can use a replacement mounted file and controlled reload/restart; old/new Life token identities overlap only for a bounded planned interval. Clients receive the new secret before revocation, and operation idempotency remains stable across token rotation."

Health Probes and Diagnostics

From docs/technical/operations-security.md#health-migrations-and-shutdown:

"/health/live checks the process can respond, not backend availability. /health/ready checks configuration, migration completion and writable journal storage; detailed authenticated output distinguishes backend/capability degradation. Hindsight/Jev outages do not trigger restart loops or make live agenda inherently unavailable. Radicale failure disables calendar-dependent actions but should not restart the process. Public probes reveal no hostnames, collection contents or tokens."

CLI Doctor Command

From docs/technical/cli-api.md#command-to-route-table:
| life doctor | GET /health/live and GET /health/ready | read for detailed readiness |

From docs/technical/cli-api.md#transport-and-common-contract:

"life doctor prints one combined envelope with sanitized process/storage/backend diagnostics, not secret configuration."

Deployment Scenario D02 Requirement

From docs/technical/verification.md#deployment-scenarios:

ID	Input / operation	Observable acceptance
D02	Missing timezone, invalid collection, unknown TOML key or unreadable secret	Configuration validation fails before serving/mutating

Formal Section Citations

• docs/technical/operations-security.md#configuration-contract: Defines the TOML sections, required keys, secret loading rules, and timezone alignment.
• docs/technical/operations-security.md#health-migrations-and-shutdown: Details health check separation and backend degradation isolation.
• docs/technical/cli-api.md#command-to-route-table: Defines life doctor route mapping and sanitization.
• docs/technical/verification.md#deployment-scenarios: Formulates normative scenario D02 for configuration validation.

What to Build (Step-by-Step)

1. Implement Configuration Models (src/config.rs):

   • Define Serde structs with #[serde(deny_unknown_fields)] for strict schema enforcement:
     • ServiceConfig containing ServerConfig, RadicaleConfig, Option<HindsightConfig>, Option<JevConfig>, StorageConfig, PlanningConfig.
     • ClientConfigFile containing ClientServiceConfig, ClientDisplayConfig.
   • Implement custom deserialization or validation for secret files:
     • Helper read_secret_file(path: &Path) -> Result<Zeroizing<String>, ConfigError> that reads file content, verifies it is valid UTF-8, trims trailing \r\n or \n, ensures it is non-empty, and wraps secret bytes in a zeroizing container (e.g. zeroize::Zeroizing).
   • Validate timezone strings against chrono_tz::Tz::from_str.
   • Validate invariant: planning.timezone == radicale.timezone.
   • Validate that planning.collection_ids reference known keys in radicale.collections.

2. Implement Health Endpoints in Axum (src/api.rs):

   • GET /health/live:
     • Returns HTTP 200 with minimal JSON {"status": "ok"}.
   • GET /health/ready:
     • Checks internal state:
       1. Database worker channel is active and journal storage is writable.
       2. Schema migrations have executed.
       3. Checks upstream reachability for informational status (without failing readiness if optional services like Hindsight are degraded).
     • Returns HTTP 200 with sanitized JSON capability report if core storage and config are valid, or HTTP 503 dependency_unavailable if storage is unwritable or initialization incomplete.

3. Implement life doctor CLI Command (src/cli.rs):

   • Issue HTTP GET requests to /health/live and /health/ready.
   • Aggregate status into a diagnostic data struct containing:
     • process: status, uptime, version.
     • storage: sqlite status, journal mode, integrity flag.
     • capabilities: radicale, hindsight, jev reachability.
     • Redacted configuration overview (displaying collection count and token count, without sensitive URLs or paths).
   • Format and output using ResponseEnvelope on stdout.
   • Exit with code 0 if fully healthy, exit with code 5 if core dependencies are unavailable.

4. Integration Tests for Configuration Validation (Scenario D02):

   • Test loading TOML with an unknown key: verify immediate error and exit code 2.
   • Test loading TOML with a missing timezone field: verify immediate error and exit code 2.
   • Test loading TOML with an invalid IANA timezone (e.g. Fake/Zone): verify error and exit code 2.
   • Test loading TOML referencing a nonexistent *_file path: verify failure and exit code 2, ensuring the error message does not leak sensitive environment details.
   • Test loading TOML with mismatched planning and calendar timezones: verify failure and exit code 2.

Acceptance Criteria

• Specifying an unknown key in the TOML configuration causes deserialization to fail with an explicit validation error mentioning the unknown field.
• Specifying an unreadable or nonexistent secret file path causes configuration loading to fail with exit code 2.
• Specifying an invalid timezone string (e.g. Invalid/Zone) causes startup to fail with exit code 2.
• A mismatch between planning.timezone and radicale.timezone causes startup to fail with exit code 2.
• /health/live returns HTTP 200 as long as the process is alive.
• /health/ready returns HTTP 200 when storage is writable and migrations are complete, even if Hindsight or Jev are disabled/unreachable (verifying Scenario D07: distinguishing write/storage failure from optional backend degradation).
• life doctor --json outputs diagnostic JSON containing sanitized system status without leaking secret tokens or sensitive host paths.
• Directly proves verification Scenario D02 from docs/technical/verification.md.

Blocked by

• #002: Cargo Workspace & Single Binary CLI/API Scaffold

## Summary Implement the typed TOML configuration loader for `life serve` and `life` CLI, supporting mounted secret files, strict path validation, fail-fast configuration checks, and the foundational `/health/live` and `/health/ready` diagnostic endpoints and `life doctor` CLI command. ## Background & Technical Rationale The Life service requires strict configuration hygiene to prevent deployment accidents, credential leakage, and silent semantic divergence: 1. **Secret File Invariant**: Credentials (HTTP basic auth passwords, API bearer tokens) must never be passed as plain text strings in TOML literals, command-line arguments (`argv`), container environment variables, or image layers. Instead, configuration must reference explicit mounted secret file paths (`*_file`), reading the secret value at runtime and trimming trailing whitespace/newlines. Permissions must be validated. 2. **Fail-Fast Validation**: Startup must halt immediately with exit code 2 if unknown configuration keys are encountered, required fields are missing, IANA timezone strings are invalid, collection paths are malformed, or secret files are unreadable. 3. **Timezone Equivalence**: The `planning.timezone` must strictly match the `radicale.timezone`. A silent divergence between planning and calendar timezones causes corrupted scheduling boundaries. 4. **Health Check Separation**: - `/health/live` checks purely that the Axum HTTP process is running and responding. - `/health/ready` verifies that configuration is valid, schema migrations have run, and the local SQLite journal storage is writable. Degradation of optional upstream backends (e.g. Hindsight or Jev) is reported as a capability warning, not an unhealthy process restart trigger. 5. **Sanitization**: Public health probes and diagnostic CLI outputs (`life doctor`) must never leak secret tokens, passwords, raw host paths, or personal collection contents. ## Exact Technical References & Specifications ### Configuration Contract & Tables From `docs/technical/operations-security.md#configuration-contract`: > "Future service/client configuration is TOML. Unknown keys, missing timezone/collection mapping, unreadable secrets, invalid TLS URLs and contradictory limits fail startup. No actual config file is created here. Paths are explicit; local development data is outside the repository under the configured data directory. Endpoint URLs, bank/collection identifiers, IANA timezone, Swarm node and backup destinations are deployment inputs, not inferred from memory." | Service section | Required fields | Optional proposed settings | |---|---|---| | server | listen, token_files mapping token identity to secret file and scopes, trusted_proxy policy | request_body_max_bytes=1048576, request_timeout_seconds=30, shutdown_timeout_seconds=30 | | radicale | base_url, username_file, password_file, timezone, collections mapping logical ID to approved discovered URL | connect_timeout_seconds=5, request_timeout_seconds=20, reconcile_attempts=3 | | hindsight | enabled; when enabled base_url, bank_id, authorization_file | request_timeout_seconds=20, recall_max_tokens=1024 | | jev | enabled; when enabled base_url, bearer_token_file, model=jev-1.13.0 | request_timeout_seconds=10; evaluation-approved thresholds only | | storage | data_directory, database_path | SQLite settings fixed by timeline contract; backup staging directory outside source | | planning | timezone, working_windows, collection_ids | weekday_career_floor=1, daily_outcome_max=3, active_discretionary_projects=1, unscheduled_fraction=0.40, deferral_review_count=2 | | Client section | Fields | |---|---| | service | url (HTTPS), token_file, timeout_seconds; optional explicit CA bundle | | display | timezone (must agree with requested service temporal interpretation), default_json; no silent host timezone fallback | > "Planning timezone must equal the configured calendar timezone; a future multi-zone policy needs a reviewed contract rather than silent divergence. working_windows defines local weekday start/end pairs, with no implicit business hours. Empty windows mean unknown planning capacity, not an empty agenda. collection_ids must reference configured collections. Hindsight/Jev can be deliberately disabled; dependent commands return named unavailable capability while deterministic functions remain usable. > > Secrets contain only the required credential value/Authorization string, with file permissions limited to the service UID or intended agent. No credentials in TOML literals, argv, image layers, URLs, example production hostnames or logs. Rotation can use a replacement mounted file and controlled reload/restart; old/new Life token identities overlap only for a bounded planned interval. Clients receive the new secret before revocation, and operation idempotency remains stable across token rotation." ### Health Probes and Diagnostics From `docs/technical/operations-security.md#health-migrations-and-shutdown`: > "`/health/live` checks the process can respond, not backend availability. `/health/ready` checks configuration, migration completion and writable journal storage; detailed authenticated output distinguishes backend/capability degradation. Hindsight/Jev outages do not trigger restart loops or make live agenda inherently unavailable. Radicale failure disables calendar-dependent actions but should not restart the process. Public probes reveal no hostnames, collection contents or tokens." ### CLI Doctor Command From `docs/technical/cli-api.md#command-to-route-table`: `| life doctor | GET /health/live and GET /health/ready | read for detailed readiness |` From `docs/technical/cli-api.md#transport-and-common-contract`: > "`life doctor` prints one combined envelope with sanitized process/storage/backend diagnostics, not secret configuration." ### Deployment Scenario D02 Requirement From `docs/technical/verification.md#deployment-scenarios`: | ID | Input / operation | Observable acceptance | |---|---|---| | D02 | Missing timezone, invalid collection, unknown TOML key or unreadable secret | Configuration validation fails before serving/mutating | ### Formal Section Citations - `docs/technical/operations-security.md#configuration-contract`: Defines the TOML sections, required keys, secret loading rules, and timezone alignment. - `docs/technical/operations-security.md#health-migrations-and-shutdown`: Details health check separation and backend degradation isolation. - `docs/technical/cli-api.md#command-to-route-table`: Defines `life doctor` route mapping and sanitization. - `docs/technical/verification.md#deployment-scenarios`: Formulates normative scenario D02 for configuration validation. ## What to Build (Step-by-Step) 1. **Implement Configuration Models (`src/config.rs`)**: - Define Serde structs with `#[serde(deny_unknown_fields)]` for strict schema enforcement: - `ServiceConfig` containing `ServerConfig`, `RadicaleConfig`, `Option<HindsightConfig>`, `Option<JevConfig>`, `StorageConfig`, `PlanningConfig`. - `ClientConfigFile` containing `ClientServiceConfig`, `ClientDisplayConfig`. - Implement custom deserialization or validation for secret files: - Helper `read_secret_file(path: &Path) -> Result<Zeroizing<String>, ConfigError>` that reads file content, verifies it is valid UTF-8, trims trailing `\r\n` or `\n`, ensures it is non-empty, and wraps secret bytes in a zeroizing container (e.g. `zeroize::Zeroizing`). - Validate timezone strings against `chrono_tz::Tz::from_str`. - Validate invariant: `planning.timezone == radicale.timezone`. - Validate that `planning.collection_ids` reference known keys in `radicale.collections`. 2. **Implement Health Endpoints in Axum (`src/api.rs`)**: - `GET /health/live`: - Returns HTTP 200 with minimal JSON `{"status": "ok"}`. - `GET /health/ready`: - Checks internal state: 1. Database worker channel is active and journal storage is writable. 2. Schema migrations have executed. 3. Checks upstream reachability for informational status (without failing readiness if optional services like Hindsight are degraded). - Returns HTTP 200 with sanitized JSON capability report if core storage and config are valid, or HTTP 503 `dependency_unavailable` if storage is unwritable or initialization incomplete. 3. **Implement `life doctor` CLI Command (`src/cli.rs`)**: - Issue HTTP GET requests to `/health/live` and `/health/ready`. - Aggregate status into a diagnostic data struct containing: - `process`: status, uptime, version. - `storage`: sqlite status, journal mode, integrity flag. - `capabilities`: radicale, hindsight, jev reachability. - Redacted configuration overview (displaying collection count and token count, without sensitive URLs or paths). - Format and output using `ResponseEnvelope` on `stdout`. - Exit with code 0 if fully healthy, exit with code 5 if core dependencies are unavailable. 4. **Integration Tests for Configuration Validation (Scenario D02)**: - Test loading TOML with an unknown key: verify immediate error and exit code 2. - Test loading TOML with a missing timezone field: verify immediate error and exit code 2. - Test loading TOML with an invalid IANA timezone (e.g. `Fake/Zone`): verify error and exit code 2. - Test loading TOML referencing a nonexistent `*_file` path: verify failure and exit code 2, ensuring the error message does not leak sensitive environment details. - Test loading TOML with mismatched planning and calendar timezones: verify failure and exit code 2. ## Acceptance Criteria - [x] Specifying an unknown key in the TOML configuration causes deserialization to fail with an explicit validation error mentioning the unknown field. - [x] Specifying an unreadable or nonexistent secret file path causes configuration loading to fail with exit code 2. - [x] Specifying an invalid timezone string (e.g. `Invalid/Zone`) causes startup to fail with exit code 2. - [x] A mismatch between `planning.timezone` and `radicale.timezone` causes startup to fail with exit code 2. - [x] `/health/live` returns HTTP 200 as long as the process is alive. - [x] `/health/ready` returns HTTP 200 when storage is writable and migrations are complete, even if Hindsight or Jev are disabled/unreachable (verifying Scenario D07: distinguishing write/storage failure from optional backend degradation). - [x] `life doctor --json` outputs diagnostic JSON containing sanitized system status without leaking secret tokens or sensitive host paths. - [x] Directly proves verification Scenario D02 from `docs/technical/verification.md`. ## Blocked by - #002: Cargo Workspace & Single Binary CLI/API Scaffold

yuris referenced this issue 2026-09-22 17:58:53 +00:00

SQLite WAL Connection Pool, Worker & Baseline Schema #4

yuris referenced this issue 2026-09-22 17:58:54 +00:00

CalDAV Connection Pooling, PROPFIND Discovery & Capabilities #7

yuris referenced this issue from a commit 2026-09-22 18:33:37 +00:00

feat(config): configuration loader, secret file access and health checks (#3)

yuris referenced this issue from a pull request that will close it, 2026-09-22 18:33:57 +00:00

feat(config): configuration loader, secret file access and health checks #39

yuris closed this issue 2026-09-22 18:36:53 +00:00

yuris referenced this issue from a commit 2026-09-22 18:36:54 +00:00

feat(config): configuration loader, secret file access and health checks (#3)

yuris referenced this issue from a commit 2026-09-22 18:46:02 +00:00

feat(config): configuration loader, secret file access and health checks (#39)

yuris changed title from Issue #003: Configuration Loader, Secret File Access & Health Checks to Configuration Loader, Secret File Access & Health Checks 2026-09-22 20:02:40 +00:00

Sign in to join this conversation.

No Branch/Tag specified

Branches Tags

main

v0.2.4

v0.2.3

v0.2.2

v0.2.1

v0.2.0

v0.1.0

Labels

Clear labels

No items

No labels

Milestone

Clear milestone

No items

No milestone

Projects

Clear projects

No items

No project

Assignees

Clear assignees

No assignees

1 participant

Notifications

Subscribe

Due date

The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference

yuris/life-service#3

No description provided.