Candidate Reference Resolution Engine #20

New issue

Closed

opened 2026-09-22 17:58:56 +00:00 by yuris · 0 comments

yuris commented 2026-09-22 17:58:56 +00:00

Owner

Copy link

Summary

Implement the candidate reference resolution engine (POST /v1/resolve / life resolve), providing deterministic exact matching over live calendar candidates, calibrated Jev reference_match.v1 semantic judgment for ambiguous phrases, explicit candidate pool narrowing disclosures, and interactive clarification (exit code 3) when uncertainty remains. Strictly enforce that semantic confidence never authorizes mutations.

Background & Technical Rationale

Users and AI agents frequently refer to tasks or calendar events using informal titles, conversational paraphrases, or partial keywords (e.g., "finish portfolio outline", "weekly sync"). To enable safe operations without hardcoding fragile lookups, the candidate reference resolution engine maps informal references to an authoritative calendar item ID (itm_...) and current ETag.

The design enforces strict architectural invariants:

1. Deterministic Boundary First:
   • Any query that exactly matches an item UID or matches an existing item's title verbatim (case-insensitive) resolves immediately with method: "deterministic".
   • In the presence of an exact match, the TypeSafe/Jev semantic adapter is never invoked, preserving API quota and eliminating latency.
2. Deterministic Candidate Pool for Semantic Evaluation:
   • When exact deterministic matching does not resolve uniquely, the engine queries active local calendar items to construct a candidate pool.
   • The candidate pool sent to Jev is capped at at most 20 live candidates, plus a fixed no_match option.
   • If more than 20 candidate items match initial token/substring heuristics, the pool is capped at 20, and the response must include an explicit warning: candidate_narrowed. If narrowing could exclude the intended reference, the engine prompts the user to narrow scope rather than treating the top candidate as exhaustive.
3. No Write Authority from Model Confidence:
   • Resolution NEVER automatically writes or mutates state based on model confidence.
   • Semantic reference matching produces an advisory identity suggestion. Until per-question evaluation supports a validated write threshold, semantic reference matches require user confirmation or return clarification.
4. Clarification and Abstention:
   • When Jev selects no_match, when the probability distribution is uncalibrated/ambiguous, when multiple candidates tie, or when Jev is unavailable, the engine returns status: "error" with code clarification_required (or needs_clarification structure in HTTP 200 data, terminating with CLI exit code 3).
   • The response includes the candidate options so the user or calling agent can select the intended item explicitly.
5. Data Privacy & In-Memory State:
   • User-supplied reference text is placed strictly within the request state object as untrusted data, never interpolated into Jev system instructions or rubrics.
   • Sensitive text can be provided via standard input (--input -) to prevent leaking private information into process argument lists (argv).

Exact Technical References & Specifications

Normative Question Specification

From docs/technical/jev.md (lines 15, 23–25, 32):

"Provider maximum Choice size is 255 options; Life sends at most 20 deterministic item candidates (plus a fixed no_match alternative), disclosing narrowing. If narrowing could exclude the intended reference, ask to narrow scope rather than treating the top candidate as exhaustive."

Question family	Primitive and purpose	Deterministic boundary
reference_match.v1	Choice over live IDs and no_match	Exact match first; calibrated abstention, then user clarification; never write from confidence

"No arbitrary confidence threshold grants write authority. Until per-question evaluation supports a threshold, semantic reference matches require clarification and recommendation scores remain advisory. Explicit deterministic task commands keep working without Jev."

CLI & API Route Contract

From docs/technical/cli-api.md (lines 41, 57, 88, 113, 127):

Command	Method and route	Scope
life resolve --text TEXT --kind task|event	POST /v1/resolve	read

• Command Payload & Results:
  • Required / optional input: text, kind (task | event); optional collection_ids.
  • Response data: item_id, etag, method (deterministic | semantic), candidate narrowing and judgment provenance; or clarification.
  • Flags: --text is for short non-sensitive references; resolve also accepts --input - with {text, kind} instead of those flags to avoid private text in argv.
• Error & Warning Identifiers:
  • clarification_required: Resolve ambiguity; exit code 3.
  • Warnings: candidate_narrowed, judgment_unavailable.

Verification Scenario W05

From docs/technical/verification.md (lines 19):

ID	Requirement and specification	Input / action	Observable acceptance
W05	Resolution	Exact ID/title match, then two plausible live references	Exact deterministic match; ambiguous match asks user, no write; narrowing disclosed

What to Build

1. src/domain/resolve.rs

• Domain service coordinating reference resolution:

  pub struct ResolutionEngine {
      calendar_client: Arc<CalendarClient>,
      jev_client: Option<Arc<JevClient>>,
  }
  
  #[derive(Debug, Clone, Serialize, Deserialize)]
  pub struct ResolveRequest {
      pub text: String,
      pub kind: ItemKind, // Task or Event
      pub collection_ids: Option<Vec<String>>,
  }
  
  #[derive(Debug, Clone, Serialize, Deserialize)]
  #[serde(tag = "status")]
  pub enum ResolveOutcome {
      #[serde(rename = "resolved")]
      Resolved {
          item_id: String,
          etag: String,
          method: ResolveMethod, // Deterministic or Semantic
          provenance: Option<JudgmentProvenance>,
      },
      #[serde(rename = "needs_clarification")]
      NeedsClarification {
          candidates: Vec<CandidateChoice>,
          prompt: String,
      },
  }
  

• Step-by-step resolution logic:
  1. Retrieve live candidate items from Radicale (open tasks for kind: task, active/upcoming events for kind: event).
  2. Exact deterministic match check:
     • Check if text matches any item's id or uid exactly.
     • Check if text.trim().to_lowercase() equals any item's title.trim().to_lowercase().
     • If exactly one item matches, return ResolveOutcome::Resolved with method: ResolveMethod::Deterministic, zero calls to Jev.
  3. Candidate filtering & narrowing:
     • Perform case-insensitive token and substring matching to select plausible items.
     • If candidate pool > 20, take top 20 candidates and record warning candidate_narrowed.
     • If candidate pool is empty, return ResolveOutcome::NeedsClarification with empty candidates and a prompt to rephrase.
  4. Semantic matching via Jev:
     • If Jev is disabled or unavailable, skip to NeedsClarification returning the top candidates with judgment_unavailable warning.
     • Construct question reference_match.v1 (choice primitive) with candidate item IDs mapped to item titles/details, plus the fixed option no_match: "None of the above candidates match the user reference".
     • Put untrusted user text into state.user_reference.
     • Execute question via JevClient.
     • Evaluate answer:
       • If choice is no_match or choice has ambiguity: return NeedsClarification with candidate choices.
       • Until per-question evaluation supports an approved threshold (docs/technical/jev.md:32, 59–61), semantic reference matches require user clarification; return NeedsClarification with candidate choices and judgment provenance. Only with an explicit evaluated and approved threshold can confidence above that threshold return Resolved.

2. src/jev/questions/reference_match.rs

• Builder for the reference_match.v1 Jev question:
  • Candidate items (including titles, statuses, and collections) are serialized strictly into the untrusted state JSON payload under state.candidates, NEVER interpolated into the criteria rubrics (docs/technical/jev.md:21).

  pub fn build_reference_match_question(
      candidates: &[Item],
  ) -> JevQuestion {
      let mut criteria = HashMap::new();
      for item in candidates {
          criteria.insert(
              item.id.clone(),
              format!("Candidate option matching item ID '{}'", item.id),
          );
      }
      criteria.insert(
          "no_match".to_string(),
          "None of the listed candidates correspond to the referenced item.".to_string(),
      );
  
      JevQuestion::Choice {
          instructions: "Select the candidate ID that corresponds to the referenced item described in state.candidates. If none match or ambiguity is high, select no_match.".to_string(),
          criteria,
      }
  }
  

3. Axum API Endpoint (src/api/routes/resolve.rs)

• Route POST /v1/resolve.
• Deserializes ResolveRequest from JSON body.
• Returns standard envelope:

  {
    "schema_version": 1,
    "request_id": "req_resolve_demo",
    "status": "ok",
    "data": {
      "item_id": "itm_demo",
      "etag": "\"v1\"",
      "method": "deterministic",
      "judgment_provenance": null
    },
    "warnings": [],
    "error": null
  }
  

• Or canonical clarification envelope (HTTP 200, exit code 3):

  {
    "schema_version": 1,
    "request_id": "req_resolve_clarify",
    "status": "needs_clarification",
    "data": {
      "questions": [
        {
          "id": "reference_selection",
          "prompt": "Multiple items match reference. Which item did you mean?",
          "options": [
            {"label": "Draft portfolio outline", "value": "itm_1"},
            {"label": "Review portfolio slides", "value": "itm_2"}
          ]
        }
      ],
      "candidates": [
        {"item_id": "itm_1", "title": "Draft portfolio outline", "etag": "\"v1\""},
        {"item_id": "itm_2", "title": "Review portfolio slides", "etag": "\"v2\""}
      ],
      "method": "semantic",
      "judgment_provenance": {
        "model": "jev-1.13.0",
        "question": "reference_match.v1"
      }
    },
    "warnings": [],
    "error": null
  }
  

4. CLI Subcommand (src/cli/commands/resolve.rs)

• life resolve:
  • Flags: --text <TEXT>, --kind <task|event>, --collection <ID>, --input -, --json.
  • Supports reading JSON payload { "text": "...", "kind": "task" } from stdin (--input -).
  • Prints resolved item ID and ETag to stdout on success (exit code 0).
  • On clarification_required, prints candidate choices to stderr/stdout and exits with code 3.

Acceptance Criteria

• Exact ID or title match resolves immediately with method: "deterministic" without calling Jev (Scenario W05).
• Resolving two or more plausible matches invokes Jev reference_match.v1; if ambiguous or uncertain, returns needs_clarification (Exit code 3) without mutating any state (Scenario W05).
• Candidate titles and calendar text are passed strictly via untrusted state data, never formatted into criteria rubrics.
• Absent an evaluated and approved threshold, semantic reference matching returns needs_clarification with candidate choices and exit code 3; confidence alone does not authorize resolution.
• Clarification response matches the canonical needs_clarification envelope with questions and candidates.
• Candidate pool sent to Jev never exceeds 20 items and always includes no_match.
• When more than 20 items match candidate heuristics, the response contains warning candidate_narrowed.
• When Jev is disabled or unavailable, deterministic matching still succeeds for exact matches, and ambiguous references degrade to user clarification with candidate choices (Scenario I07).
• Sensitive references passed via --input - are processed identically without exposing text in process arguments.
• Resolving a reference never performs writes to Radicale or SQLite.

Notes

• reference_match.v1 plus the resolve route/CLI only; the remaining families and the advisory fallback policy landed in #21.
• Candidate budget is 19 live candidates + the fixed no_match = 20 total, the stricter reading of Life's own MAX_CHOICE_CANDIDATES, with exclusions disclosed as candidate_narrowed.
• No dependency added, so the deferred Cargo.toml authors chore stays pending.

Blocked by

• #019: TypeSafe/Jev HTTP Adapter & Wire Primitives Validation

## Summary Implement the candidate reference resolution engine (`POST /v1/resolve` / `life resolve`), providing deterministic exact matching over live calendar candidates, calibrated Jev `reference_match.v1` semantic judgment for ambiguous phrases, explicit candidate pool narrowing disclosures, and interactive clarification (exit code 3) when uncertainty remains. Strictly enforce that semantic confidence never authorizes mutations. ## Background & Technical Rationale Users and AI agents frequently refer to tasks or calendar events using informal titles, conversational paraphrases, or partial keywords (e.g., "finish portfolio outline", "weekly sync"). To enable safe operations without hardcoding fragile lookups, the candidate reference resolution engine maps informal references to an authoritative calendar item ID (`itm_...`) and current `ETag`. The design enforces strict architectural invariants: 1. **Deterministic Boundary First:** - Any query that exactly matches an item UID or matches an existing item's title verbatim (case-insensitive) resolves immediately with `method: "deterministic"`. - In the presence of an exact match, the TypeSafe/Jev semantic adapter is **never invoked**, preserving API quota and eliminating latency. 2. **Deterministic Candidate Pool for Semantic Evaluation:** - When exact deterministic matching does not resolve uniquely, the engine queries active local calendar items to construct a candidate pool. - The candidate pool sent to Jev is capped at at most 20 live candidates, plus a fixed `no_match` option. - If more than 20 candidate items match initial token/substring heuristics, the pool is capped at 20, and the response must include an explicit warning: `candidate_narrowed`. If narrowing could exclude the intended reference, the engine prompts the user to narrow scope rather than treating the top candidate as exhaustive. 3. **No Write Authority from Model Confidence:** - Resolution NEVER automatically writes or mutates state based on model confidence. - Semantic reference matching produces an advisory identity suggestion. Until per-question evaluation supports a validated write threshold, semantic reference matches require user confirmation or return clarification. 4. **Clarification and Abstention:** - When Jev selects `no_match`, when the probability distribution is uncalibrated/ambiguous, when multiple candidates tie, or when Jev is unavailable, the engine returns `status: "error"` with code `clarification_required` (or `needs_clarification` structure in HTTP 200 data, terminating with CLI exit code 3). - The response includes the candidate options so the user or calling agent can select the intended item explicitly. 5. **Data Privacy & In-Memory State:** - User-supplied reference text is placed strictly within the request `state` object as untrusted data, never interpolated into Jev system instructions or rubrics. - Sensitive text can be provided via standard input (`--input -`) to prevent leaking private information into process argument lists (`argv`). ## Exact Technical References & Specifications ### Normative Question Specification From `docs/technical/jev.md` (lines 15, 23–25, 32): > "Provider maximum Choice size is 255 options; Life sends at most 20 deterministic item candidates (plus a fixed no_match alternative), disclosing narrowing. If narrowing could exclude the intended reference, ask to narrow scope rather than treating the top candidate as exhaustive." | Question family | Primitive and purpose | Deterministic boundary | |---|---|---| | reference_match.v1 | Choice over live IDs and no_match | Exact match first; calibrated abstention, then user clarification; never write from confidence | > "No arbitrary confidence threshold grants write authority. Until per-question evaluation supports a threshold, semantic reference matches require clarification and recommendation scores remain advisory. Explicit deterministic task commands keep working without Jev." ### CLI & API Route Contract From `docs/technical/cli-api.md` (lines 41, 57, 88, 113, 127): | Command | Method and route | Scope | |---|---|---| | `life resolve --text TEXT --kind task\|event` | POST /v1/resolve | read | - **Command Payload & Results:** - *Required / optional input:* `text`, `kind` (`task` | `event`); optional `collection_ids`. - *Response data:* `item_id`, `etag`, `method` (`deterministic` | `semantic`), candidate narrowing and judgment provenance; or clarification. - *Flags:* `--text` is for short non-sensitive references; resolve also accepts `--input -` with `{text, kind}` instead of those flags to avoid private text in `argv`. - **Error & Warning Identifiers:** - `clarification_required`: Resolve ambiguity; exit code 3. - Warnings: `candidate_narrowed`, `judgment_unavailable`. ### Verification Scenario W05 From `docs/technical/verification.md` (lines 19): | ID | Requirement and specification | Input / action | Observable acceptance | |---|---|---|---| | W05 | [Resolution](jev.md#versioned-questions) | Exact ID/title match, then two plausible live references | Exact deterministic match; ambiguous match asks user, no write; narrowing disclosed | ## What to Build ### 1. `src/domain/resolve.rs` - Domain service coordinating reference resolution: ```rust pub struct ResolutionEngine { calendar_client: Arc<CalendarClient>, jev_client: Option<Arc<JevClient>>, } #[derive(Debug, Clone, Serialize, Deserialize)] pub struct ResolveRequest { pub text: String, pub kind: ItemKind, // Task or Event pub collection_ids: Option<Vec<String>>, } #[derive(Debug, Clone, Serialize, Deserialize)] #[serde(tag = "status")] pub enum ResolveOutcome { #[serde(rename = "resolved")] Resolved { item_id: String, etag: String, method: ResolveMethod, // Deterministic or Semantic provenance: Option<JudgmentProvenance>, }, #[serde(rename = "needs_clarification")] NeedsClarification { candidates: Vec<CandidateChoice>, prompt: String, }, } ``` - **Step-by-step resolution logic:** 1. Retrieve live candidate items from Radicale (open tasks for `kind: task`, active/upcoming events for `kind: event`). 2. **Exact deterministic match check:** - Check if `text` matches any item's `id` or `uid` exactly. - Check if `text.trim().to_lowercase()` equals any item's `title.trim().to_lowercase()`. - If exactly one item matches, return `ResolveOutcome::Resolved` with `method: ResolveMethod::Deterministic`, zero calls to Jev. 3. **Candidate filtering & narrowing:** - Perform case-insensitive token and substring matching to select plausible items. - If candidate pool $> 20$, take top 20 candidates and record warning `candidate_narrowed`. - If candidate pool is empty, return `ResolveOutcome::NeedsClarification` with empty candidates and a prompt to rephrase. 4. **Semantic matching via Jev:** - If Jev is disabled or unavailable, skip to `NeedsClarification` returning the top candidates with `judgment_unavailable` warning. - Construct question `reference_match.v1` (`choice` primitive) with candidate item IDs mapped to item titles/details, plus the fixed option `no_match: "None of the above candidates match the user reference"`. - Put untrusted user text into `state.user_reference`. - Execute question via `JevClient`. - Evaluate answer: - If choice is `no_match` or choice has ambiguity: return `NeedsClarification` with candidate choices. - Until per-question evaluation supports an approved threshold (`docs/technical/jev.md:32, 59–61`), semantic reference matches require user clarification; return `NeedsClarification` with candidate choices and judgment provenance. Only with an explicit evaluated and approved threshold can confidence above that threshold return `Resolved`. ### 2. `src/jev/questions/reference_match.rs` - Builder for the `reference_match.v1` Jev question: - Candidate items (including titles, statuses, and collections) are serialized strictly into the untrusted state JSON payload under `state.candidates`, NEVER interpolated into the criteria rubrics (`docs/technical/jev.md:21`). ```rust pub fn build_reference_match_question( candidates: &[Item], ) -> JevQuestion { let mut criteria = HashMap::new(); for item in candidates { criteria.insert( item.id.clone(), format!("Candidate option matching item ID '{}'", item.id), ); } criteria.insert( "no_match".to_string(), "None of the listed candidates correspond to the referenced item.".to_string(), ); JevQuestion::Choice { instructions: "Select the candidate ID that corresponds to the referenced item described in state.candidates. If none match or ambiguity is high, select no_match.".to_string(), criteria, } } ``` ### 3. Axum API Endpoint (`src/api/routes/resolve.rs`) - Route `POST /v1/resolve`. - Deserializes `ResolveRequest` from JSON body. - Returns standard envelope: ```json { "schema_version": 1, "request_id": "req_resolve_demo", "status": "ok", "data": { "item_id": "itm_demo", "etag": "\"v1\"", "method": "deterministic", "judgment_provenance": null }, "warnings": [], "error": null } ``` - Or canonical clarification envelope (HTTP 200, exit code 3): ```json { "schema_version": 1, "request_id": "req_resolve_clarify", "status": "needs_clarification", "data": { "questions": [ { "id": "reference_selection", "prompt": "Multiple items match reference. Which item did you mean?", "options": [ {"label": "Draft portfolio outline", "value": "itm_1"}, {"label": "Review portfolio slides", "value": "itm_2"} ] } ], "candidates": [ {"item_id": "itm_1", "title": "Draft portfolio outline", "etag": "\"v1\""}, {"item_id": "itm_2", "title": "Review portfolio slides", "etag": "\"v2\""} ], "method": "semantic", "judgment_provenance": { "model": "jev-1.13.0", "question": "reference_match.v1" } }, "warnings": [], "error": null } ``` ### 4. CLI Subcommand (`src/cli/commands/resolve.rs`) - `life resolve`: - Flags: `--text <TEXT>`, `--kind <task|event>`, `--collection <ID>`, `--input -`, `--json`. - Supports reading JSON payload `{ "text": "...", "kind": "task" }` from stdin (`--input -`). - Prints resolved item ID and ETag to stdout on success (exit code 0). - On `clarification_required`, prints candidate choices to stderr/stdout and exits with code 3. ## Acceptance Criteria - [x] Exact ID or title match resolves immediately with `method: "deterministic"` without calling Jev (Scenario W05). - [x] Resolving two or more plausible matches invokes Jev `reference_match.v1`; if ambiguous or uncertain, returns `needs_clarification` (Exit code 3) without mutating any state (Scenario W05). - [x] Candidate titles and calendar text are passed strictly via untrusted state data, never formatted into criteria rubrics. - [x] Absent an evaluated and approved threshold, semantic reference matching returns `needs_clarification` with candidate choices and exit code 3; confidence alone does not authorize resolution. - [x] Clarification response matches the canonical `needs_clarification` envelope with questions and candidates. - [x] Candidate pool sent to Jev never exceeds 20 items and always includes `no_match`. - [x] When more than 20 items match candidate heuristics, the response contains warning `candidate_narrowed`. - [x] When Jev is disabled or unavailable, deterministic matching still succeeds for exact matches, and ambiguous references degrade to user clarification with candidate choices (Scenario I07). - [x] Sensitive references passed via `--input -` are processed identically without exposing text in process arguments. - [x] Resolving a reference never performs writes to Radicale or SQLite. ## Notes - `reference_match.v1` plus the resolve route/CLI only; the remaining families and the advisory fallback policy landed in #21. - Candidate budget is 19 live candidates + the fixed `no_match` = 20 total, the stricter reading of Life's own `MAX_CHOICE_CANDIDATES`, with exclusions disclosed as `candidate_narrowed`. - No dependency added, so the deferred `Cargo.toml` authors chore stays pending. ## Blocked by - #019: TypeSafe/Jev HTTP Adapter & Wire Primitives Validation

yuris changed title from Issue #020: Candidate Reference Resolution Engine to Candidate Reference Resolution Engine 2026-09-22 20:02:36 +00:00

yuris referenced this issue 2026-09-22 22:49:31 +00:00

feat(jev): reqwest adapter and wire primitive validation #54

yuris referenced this issue 2026-09-22 22:50:36 +00:00

TypeSafe/Jev HTTP Adapter & Wire Primitives Validation #19

yuris referenced this issue 2026-09-22 23:20:16 +00:00

feat(jev): advisory question families and graceful degradation #57

yuris referenced this issue from a pull request that will close it, 2026-09-22 23:27:11 +00:00

feat(jev): candidate reference resolution with deterministic first #58

yuris closed this issue 2026-09-22 23:28:59 +00:00

yuris referenced this issue from a commit 2026-09-22 23:29:00 +00:00

feat(jev): candidate reference resolution with deterministic first (#58)

Sign in to join this conversation.

No Branch/Tag specified

Branches Tags

main

v0.2.4

v0.2.3

v0.2.2

v0.2.1

v0.2.0

v0.1.0

Labels

Clear labels

No items

No labels

Milestone

Clear milestone

No items

No milestone

Projects

Clear projects

No items

No project

Assignees

Clear assignees

No assignees

1 participant

Notifications

Subscribe

Due date

The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference

yuris/life-service#20

No description provided.