Selection
Evidence precision and recall separate finding the right event from including the whole transcript.
Executable fixture v1
Local baselines
No Palo model result
Current evidence work
Conversation memory, measured before it is marketed.
The first benchmark asks whether a memory system can retrieve current evidence from a longitudinal conversation, suppress stale history, preserve required details, and return no data when the history contains no answer.
This page reports deterministic controls and a lexical baseline. It does not report Palo Bloom quality, model latency, production cost, or human-like memory.
Frozen fixture
11
Stored events
7
Later questions
5
Control and baseline arms
1
Evaluator oracle
The cases cover a changed preference, changed plan, action and outcome, unresolved application, durable fact, semantic paraphrase, distractors, and an explicit no-data question. Ground truth is visible to the evaluator and oracle only.
Baseline receipt
What the local systems actually did
| Arm | Precision | Recall | Current state | Stale suppression | No data |
|---|---|---|---|---|---|
| No memory | 0.0% | 0.0% | 0.0% | 100.0% | 100.0% |
| Full transcript | 9.1% | 100.0% | 16.7% | 28.6% | 0.0% |
| Deterministic random key | 0.0% | 0.0% | 0.0% | 100.0% | 0.0% |
| Lexical retrieval | 83.3% | 83.3% | 83.3% | 100.0% | 100.0% |
| Mock v1 contract | 83.3% | 83.3% | 83.3% | 100.0% | 100.0% |
| Oracle evaluator check | 100.0% | 100.0% | 100.0% | 100.0% | 100.0% |
The oracle reads ground truth directly. It verifies the evaluator and is not a memory system. Mock v1 uses the same lexical behavior as the lexical row, so matching results are expected.
Useful failure
Lexical matching does not understand a paraphrase.
When asked how the user's morning drink should be prepared, lexical retrieval misses the changed coffee preference. It selects an unrelated grant event because that event happens to contain the word "morning."
That miss is kept in the fixture. A future Bloom adapter must retrieve the right stored value without receiving the expected event ID, and it must do so on the same frozen history.
What each metric means
Continuity
Current-state accuracy requires the right evidence and rejects named stale evidence.
Honesty
No-data accuracy prevents a fluent renderer from inventing a memory that does not exist.
Next evidence gate
Connect Bloom without changing the test.
The first real adapter must retrieve separately stored values and report model, representation, runtime, and fixture versions. Retrieval will be scored before rendering. A later reader-controlled stage will test whether selected memory improves a defined answer or action at matched context and cost.