# Cost accounting spike T2: per-call cost records in the pi-durable session

> > **Status:** FINDINGS of Kanbus `chatticus-9e7710` (epic `chatticus-8c857b`), measured against `@earendil-works/pi-durable` > 1.0.2 and `@earendil-works/pi-ai` as installed at the repo root, on the production `Indexe...

Canonical: https://chattic.us/wiki/cost-spike-pi-durable/

> **Status:** FINDINGS of Kanbus `chatticus-9e7710` (epic `chatticus-8c857b`), measured against `@earendil-works/pi-durable`
> 1.0.2 and `@earendil-works/pi-ai` as installed at the repo root, on the production `IndexedStorage` (DynamoDB plus S3 on moto).
> `docs/COST_ACCOUNTING.md` (PR 533) is not on `develop` yet, so these findings live here. Fold this file into section 3.2
> and 3.3 of that document when it lands, then delete this file. Nothing here changes production behavior.
>
> Evidence is the vitest `conversation/test/pi-durable-cost-contract.test.ts` (14 tests, all passing). It is kept as a guard
> of the library contract the design relies on: if a pi-durable upgrade changes any of these facts, the test fails
> and the design must be re-read. Items labelled INFERRED were read from library code and not run.

## The question

Can a per-model-call cost record be written into the durable session in the same commit as the response, so cost rides with
the session across handoffs, crashes, retries and takeovers? The design (`COST_ACCOUNTING.md` section 3.2) assumed the
`afterResponse` hook of `GenerationHooks` could write a `defineDoc` document atomically with the response.

## Short answer

No. The `afterResponse` hook cannot join the response commit. It runs before the commit, with a read-only API, so anything it
writes is a separate commit. That is not a blocker: the response commit already contains everything a cost record needs
(the assistant entry carries usage and pi-ai's cost estimate, and pi-durable's own `pi.usage` document is updated in that same
commit). The right design is the fallback in the ticket: derive keyed records from the session at the existing finalize
points. It is idempotent by construction and needs no hook.

## Findings

### 1. Atomicity: the hook is outside the commit

* `GenerationHooks.afterResponse(message, api, context)` is declared to return `void | Promise<void>`, and `HookApi` extends
  `DocumentReader` (reads) plus `taskId`, `conversationId` and `memo`. There is no `tx` and no write method
  (`node_modules/@earendil-works/pi-durable/dist/harness/types.d.ts`).
* In `dist/harness/generation.js`, `classify()` calls `runtime.hooks.each("afterResponse", ...)` and only afterwards opens the
  commit that appends the assistant entry (`appendAssistant`, which also calls `recordUsage` on `pi.usage`). The hook is
  therefore strictly before the response commit.
* The object handed to the hook is the scheduler runtime, which at runtime does have a `commit` function that `HookApi` does not
  declare. The experiment calls it (the test casts the type away). The write works, but it is its own commit, and it relies
  on an undocumented, untyped surface. Commit log from the test "afterResponse has no transaction":
  the commit holding `doc:spike.cost-record#response-<id>` has a lower sequence than the commit holding
  `entry:pi.assistant`, and no commit contains both.
* What is atomic with the response: the assistant entry and `doc:pi.usage` appear in the same commit (test "pi.usage rides in the
  same commit as the assistant entry"). Failed responses go through the same `appendAssistant`, so they are entries and in
  `pi.usage` too. The cost-bearing record is the entry itself.
* Process death between the hook commit and the response commit (test "a death between the hook commit and the response
  commit", a storage wrapper that refuses the commit containing the assistant entry and every commit after it): the keyed record
  survives, the assistant entry does not exist, and `pi.usage` is still empty. So a hook-written record is not "with the
  response"; it can exist for a response the session never learned about.

### 2. Idempotency under retry and takeover

* After that death, a new owner with a newer fence resumes the same submission. The generation task is still in its request
  phase, so it asks the model again. This is a second paid call, with a second response and a second hook call
  (test "after the death a new fence re-requests the model"). With distinct provider response ids the hook records two
  keyed documents, which is correct (two calls were paid for, the first answer was lost). With the same response id (what a
  re-poll of one deferred response would show) the keyed write is a no-op and there is one record. The final session holds one
  assistant entry and `pi.usage` counts only that one call: the first call exists only in the hook record.
* A stale owner cannot write after a takeover: a commit from the first owner after the second claimed the partition is rejected
  with `OwnershipLost`, and no record appears (test "a stale owner cannot write a record after a takeover").
* The fallback is idempotent without any hook: `deriveCostRecordsFromEntries` (in the test) scans assistant entries and writes a
  family document keyed `entry-<entryId>`, skipping filled ones, in one commit. Run on the first owner, then twice on the
  second owner with a new fence: 2 written, then 0, then 0 (test "the fallback derives keyed records from entries"). A crash after
  the response commit and before any record write loses nothing, because the new owner derives the same records from the
  entries (test "a crash after the response commit").
* Residual loss window, for any session-side design: a crash after the provider answered and before the response commit (or,
  for the hook design, before the hook). The call was paid and the session never sees it. Only an observation made outside the
  session (the gateway, ticket T5) covers that.

### 3. Readable by key, and carried with the session

* A `defineDocFamily` keyed document (scope conversation, history latest) is read with `session.snapshot(token, conversationId,
  key, context)` and listed with `storage.scanDocuments({ scope, at: "current", kind })`, from a reader built with
  `createSession(await IndexedStorage.open({ ...no fence }))`. No Harness is opened, no task runs, no entries are replayed
  (test "readable by key and by listing through a reader session that replays nothing"). `pi.usage` is read the same way.
* A different owner opening the partition sees the same records with no rebuild (test "a session opened by a different owner").
* Forks (matters for T6): with `fork: "initial"` a forked conversation starts with no records, and with `fork: "current"` it copies all
  of the parent's. A cost document must use `fork: "initial"`, otherwise summing across conversations counts the parent's
  history twice. `pi.usage` already uses `fork: "initial"`, so a fork's `pi.usage` is empty (test "a fork starts empty").
  Note that `storage.scanEntries({ conversationId: fork })` returns the parent's entries inherited by the fork, so derivation
  must filter `entry.conversationId === conversationId`.
* Sub-agents (matters for T6): a child created in a tool with `ownership: { kind: "task", taskId }` is found by
  `scanConversations({ ownerTaskId })` (index `CT#`), by `scanConversations({ ownerConversationId })` (index `CC#`), and by
  `scanConversations({})` (all conversations of the partition). The child has its own `pi.usage` and its own cost documents, and
  `harness.usage()` sums root and child (test "task-owned child conversations enumerate"). That sum is a ready cross-check
  for a turn total.
* Scope note: a session partition is per bot and channel (`PI#{tenant}#{bot}#{channel}`). Records ride with that
  partition. A channel with several bots spans several partitions, so a channel total cannot come from the session alone.

### 4. Is usage plus pi-ai's estimate enough to price with our own book?

Yes, with no new hook. Each assistant entry stores `usage.input` (uncached, because the OpenAI adapter subtracts cached and cache
write tokens from `input_tokens`), `cacheRead`, `cacheWrite`, `output` (reasoning is a subset of output, with `reasoning` carried
separately), `totalTokens`, and `cost` from pi-ai's catalog, plus `provider`, `model`, `responseModel`, `responseId` and
`stopReason`. The context size for tier selection is `input + cacheRead + cacheWrite`. The test recomputes dollars from a
separate price book (base rates plus a tier above 272,000 context tokens) and matches the stored estimate to 12 decimal places
for a normal and a long-context call. So the cost record can be a pure function of (stored usage, our price book): store the
raw counters, the price book id and our computed dollars, and keep pi-ai's `cost.total` only as the cross-check the plan wants.

What the stored response does not carry:

* **Compaction summarization.** `CompactionTask` calls the model and records the usage into `pi.usage`
  (`dist/harness/compaction.js`, `recordUsage`) but appends no assistant entry, and compaction has no `afterResponse` hook. In the test the
  summary call (900 input, 60 output) is in `harness.usage()` and absent from the entries, and the hook was called only for the two
  generation responses. It can only be recovered as `pi.usage` totals minus the sum over entries, per conversation and model.
* **Errored responses with usage** are entries and in `pi.usage` (test "an errored response with usage"). **Aborted partial
  responses** are converted into an aborted assistant entry by `convertPartial` (INFERRED from the code, not run); an attempt that
  produced no partial has no usage anywhere.
* **Service tier pricing** (`applyServiceTierPricing`, flex or priority multipliers in the OpenAI adapter) changes
  the stored `cost` but the message carries no service tier field, so our own book cannot reproduce it from the entry alone
  (INFERRED, not run). Chatticus does not request a service tier today; if it ever does, record the tier alongside the call.
* `pi.usage` itself is bucketed by `provider/model` per conversation, not per call, so it is a total to reconcile against, not a
  per-call source.

## Recommendation

**Do not build on the `afterResponse` hook. Use the fallback: idempotent keyed derivation at the existing finalize points.**

1. The cost record is derived from the assistant entries of a conversation, keyed `entry-<entryId>` (family document, scope
   conversation, history latest, `fork: "initial"`), written in one commit per derivation, skipping filled keys. It is
   atomic with nothing extra to prove, because its source (the entry and `pi.usage`) is already in the response commit.
2. Derive at conclude, fail, yield and park (the safety-net points already in section 3.3), and on takeover before the new owner's
   first model call. Each run is a no-op for records already written, so repeat, retry and takeover are free.
3. Price at record time from the versioned book and store `priceBookId`, tier, raw counters and our dollars on the record. Keep pi-ai's
   estimate as a field for the cross-check.
4. Account for compaction as a per-conversation remainder: `pi.usage` total minus the sum of entry usage, recomputed (not
   accumulated) at the same points, written under a key such as `compaction-remainder-<conversationId>-<turnId>`.
5. Keep the hook out of the design. It adds only one thing the derivation does not: a record for a response whose commit then
   failed. That window is narrow (hook to commit) and is already the job of the gateway observation (T5), which also covers the
   wider window before the hook. If the owner ever wants the hook as an optimisation, it must use the undocumented runtime `commit`
   and a key derived from `responseId`, and the derivation must still run, so it buys nothing.
6. Channel total (ticket item 4, a design decision rather than a measurement): sessions are per bot and channel, so the channel
   total belongs in the messaging table. Use a sibling item `pk = {tenant}#cost#channel#{channelId}` updated in the same transaction as the
   turn index item (idempotent by turn id), not a GSI: one household, a handful of channels, and a GSI adds an index to the table
   for a read that a single key serves.

## Effect on the plan and tickets

* `COST_ACCOUNTING.md` section 3.2: replace the ASSUMPTION about `afterResponse` with "resolved, no; fallback adopted". Section 3.3: strike
  "each `afterResponse` (preferred)" and keep conclude, fail, yield, park, plus takeover.
* **T4** (call records in the session): implement `deriveCostRecordsFromEntries` as the single writer; family document
  `chatticus.cost`, key `entry-<entryId>`, `fork: "initial"`, entry filter on `conversationId`. No hook, no use of the runtime commit. Retire delta
  counters as planned. Add the takeover case (derive before the first model call of a new owner).
* **T5** (complete metering): compaction comes from the `pi.usage` remainder, not from a hook. Aborted and errored responses with usage are
  already entries. Calls that never reached a commit are only reachable through the gateway observation, which therefore stays in scope as the
  only cover for that window, and reconciliation should treat a small positive observed-minus-session gap as expected.
* **T6** (sub-agent and fork accounting): enumerate with `scanConversations({})` per partition, `{ ownerTaskId }` for one task and
  `{ ownerConversationId }` for the children of a conversation; `fork: "initial"` on the cost document; filter inherited entries on a fork; use
  `harness.usage()` (which sums children) as the cross-check for the turn total.
* **T3** (price book): add a service tier field to the record if a service tier is ever requested.