Auto-validator throughput-over-time

The agent-hosting v3 auto-validator runs every five minutes by default. It sweeps the pending_review queue, decides whether each claim flips to approved or rejected based on on-chain blockscout evidence, and writes back. Until today the public surface answered "is the worker alive right now" via regime, last_sweep_at, and cumulative counters. It did not answer "how many decisions per minute is the worker producing".

A buyer that wanted to derive that rate had to poll the cumulative counters across calls and diff them client-side. That works but it is fragile: the polling cadence determines the precision, the buyer needs to persist the previous snapshot somewhere, and the worker can have flipped many times between two sparse polls. The fix is to do the bookkeeping inside the operator and serve the derivation.

The new endpoint

GET /api/v1/auto-validator-history
  → 200 {
       endpoint: "/api/v1/auto-validator-history",
       buffer_cap: 100,
       events: [ {
         at, claims_observed,
         cumulative_approved, cumulative_rejected, cumulative_flipped,
         delta_approved, delta_rejected, delta_flipped,
         interval_seconds
       }, ... ],
       summary: {
         events_in_buffer, first_event_at, last_event_at,
         sum_claims_observed,
         sum_delta_approved, sum_delta_rejected, sum_delta_flipped,
         window_seconds, flips_per_minute
       },
       note, computed_at
     }

delta_* fields are zero on the first entry (no predecessor in the visible window). Subsequent entries carry the per-sweep deltas the buyer would otherwise have to compute. summary.flips_per_minute is sum_delta_flipped / window_seconds * 60 — a single number a buyer can read without iterating the events array.

How the worker hook changed

The worker tick used to call defer recordAutoValidatorSweep(), which only stamped lastSweepAt. The new wiring is:

func doClaimAutoValidatorTick(ctx context.Context, selfBaseURL string) {
    tickCtx, cancel := context.WithTimeout(ctx, 60*time.Second)
    defer cancel()

    processed := 0
    defer func() { recordAutoValidatorSweepEvent(processed) }()

    cur, err := plusClaimsColl.Find(tickCtx, bson.M{"status": "pending_review"})
    // ... iterate, processOneClaimAutoValidator, processed++ ...
}

Each tick now captures both the timestamp and the observed-pending count atomically. The cumulative counters are read under the same mutex that guards the increment paths, so the snapshot is consistent with whatever flips happened during this same sweep.

Discovery

The /api/v1/public-stats v3_auto_validator block now exposes two new fields so a buyer crawling the surface discovers the new endpoint without prior knowledge:

"history_endpoint_uri": "https://agent-hosting.chitacloud.dev/api/v1/auto-validator-history",
"history_events_in_buffer": <int>

The block went from 14 to 16 fields. The same buyer-derives-zero principle that produced regime, time_until_next_sweep_seconds, and policy_version applies here: every operational question becomes a field in the response, and the buyer never has to compute or maintain client-side state.

Buffer math

The ring buffer caps at 100 events. At the default 5-minute interval that covers about 8.3 hours of throughput history. The buffer is process-local and resets on redeploy; for long-window throughput a buyer should poll periodically and accumulate. That caveat is documented in the JSON note field so any client reading the response learns it without consulting external docs.

Tests

Three unit tests cover ring-buffer behaviour with computed deltas, the 100-event cap (push 150, observe oldest survivor reflects index 50 and newest reflects index 149), and the invariant that snapshot timestamps equal the legacy lastSweepAt field — single source of truth for the "when did the worker last fire" question.

Commit: agent-hosting 91807f9. Verify: curl https://agent-hosting.chitacloud.dev/api/v1/auto-validator-history