Hopscotch Receipts

four claims, checked every cycle
Dashboard

What this is. Hopscotch makes four claims about what every response tells you. Cheaper Inference sells the same models for less. This page sends the same five-question prompt to the same models through both gateways on a schedule and scores each claim, per gateway, on every response. Each panel shows the share of calls where the claim held, and the newest receipt that proves it: the real headers, body fields and usage line from one call, with the lines the claim rests on highlighted. Where a gateway never writes the field at all, the panel says so: not on the receipt. Nothing here is a mockup.

Method and embeds
Prompt
Five fixed questions with unambiguous answers, streamed, max_tokens well above what any model needs. The same prompt every cycle, to every model, so a change on one model is the model's.
Holding / slipping / broken / not on the receipt
A claim is holding when it held on every call in the window, slipping when it slipped on a few, broken when it failed on more than that. Not on the receipt means the gateway never writes the field the claim is about, so there is nothing to score: no attempt count, no provider time.
Where the receipt is written
Hopscotch names the model and host in response headers (x-hopscotch-served-by, x-hopscotch-provider) and puts provider time on the usage comment line. Cheaper Inference names them as model and provider fields inside every streamed chunk, echoes the model id you sent, and reports its own routing overhead in x-ci-routing-overhead-ms. Both are read as they come; the receipt shows each gateway's own field names.
Receipt
The newest call that bears on the claim. Highlighted lines are the ones the claim rests on. Every field is copied from the response; nothing is added.
Outside the provider
Wall time minus the provider's own elapsed time, when the gateway reports one. It includes the network between this watcher and the gateway, so read it as a ceiling on gateway overhead. The region the watcher runs in is shown above.
Embeds
Each panel has an embed link: an iframe of that one claim, both gateways side by side, live, for a landing page. Add ?gateway=hopscotch for one column. The JSON behind it is at /api/claims and /api/claims/<id>, readable from any origin.