We Charged 900 Credits for a Video That Didn't Exist

TL;DR: If you bill for AI work by the unit, you eventually write a rule like "we charge when the vendor already charged us." It's the intuitive rule and it's wrong, because vendor spend isn't one thing. Speech synthesis is an input; the avatar render is the deliverable. Bill on the first and you charge for a video that doesn't exist. Ours did, once, for 900 credits.

Bill by the unit and you eventually face an uncomfortable question: the run failed, we already paid a vendor, who eats it?

The tempting answer is symmetry. We paid, so you pay. It feels fair, it's easy to explain, and you can implement it in an afternoon by checking a boolean.

We shipped that rule on Clipwright(opens in new tab), our video generation API. It survived review, it read as obviously correct, and then a live run charged a customer 900 credits for a video that was never produced. The rule wasn't buggy: the implementation did exactly what the rule said. The rule was the defect.

"Did we pay the vendor?" is the wrong question

A Clipwright run turns a script into a talking actor. Two vendor calls matter: ElevenLabs(opens in new tab) synthesizes the speech, HeyGen(opens in new tab) creates the avatar that speaks it. Both cost money the moment they're invoked.

The old rule read, roughly: failed run after a paid vendor call → charge. Now walk a run that synthesizes speech and then dies before the avatar stage. Did we pay a vendor? Yes. Does the customer have a video? No. They have nothing, and under that rule they'd be billed for it.

That's not an edge case you can put in a backlog. It's the most common failure shape, because the avatar step is slower, heavier, and likelier to fail than the text-to-speech step that precedes it.

The mistake wasn't arithmetic. It was collapsing a set into a boolean. "Have we spent money" flattened two very different kinds of spend into one bit, and the bit had no idea that one of them produces something the customer bought and the other doesn't.

The fix is a table, not a condition

What separates the outcomes isn't whether we paid. It's what kind of work we paid for:

1export const VENDOR_WORK_ROLE: Record<VendorOperationKind, VendorWorkRole> = {
2  speech: "input",
3  avatar_create: "deliverable",
4};

Speech is an input, a raw material we consume on the way to making the thing. The avatar render is the deliverable, the thing itself. The customer is buying a video, not the pipeline that produces one.

Two properties make this hold up better than a condition would.

The table is total. It's a Record over every vendor operation kind, so adding a new vendor call without classifying it doesn't compile. Same trick we use for our input field dispositions: the compiler refuses to let a decision go unmade.

The derived list comes from the table. The set of deliverable-producing kinds is computed by filtering the table, not maintained beside it. A second hand-written list would drift from the first, and the drift would be invisible until it cost money again.

Four outcomes

With work roles in place, settlement has four branches rather than two:

Our side is checked first, and the ordering is deliberate: the spend guard can trip after speech is already paid for. If you check "did any vendor bill us" before "was this our fault," a run we killed ourselves bills the customer.

So the partial-success case (speech succeeded, avatar failed) settles at zero. The synthesis is our cost, not theirs. They bought a video and there isn't one.

The opposite rule is also wrong

Here's where it stops being a story about being generous. "Full refund on any failure" is the rule a sympathetic reader arrives at, and we rejected it too, with the reasoning recorded next to the code: it makes launch-and-cancel free.

That's not a fraud scenario. It's the default behavior of a retrying agent. An autonomous caller that starts a run, doesn't like the wait, abandons it, and starts another isn't attacking you. It's doing what retry logic does. If abandonment is free, an agent loop can generate unbounded vendor spend on your account without a single malicious actor.

That's the real shape of the problem: you're not choosing between "fair to the customer" and "fair to us." You're choosing a rule that a machine will execute thousands of times without ever reading your terms of service.

Two witnesses, because one lies

One more piece worth stealing. "Was the vendor paid?" is answered by two independent sources: a row in our vendor_operations table, and, if that row is missing, payment markers in object storage.

Two witnesses exist because a single source has a failure mode that's silent. If the process dies between calling the vendor and writing the row, the database says "never paid" and the vendor says otherwise. You find out at month end, from an invoice.

And both witnesses answer with a set of work kinds, not a yes. Collapsing that set into a boolean is the exact mechanism that produced the 900-credit charge. The fix wasn't only new logic; it was refusing to let the answer be one bit wide.

Keep the policy readable

The settlement function is pure: it takes the run state, the billed work kinds, and the measured duration, and returns a decision. It doesn't touch the database. Persistence happens at the transaction boundary, separately.

This isn't architectural purity for its own sake. Billing policy is the code your team will argue about in a year, when a customer disputes a charge and someone has to say what should have happened. A pure function is a table of examples you can read and test. The same policy woven through transaction handling is something you have to simulate in your head to review, and the whole reason we're here is that the last version read as obviously correct.

What our numbers actually are

Since a post about billing that hides its prices is a bit rich: 1000 credits is $10, a finished second of video costs 30 credits, so $0.30 per finished second. The trial is 300 credits, ten seconds. Those numbers are on the landing page, and they're derived from the same constants the code bills with rather than typed into the marketing copy separately.

What I won't give you is a failure rate. The product is in invite-only beta; the number of paid runs is small enough that any percentage I quoted would be noise wearing a lab coat. And unit cost and margin are internal.

If you're building metered AI billing

1. List every vendor call and label it: input or deliverable. If the answer is "depends," you've found the bug already.
2. Make the classification total. A Record over your operation kinds means a new vendor call can't ship unclassified.
3. Check your own faults first. Whatever ordering you pick, a run you killed must never bill the customer, even when the vendor already charged you.
4. Price the abandon path. Free cancellation plus a retrying agent equals unbounded spend on your account.
5. Never let "did we pay" be a boolean. It's a set of work kinds, and the difference between them is the whole policy.

Pull up your own billing code and find the line where a failed run decides whether to charge. If that line reads a single boolean, you have the bug we had. It'll cost you the first time a run dies in the middle.

Clipwright is in invite-only beta. The settlement policy described here is what runs in production today.

FAQ

What happens if speech succeeds but the avatar fails?

Zero charge. Speech is an input, not a deliverable. The customer bought a video; there isn't one, so the synthesis cost stays ours.

Why not refund every failed run?

Because it makes launch-and-cancel free, and an agent that retries aggressively reproduces that pattern without meaning to. Unbounded vendor spend on our account is the outcome, and no malicious user is required.

Why two sources for "was the vendor paid"?

A process can die between calling the vendor and recording the call. The database would say "never paid" while the invoice says otherwise. The second witness, payment markers in storage, closes that window.

Sources

• Clipwright(opens in new tab), the API described here
• Your agent isn't confused. Your schema lied to it., the disposition registry using the same totality trick
• Why multi-step agents compound failure