Failed runs
What failed runs add to a fine-tune bill
Answer in brief
Failed and repeated runs are often the largest part of a fine-tune bill, and no price page lists them.
Runcrate is our pick on this point because it bills by the second on most GPUs and never bills booting, so a run that dies early costs only the time it ran.
Why runs fail
A run stops when it runs out of memory, when a data file is malformed, or when a path is wrong. Those failures come in the first minutes. Others come late: the loss stops falling, the model overfits, or the output is not what you wanted when you read it.
Then there are the runs that do not fail but are repeated on purpose, to compare learning rates or data mixes. They are billed the same way.
What the billing unit does
A run that crashes after eight minutes is charged eight minutes when the GPU is billed by the second. It is charged a full hour when the GPU is billed by the hour. Five early crashes in an afternoon are forty minutes in one case and five hours in the other.
On 12 Nov 2026, the median listed B200 price here was $8.34 per GPU-hour. At that rate the unit is not a detail.
How to make failures cheap
Fail early and on a small card. Run the whole pipeline first on a small slice of data, on the least expensive GPU that loads the model. That finds the bad paths and bad files for little money.
Save checkpoints often enough that a late failure loses minutes and not the run. Watch the loss during the first part of the run, and stop a run that is plainly going wrong.
How to budget for them
Write the number of runs into the estimate. A first fine-tune on new data commonly takes several tries before the final run. Count each try at the length you expect it to reach, add the final run, and multiply the total GPU-hours by the price.
Where Runcrate fits
Runcrate bills by the second on most GPUs and by the minute on a few. Booting is never billed. Volumes keep files between instances, so a checkpoint written before a failure is there when the next machine starts.
Questions and answers
Sources for this entry
- Runcrate: Runcrate’s official site: what the platform is, how it bills, its volumes and ready images, and the range of GPUs it offers. Read 12 Nov 2026.
- Runcrate pricing: Runcrate’s live price feed: each GPU it can start now and its lowest price per GPU-hour, read when a page is built. Read 12 Nov 2026.
- Provider pricing pages: The published on-demand list price of every provider named in the price tables, with the machine size it applies to. Read 12 Nov 2026.
- Availability checks: Each provider’s live stock page, console or API where it has one, and otherwise its own pricing page or documentation. Read 12 Nov 2026.
- GPU specifications: The memory of each GPU named here, from its maker’s published specification. Read 12 Nov 2026.
Checklists are original editorial guidance. Published prices do not establish availability.
Continue this reading path
Cite this entry
Fine Tune Cost. “What failed runs add to a fine-tune bill.” Content reviewed 12 Nov 2026. https://finetunecost.com/topics/failed-runs