Scenario · 70B parameters · Full fine-tune

70B full fine-tune on 1 billion tokens: cost estimate

70 billion parameters, 1 billion tokens, full fine-tune. The lowest estimate across 11 GPU entries is $345 on Vultr with MI355X, 5.6 days of GPU time at 35% MFU. An estimate, not a quote.

Step by step

The arithmetic

1. Training FLOPs. 6 × 70 × 10^9 parameters × 1 billion tokens = 4.20 × 10^20 FLOPs. About 2 x parameters x tokens for the forward pass plus about 4 x for the backward pass, because the backward pass computes gradients for activations and for every weight.
2. Useful throughput per GPU. Dense BF16 TFLOPS × 10^12 × 35% MFU. The table gives it for each GPU.
3. GPU-hours. FLOPs ÷ throughput ÷ 3,600. For L40S (362.05 TFLOPS): 4.20 × 10^20 ÷ 1.27 × 10^14 ÷ 3,600 = 921 GPU-hours.
4. Wall-clock. GPU-hours ÷ 8 GPUs, assuming the work spreads evenly.
5. Cost. GPU-hours × each provider's lowest listed on-demand price per GPU-hour for that GPU.

Result

Estimated cost by GPU

GPU	Dense BF16 TFLOPS	GPU-hours	8 GPUs	Providers	Lowest	Median	Highest
MI355X	2500	133	17 h	2	$345Vultr	$746	$1,147Oracle Cloud
MI300X	1307.4	255	32 h	4	$472Vultr	$770	$1,530Oracle Cloud
MI325X	1307.4	255	32 h	2	$510Vultr	$739	$969DigitalOcean (Paperspace)
H100 SXMpreliminary	1000	333	42 h	6	$983Thunder Compute	$1,057	$1,696Verda
GH200	990	337	42 h	2	$670Vultr	$1,429	$2,189CoreWeave
H100 PCIepreliminary	800	417	2.2 days	2	$1,042Hyperstack	$1,123	$1,204RunPod
L40S	362.05	921	4.8 days	14	$506Runcrate	$1,413	$3,222Oracle Cloud
A100 80GB	312	1,068	5.6 days	12	$1,165Thunder Compute	$2,025	$3,665Amazon Web Services
L40	181.05	1,841	9.6 days	6	$1,454Thunder Compute	$1,712	$2,320Runcrate
A40	149.7	2,227	11.6 days	3	$1,314RunPod	$3,812	$5,923Runcrate
A10	125	2,667	13.9 days	4	$2,667OVHcloud	$5,133	$8,533Microsoft Azure

Prices are each provider's lowest listed on-demand rate per GPU-hour, read 12 November 2026. Only GPUs whose maker prints a dense BF16 figure appear; the coverage page says which are out and why.

Sensitivity

What if MFU were different? (L40S)

MFU assumed	GPU-hours	8 GPUs	Lowest cost
20%	1,611	8.4 days	$886
35% (default)	921	4.8 days	$506
50%	644	3.4 days	$354

The 20% and 50% rows are what-if inputs chosen to show sensitivity, not measurements. Lowest cost uses Runcrate at $0.55 per GPU-hour.

Try it

Change the assumptions

Model parameters (billions)

Training tokens (millions)

Method

FLOPs per parameter per token

GPU (dense BF16 printed by the maker)

Number of GPUs

MFU, percent of peak achieved

Estimate, step by step

1. Training FLOPs = 6 × 70 billion parameters × 1000 million tokens = 4.20 × 10^20
2. Useful FLOP/s per GPU = 362.05 TFLOPS × 35% MFU = 1.27 × 10^14
3. GPU-hours = step 1 ÷ step 2 ÷ 3,600 = 920.7
4. Wall-clock on 8 GPUs = step 3 ÷ 8 = 4.8 days, assuming the work spreads evenly
5. Cost = step 3 × each provider's lowest listed price per GPU-hour

Provider	Billing	$/GPU-hr	Estimated cost
Runcrateours, live	per second	$0.55	$506
CloudRift	per second	$0.69	$635
Massed Compute	billing not stated	$0.97	$893
RunPod	per second	$1.09	$1,004
Koyeb	per second	$1.20	$1,105
Civo	per hour	$1.29	$1,188
Crusoe	per hour	$1.50	$1,381
DigitalOcean (Paperspace)	per second	$1.57	$1,445
Vultr	per hour	$1.67	$1,538
OVHcloud	per hour	$1.80	$1,657
Amazon Web Services	per second	$1.86	$1,713
Verda	billing not stated	$2.00	$1,840
CoreWeave	billing not stated	$2.25	$2,072
Oracle Cloud	billing not stated	$3.50	$3,222

Range $506 to $3,222, median $1,413 across 14 providers. An estimate, not a quote: real runs differ in utilisation, memory fit, communication and restarts, and providers that bill per hour round up.

Questions

70B full fine-tune on 1 billion tokens: common questions

How many FLOPs does this take?

About 4.20 × 10^20: 6 × 70 billion parameters × 1 billion tokens. About 2 x parameters x tokens for the forward pass plus about 4 x for the backward pass, because the backward pass computes gradients for activations and for every weight.

What does it cost at the low end?

$345, on Vultr with MI355X at $2.59 per GPU-hour, for 133 GPU-hours.

Is this a quote?

No. It is an estimate from a stated formula and published list prices. Real runs differ in utilisation, memory fit and restarts, and the MFU is an assumption.

Why are some GPUs missing?

A GPU needs a dense BF16 figure printed by its maker. Those that print only a sparse figure are excluded rather than halved.

Fine-Tune Cost Lab is published by the Runcrate team, who also run Runcrate, an AI cloud. Runcrate is listed in the table, marked "ours, live". It is sorted by price like every other row, with no pinned position. Other providers' prices are read from their own public pricing pages. We have no paid placement.