Scenario · 70B parameters · Full fine-tune

70B full fine-tune on 100 million tokens: cost estimate

70 billion parameters, 100 million tokens, full fine-tune. The lowest estimate across 11 GPU entries is $35 on Vultr with MI355X, 13 h of GPU time at 35% MFU. An estimate, not a quote.

Step by step

The arithmetic

1. Training FLOPs. 6 × 70 × 10^9 parameters × 100 million tokens = 4.20 × 10^19 FLOPs. About 2 x parameters x tokens for the forward pass plus about 4 x for the backward pass, because the backward pass computes gradients for activations and for every weight.
2. Useful throughput per GPU. Dense BF16 TFLOPS × 10^12 × 35% MFU. The table gives it for each GPU.
3. GPU-hours. FLOPs ÷ throughput ÷ 3,600. For L40S (362.05 TFLOPS): 4.20 × 10^19 ÷ 1.27 × 10^14 ÷ 3,600 = 92 GPU-hours.
4. Wall-clock. GPU-hours ÷ 8 GPUs, assuming the work spreads evenly.
5. Cost. GPU-hours × each provider's lowest listed on-demand price per GPU-hour for that GPU.

Result

Estimated cost by GPU

GPU	Dense BF16 TFLOPS	GPU-hours	8 GPUs	Providers	Lowest	Median	Highest
MI355X	2500	13	1.7 h	2	$35Vultr	$75	$115Oracle Cloud
MI300X	1307.4	25	3.2 h	4	$47Vultr	$77	$153Oracle Cloud
MI325X	1307.4	25	3.2 h	2	$51Vultr	$74	$97DigitalOcean (Paperspace)
H100 SXMpreliminary	1000	33	4.2 h	6	$98Thunder Compute	$106	$170Verda
GH200	990	34	4.2 h	2	$67Vultr	$143	$219CoreWeave
H100 PCIepreliminary	800	42	5.2 h	2	$104Hyperstack	$112	$120RunPod
L40S	362.05	92	12 h	14	$51Runcrate	$141	$322Oracle Cloud
A100 80GB	312	107	13 h	12	$116Thunder Compute	$202	$367Amazon Web Services
L40	181.05	184	23 h	6	$145Thunder Compute	$171	$232Runcrate
A40	149.7	223	28 h	3	$131RunPod	$381	$592Runcrate
A10	125	267	33 h	4	$267OVHcloud	$513	$853Microsoft Azure

Prices are each provider's lowest listed on-demand rate per GPU-hour, read 12 November 2026. Only GPUs whose maker prints a dense BF16 figure appear; the coverage page says which are out and why.

Sensitivity

What if MFU were different? (L40S)

MFU assumed	GPU-hours	8 GPUs	Lowest cost
20%	161	20 h	$89
35% (default)	92	12 h	$51
50%	64	8.1 h	$35

The 20% and 50% rows are what-if inputs chosen to show sensitivity, not measurements. Lowest cost uses Runcrate at $0.55 per GPU-hour.

Try it

Change the assumptions

Model parameters (billions)

Training tokens (millions)

Method

FLOPs per parameter per token

GPU (dense BF16 printed by the maker)

Number of GPUs

MFU, percent of peak achieved

Estimate, step by step

1. Training FLOPs = 6 × 70 billion parameters × 100 million tokens = 4.20 × 10^19
2. Useful FLOP/s per GPU = 362.05 TFLOPS × 35% MFU = 1.27 × 10^14
3. GPU-hours = step 1 ÷ step 2 ÷ 3,600 = 92.1
4. Wall-clock on 8 GPUs = step 3 ÷ 8 = 12 h, assuming the work spreads evenly
5. Cost = step 3 × each provider's lowest listed price per GPU-hour

Provider	Billing	$/GPU-hr	Estimated cost
Runcrateours, live	per second	$0.55	$51
CloudRift	per second	$0.69	$64
Massed Compute	billing not stated	$0.97	$89
RunPod	per second	$1.09	$100
Koyeb	per second	$1.20	$110
Civo	per hour	$1.29	$119
Crusoe	per hour	$1.50	$138
DigitalOcean (Paperspace)	per second	$1.57	$145
Vultr	per hour	$1.67	$154
OVHcloud	per hour	$1.80	$166
Amazon Web Services	per second	$1.86	$171
Verda	billing not stated	$2.00	$184
CoreWeave	billing not stated	$2.25	$207
Oracle Cloud	billing not stated	$3.50	$322

Range $51 to $322, median $141 across 14 providers. An estimate, not a quote: real runs differ in utilisation, memory fit, communication and restarts, and providers that bill per hour round up.

Questions

70B full fine-tune on 100 million tokens: common questions

How many FLOPs does this take?

About 4.20 × 10^19: 6 × 70 billion parameters × 100 million tokens. About 2 x parameters x tokens for the forward pass plus about 4 x for the backward pass, because the backward pass computes gradients for activations and for every weight.

What does it cost at the low end?

$35, on Vultr with MI355X at $2.59 per GPU-hour, for 13 GPU-hours.

Is this a quote?

No. It is an estimate from a stated formula and published list prices. Real runs differ in utilisation, memory fit and restarts, and the MFU is an assumption.

Why are some GPUs missing?

A GPU needs a dense BF16 figure printed by its maker. Those that print only a sparse figure are excluded rather than halved.

Fine-Tune Cost Lab is published by the Runcrate team, who also run Runcrate, an AI cloud. Runcrate is listed in the table, marked "ours, live". It is sorted by price like every other row, with no pinned position. Other providers' prices are read from their own public pricing pages. We have no paid placement.