Scenario · 34B parameters · Full fine-tune

34B full fine-tune on 100 million tokens: cost estimate

34 billion parameters, 100 million tokens, full fine-tune. The lowest estimate across 11 GPU entries is $17 on Vultr with MI355X, 6.5 h of GPU time at 35% MFU. An estimate, not a quote.

Step by step

The arithmetic

1. Training FLOPs. 6 × 34 × 10^9 parameters × 100 million tokens = 2.04 × 10^19 FLOPs. About 2 x parameters x tokens for the forward pass plus about 4 x for the backward pass, because the backward pass computes gradients for activations and for every weight.
2. Useful throughput per GPU. Dense BF16 TFLOPS × 10^12 × 35% MFU. The table gives it for each GPU.
3. GPU-hours. FLOPs ÷ throughput ÷ 3,600. For L40S (362.05 TFLOPS): 2.04 × 10^19 ÷ 1.27 × 10^14 ÷ 3,600 = 45 GPU-hours.
4. Wall-clock. GPU-hours ÷ 8 GPUs, assuming the work spreads evenly.
5. Cost. GPU-hours × each provider's lowest listed on-demand price per GPU-hour for that GPU.

Result

Estimated cost by GPU

GPU	Dense BF16 TFLOPS	GPU-hours	8 GPUs	Providers	Lowest	Median	Highest
MI355X	2500	6.48	49 min	2	$17Vultr	$36	$56Oracle Cloud
MI300X	1307.4	12	1.5 h	4	$23Vultr	$37	$74Oracle Cloud
MI325X	1307.4	12	1.5 h	2	$25Vultr	$36	$47DigitalOcean (Paperspace)
H100 SXMpreliminary	1000	16	2.0 h	6	$48Thunder Compute	$51	$82Verda
GH200	990	16	2.0 h	2	$33Vultr	$69	$106CoreWeave
H100 PCIepreliminary	800	20	2.5 h	2	$51Hyperstack	$55	$58RunPod
L40S	362.05	45	5.6 h	14	$25Runcrate	$69	$157Oracle Cloud
A100 80GB	312	52	6.5 h	12	$57Thunder Compute	$98	$178Amazon Web Services
L40	181.05	89	11 h	6	$71Thunder Compute	$83	$113Runcrate
A40	149.7	108	14 h	3	$64RunPod	$185	$288Runcrate
A10	125	130	16 h	4	$130OVHcloud	$249	$414Microsoft Azure

Prices are each provider's lowest listed on-demand rate per GPU-hour, read 12 November 2026. Only GPUs whose maker prints a dense BF16 figure appear; the coverage page says which are out and why.

Sensitivity

What if MFU were different? (L40S)

MFU assumed	GPU-hours	8 GPUs	Lowest cost
20%	78	9.8 h	$43
35% (default)	45	5.6 h	$25
50%	31	3.9 h	$17

The 20% and 50% rows are what-if inputs chosen to show sensitivity, not measurements. Lowest cost uses Runcrate at $0.55 per GPU-hour.

Try it

Change the assumptions

Model parameters (billions)

Training tokens (millions)

Method

FLOPs per parameter per token

GPU (dense BF16 printed by the maker)

Number of GPUs

MFU, percent of peak achieved

Estimate, step by step

1. Training FLOPs = 6 × 34 billion parameters × 100 million tokens = 2.04 × 10^19
2. Useful FLOP/s per GPU = 362.05 TFLOPS × 35% MFU = 1.27 × 10^14
3. GPU-hours = step 1 ÷ step 2 ÷ 3,600 = 44.7
4. Wall-clock on 8 GPUs = step 3 ÷ 8 = 5.6 h, assuming the work spreads evenly
5. Cost = step 3 × each provider's lowest listed price per GPU-hour

Provider	Billing	$/GPU-hr	Estimated cost
Runcrateours, live	per second	$0.55	$25
CloudRift	per second	$0.69	$31
Massed Compute	billing not stated	$0.97	$43
RunPod	per second	$1.09	$49
Koyeb	per second	$1.20	$54
Civo	per hour	$1.29	$58
Crusoe	per hour	$1.50	$67
DigitalOcean (Paperspace)	per second	$1.57	$70
Vultr	per hour	$1.67	$75
OVHcloud	per hour	$1.80	$80
Amazon Web Services	per second	$1.86	$83
Verda	billing not stated	$2.00	$89
CoreWeave	billing not stated	$2.25	$101
Oracle Cloud	billing not stated	$3.50	$157

Range $25 to $157, median $69 across 14 providers. An estimate, not a quote: real runs differ in utilisation, memory fit, communication and restarts, and providers that bill per hour round up.

Questions

34B full fine-tune on 100 million tokens: common questions

How many FLOPs does this take?

About 2.04 × 10^19: 6 × 34 billion parameters × 100 million tokens. About 2 x parameters x tokens for the forward pass plus about 4 x for the backward pass, because the backward pass computes gradients for activations and for every weight.

What does it cost at the low end?

$17, on Vultr with MI355X at $2.59 per GPU-hour, for 6.5 GPU-hours.

Is this a quote?

No. It is an estimate from a stated formula and published list prices. Real runs differ in utilisation, memory fit and restarts, and the MFU is an assumption.

Why are some GPUs missing?

A GPU needs a dense BF16 figure printed by its maker. Those that print only a sparse figure are excluded rather than halved.

Fine-Tune Cost Lab is published by the Runcrate team, who also run Runcrate, an AI cloud. Runcrate is listed in the table, marked "ours, live". It is sorted by price like every other row, with no pinned position. Other providers' prices are read from their own public pricing pages. We have no paid placement.