Scenario · 70B parameters · LoRA or QLoRA

70B LoRA fine-tune on 1 billion tokens: cost estimate

70 billion parameters, 1 billion tokens, lora or qlora. The lowest estimate across 11 GPU entries is $230 on Vultr with MI355X, 3.7 days of GPU time at 35% MFU. An estimate, not a quote.

Step by step

The arithmetic

1. Training FLOPs. 4 × 70 × 10^9 parameters × 1 billion tokens = 2.80 × 10^20 FLOPs. The forward pass costs about 2 x parameters x tokens and the backward pass about 2 x, because gradients for the frozen base weights are not computed. The small adapter matrices and QLoRA dequantisation are not counted.
2. Useful throughput per GPU. Dense BF16 TFLOPS × 10^12 × 35% MFU. The table gives it for each GPU.
3. GPU-hours. FLOPs ÷ throughput ÷ 3,600. For L40S (362.05 TFLOPS): 2.80 × 10^20 ÷ 1.27 × 10^14 ÷ 3,600 = 614 GPU-hours.
4. Wall-clock. GPU-hours ÷ 8 GPUs, assuming the work spreads evenly.
5. Cost. GPU-hours × each provider's lowest listed on-demand price per GPU-hour for that GPU.

Result

Estimated cost by GPU

GPU	Dense BF16 TFLOPS	GPU-hours	8 GPUs	Providers	Lowest	Median	Highest
MI355X	2500	89	11 h	2	$230Vultr	$497	$764Oracle Cloud
MI300X	1307.4	170	21 h	4	$314Vultr	$513	$1,020Oracle Cloud
MI325X	1307.4	170	21 h	2	$340Vultr	$493	$646DigitalOcean (Paperspace)
H100 SXMpreliminary	1000	222	28 h	6	$656Thunder Compute	$704	$1,130Verda
GH200	990	224	28 h	2	$447Vultr	$953	$1,459CoreWeave
H100 PCIepreliminary	800	278	35 h	2	$694Hyperstack	$749	$803RunPod
L40S	362.05	614	3.2 days	14	$338Runcrate	$942	$2,148Oracle Cloud
A100 80GB	312	712	3.7 days	12	$776Thunder Compute	$1,350	$2,444Amazon Web Services
L40	181.05	1,227	6.4 days	6	$970Thunder Compute	$1,141	$1,547Runcrate
A40	149.7	1,484	7.7 days	3	$876RunPod	$2,541	$3,949Runcrate
A10	125	1,778	9.3 days	4	$1,778OVHcloud	$3,422	$5,689Microsoft Azure

Prices are each provider's lowest listed on-demand rate per GPU-hour, read 12 November 2026. Only GPUs whose maker prints a dense BF16 figure appear; the coverage page says which are out and why.

Sensitivity

What if MFU were different? (L40S)

MFU assumed	GPU-hours	8 GPUs	Lowest cost
20%	1,074	5.6 days	$591
35% (default)	614	3.2 days	$338
50%	430	2.2 days	$236

The 20% and 50% rows are what-if inputs chosen to show sensitivity, not measurements. Lowest cost uses Runcrate at $0.55 per GPU-hour.

Try it

Change the assumptions

Model parameters (billions)

Training tokens (millions)

Method

FLOPs per parameter per token

GPU (dense BF16 printed by the maker)

Number of GPUs

MFU, percent of peak achieved

Estimate, step by step

1. Training FLOPs = 4 × 70 billion parameters × 1000 million tokens = 2.80 × 10^20
2. Useful FLOP/s per GPU = 362.05 TFLOPS × 35% MFU = 1.27 × 10^14
3. GPU-hours = step 1 ÷ step 2 ÷ 3,600 = 613.8
4. Wall-clock on 8 GPUs = step 3 ÷ 8 = 3.2 days, assuming the work spreads evenly
5. Cost = step 3 × each provider's lowest listed price per GPU-hour

Provider	Billing	$/GPU-hr	Estimated cost
Runcrateours, live	per second	$0.55	$338
CloudRift	per second	$0.69	$424
Massed Compute	billing not stated	$0.97	$595
RunPod	per second	$1.09	$669
Koyeb	per second	$1.20	$737
Civo	per hour	$1.29	$792
Crusoe	per hour	$1.50	$921
DigitalOcean (Paperspace)	per second	$1.57	$964
Vultr	per hour	$1.67	$1,026
OVHcloud	per hour	$1.80	$1,105
Amazon Web Services	per second	$1.86	$1,142
Verda	billing not stated	$2.00	$1,226
CoreWeave	billing not stated	$2.25	$1,381
Oracle Cloud	billing not stated	$3.50	$2,148

Range $338 to $2,148, median $942 across 14 providers. An estimate, not a quote: real runs differ in utilisation, memory fit, communication and restarts, and providers that bill per hour round up.

Questions

70B LoRA fine-tune on 1 billion tokens: common questions

How many FLOPs does this take?

About 2.80 × 10^20: 4 × 70 billion parameters × 1 billion tokens. The forward pass costs about 2 x parameters x tokens and the backward pass about 2 x, because gradients for the frozen base weights are not computed. The small adapter matrices and QLoRA dequantisation are not counted.

What does it cost at the low end?

$230, on Vultr with MI355X at $2.59 per GPU-hour, for 89 GPU-hours.

Is this a quote?

No. It is an estimate from a stated formula and published list prices. Real runs differ in utilisation, memory fit and restarts, and the MFU is an assumption.

Why are some GPUs missing?

A GPU needs a dense BF16 figure printed by its maker. Those that print only a sparse figure are excluded rather than halved.

Fine-Tune Cost Lab is published by the Runcrate team, who also run Runcrate, an AI cloud. Runcrate is listed in the table, marked "ours, live". It is sorted by price like every other row, with no pinned position. Other providers' prices are read from their own public pricing pages. We have no paid placement.