Scenario · 34B parameters · LoRA or QLoRA

34B LoRA fine-tune on 1 billion tokens: cost estimate

34 billion parameters, 1 billion tokens, lora or qlora. The lowest estimate across 11 GPU entries is $112 on Vultr with MI355X, 43 h of GPU time at 35% MFU. An estimate, not a quote.

Step by step

The arithmetic

1. Training FLOPs. 4 × 34 × 10^9 parameters × 1 billion tokens = 1.36 × 10^20 FLOPs. The forward pass costs about 2 x parameters x tokens and the backward pass about 2 x, because gradients for the frozen base weights are not computed. The small adapter matrices and QLoRA dequantisation are not counted.
2. Useful throughput per GPU. Dense BF16 TFLOPS × 10^12 × 35% MFU. The table gives it for each GPU.
3. GPU-hours. FLOPs ÷ throughput ÷ 3,600. For L40S (362.05 TFLOPS): 1.36 × 10^20 ÷ 1.27 × 10^14 ÷ 3,600 = 298 GPU-hours.
4. Wall-clock. GPU-hours ÷ 8 GPUs, assuming the work spreads evenly.
5. Cost. GPU-hours × each provider's lowest listed on-demand price per GPU-hour for that GPU.

Result

Estimated cost by GPU

GPU	Dense BF16 TFLOPS	GPU-hours	8 GPUs	Providers	Lowest	Median	Highest
MI355X	2500	43	5.4 h	2	$112Vultr	$242	$371Oracle Cloud
MI300X	1307.4	83	10 h	4	$153Vultr	$249	$495Oracle Cloud
MI325X	1307.4	83	10 h	2	$165Vultr	$239	$314DigitalOcean (Paperspace)
H100 SXMpreliminary	1000	108	13 h	6	$318Thunder Compute	$342	$549Verda
GH200	990	109	14 h	2	$217Vultr	$463	$709CoreWeave
H100 PCIepreliminary	800	135	17 h	2	$337Hyperstack	$364	$390RunPod
L40S	362.05	298	37 h	14	$164Runcrate	$458	$1,043Oracle Cloud
A100 80GB	312	346	43 h	12	$377Thunder Compute	$656	$1,187Amazon Web Services
L40	181.05	596	3.1 days	6	$471Thunder Compute	$554	$751Runcrate
A40	149.7	721	3.8 days	3	$425RunPod	$1,234	$1,918Runcrate
A10	125	863	4.5 days	4	$863OVHcloud	$1,662	$2,763Microsoft Azure

Prices are each provider's lowest listed on-demand rate per GPU-hour, read 12 November 2026. Only GPUs whose maker prints a dense BF16 figure appear; the coverage page says which are out and why.

Sensitivity

What if MFU were different? (L40S)

MFU assumed	GPU-hours	8 GPUs	Lowest cost
20%	522	2.7 days	$287
35% (default)	298	37 h	$164
50%	209	26 h	$115

The 20% and 50% rows are what-if inputs chosen to show sensitivity, not measurements. Lowest cost uses Runcrate at $0.55 per GPU-hour.

Try it

Change the assumptions

Model parameters (billions)

Training tokens (millions)

Method

FLOPs per parameter per token

GPU (dense BF16 printed by the maker)

Number of GPUs

MFU, percent of peak achieved

Estimate, step by step

1. Training FLOPs = 4 × 34 billion parameters × 1000 million tokens = 1.36 × 10^20
2. Useful FLOP/s per GPU = 362.05 TFLOPS × 35% MFU = 1.27 × 10^14
3. GPU-hours = step 1 ÷ step 2 ÷ 3,600 = 298.1
4. Wall-clock on 8 GPUs = step 3 ÷ 8 = 37 h, assuming the work spreads evenly
5. Cost = step 3 × each provider's lowest listed price per GPU-hour

Provider	Billing	$/GPU-hr	Estimated cost
Runcrateours, live	per second	$0.55	$164
CloudRift	per second	$0.69	$206
Massed Compute	billing not stated	$0.97	$289
RunPod	per second	$1.09	$325
Koyeb	per second	$1.20	$358
Civo	per hour	$1.29	$385
Crusoe	per hour	$1.50	$447
DigitalOcean (Paperspace)	per second	$1.57	$468
Vultr	per hour	$1.67	$498
OVHcloud	per hour	$1.80	$537
Amazon Web Services	per second	$1.86	$555
Verda	billing not stated	$2.00	$596
CoreWeave	billing not stated	$2.25	$671
Oracle Cloud	billing not stated	$3.50	$1,043

Range $164 to $1,043, median $458 across 14 providers. An estimate, not a quote: real runs differ in utilisation, memory fit, communication and restarts, and providers that bill per hour round up.

Questions

34B LoRA fine-tune on 1 billion tokens: common questions

How many FLOPs does this take?

About 1.36 × 10^20: 4 × 34 billion parameters × 1 billion tokens. The forward pass costs about 2 x parameters x tokens and the backward pass about 2 x, because gradients for the frozen base weights are not computed. The small adapter matrices and QLoRA dequantisation are not counted.

What does it cost at the low end?

$112, on Vultr with MI355X at $2.59 per GPU-hour, for 43 GPU-hours.

Is this a quote?

No. It is an estimate from a stated formula and published list prices. Real runs differ in utilisation, memory fit and restarts, and the MFU is an assumption.

Why are some GPUs missing?

A GPU needs a dense BF16 figure printed by its maker. Those that print only a sparse figure are excluded rather than halved.

Fine-Tune Cost Lab is published by the Runcrate team, who also run Runcrate, an AI cloud. Runcrate is listed in the table, marked "ours, live". It is sorted by price like every other row, with no pinned position. Other providers' prices are read from their own public pricing pages. We have no paid placement.