Self-host an LLM: the hardware, and what actually runs on it

Cheaper than an API? Almost never. Even Llama 3.2 3B Instruct, the smallest model here, needs ≈ 327M tokens a month before a 24/7 GPU beats the cheapest verified API — bigger models push that into the billions. You self-host for control, privacy, residency and offline use; the calculator prices that decision for your volume. Every figure below is computed from the index and dated.

The hardware

What decides everything is the memory pool a model must sit in — VRAM on a discrete card, unified memory on Apple and DGX. Sticker price is also shown in months of Claude Max ($200/mo, the top tier) as a familiar yardstick — a price reference, not a capability match. Prices are a dated snapshot; the 2026 memory shortage is inflating all of them.

Bare workstation GPUs (RTX 5090, RTX PRO 6000) need a host PC on top; Apple and DGX machines are complete. Electricity and idle time aren't included — a mostly-idle box loses to the API on cost every time.

What fits on what

Every listed open-weight model, grouped by the memory it needs in 4-bit. Pick your memory budget above, then see what runs. Sizes and the smallest single GPU per model are computed from the index; click a model for the full run guide with int8/fp16.

Break-even calculator

Pick a model, set your volume and GPU hours. The throughput field is yours to fill — we publish no tokens/s we haven't measured, so feasibility stays unknown until you enter your own. Hardware only, electricity excluded.

Why self-host, when the API is cheaper

Put the other way: Mac Studio M5 Ultra (256 GB unified) costs ≈ 47 months of Claude Max, and it runs open-weight models, not Claude. What earns self-hosting its place never shows up in a break-even:

• Data sovereignty. Prompts and outputs never leave your machine — no third-party terms, no retention policy to audit.
• No provider dependency. An API model can be deprecated, repriced or rate-limited overnight; downloaded weights run until you retire them.
• Compliance & residency. When contracts say the data stays in-house or in-country, on-prem inference is the direct answer.
• Offline & no quotas. Air-gapped environments, or workloads that would blow through any rate limit.

Sovereignty is your call, not a number we could honestly compute — the calculator prices the decision, it doesn't make it.

Method

VRAM ≈ params(B) × bytes/param × 1.2 (overhead: KV cache + activations + CUDA context). bytes/param: int4=0.5 · int8=1 · fp16=2. Break-even = cost of a GPU running 24/7 at the on-demand rate ÷ API price (blended $/1M): the monthly token volume above which a dedicated GPU is cheaper than the lowest-priced API. Table figures assume continuous use; real throughput (tokens/s) caps what the hardware can actually deliver — the break-even calculator models that cap from a throughput YOU measure and enter (we publish no tokens/s we haven't measured ourselves).

Model sizes verified 2026-09-10 (stated in each model's name/card, or an established published spec). GPU rates verified 2026-10-19 — on-demand, cheapest reliable tier: runpod · lambda · vast.

Machine purchase prices verified 2026-10-19: dgx_spark · mac_studio · rtx_5090 · rtx_pro_6000. Months of Claude Max = purchase price ÷ $200/mo (Claude Max 20x, the top tier), a price yardstick only.