Which open and uncensored LLMs fit 20, 24 or 96 GB of VRAM

Look up the VRAM requirements of almost any LLM and you get one number: the size of the model file. On a server that is only the floor. The graphics card also holds a cache for every conversation running at the same time, plus the software that serves the model. This guide sorts the open and uncensored models in our catalogue onto three GPU sizes: 20, 24 and 96 GB.

The short answer: 20 GB holds a 14B model, or Gemma 4 26B at 4-bit. 24 GB adds the 27B to 35B class at 4-bit, with room for long conversations. 96 GB runs gpt-oss-120b in its native format and every 30B-class model at full precision. The VRAM calculator does the sums for your own model, context and number of users.

What uses VRAM: weights, KV cache and runtime

The weights are the model itself. Most people run a compressed (quantised) file with 4 or 8 bits per weight instead of the original 16. As a rule of thumb, count 0.6 to 0.7 GB per billion parameters at 4-bit, 1.1 GB at 8-bit and 2 GB at 16-bit.

The KV cache is the model’s working memory for a conversation, and the part people forget. Every token, a word or part of a word, leaves an entry in every layer. One conversation of 32,000 tokens on Qwen3-14B needs 5.4 GB of cache.

The runtime is the software that serves the model, such as vLLM, llama.cpp or Ollama. It needs room for its own buffers, and image input adds a vision module of about 1 GB. vLLM also reserves a fixed share of the card at startup and fills it with cache, so a running server looks fuller than the model needs. Ollama vs vLLM explains the difference.

The catalogue: VRAM requirements model by model

These models come from our Private AI catalogue. All of them are licensed under Apache 2.0 or MIT, so you can use them commercially and fine-tune them on your own data. Most uncensored variants were made by removing the refusals from the weights, explained in how a model loses its “no”.

File sizes as Hugging Face lists them. MoE stands for mixture-of-experts; a dense model uses all its parameters for every token.

Mixture-of-experts saves compute, not memory

Gemma 4 26B A4B uses 3.8 billion parameters per token but needs all 25.2 billion in memory, because it picks different experts for each token. Fewer active parameters make a model faster. They don’t make it smaller.

What mixture-of-experts does allow is splitting the model. llama.cpp can keep the expert weights in the server’s main memory while the rest stays on the GPU. That is the way to use DeepSeek V4 Flash’s 156 GB file at 4-bit with a 96 GB card: Hetzner’s GEX131 has 256 GB of RAM for the experts.

KV cache: context times users

What a token costs in the cache depends more on the model’s design than on its size. Newer models keep the full context in only some of their layers, or store it in compact form.

Then multiply by users. Five such conversations on Qwen3-14B need almost 27 GB of cache, more than a 24 GB card holds before the model is even loaded. Three settings keep that in check: a limit on the context length, a limit on how many requests run at the same time, and an 8-bit cache, which takes about half the space.

Best LLMs for 20, 24 and 96 GB of VRAM

Which graphics card a local LLM needs comes down to VRAM. Calculated for llama.cpp, with 32,000 tokens per conversation and 5% of the card kept free, Gemma 4 26B A4B holds two conversations on 20 GB, Qwen3.8 27B two on 24 GB and gpt-oss-120b 25 on 96 GB.

20 GB: one 14B model, or Gemma 4 26B for one or two users

This is the RTX 4000 SFF Ada in Hetzner’s GEX44. It is enough for a 14B model that serves a few chats in parallel, or for Gemma 4 26B A4B at 4-bit: two long conversations, or one with image input. Qwen3.8 27B fits for a single user with shorter context, which is why our catalogue starts it at 24 GB.

24 GB: the 27B to 35B class at 4-bit

This is the size of cards like the RTX PRO 4000 Blackwell. Qwen3.8 27B runs here at 4-bit with 32,000 tokens of context, Qwen3.6 35B-A3B with moderate context. Gemma 4 31B’s 18.7 GB file leaves little room for context, so we list it from 96 GB. gpt-oss-120b does not fit: about two thirds of its 65 GB would have to sit in system RAM.

96 GB: unquantised 30B models, gpt-oss-120b, DeepSeek at 2-bit

This is the RTX PRO 6000 Blackwell in Hetzner’s GEX131 in Falkenstein, Germany, with 256 GB of RAM. Every 30B-class model runs here unquantised, at full 16-bit precision. gpt-oss-120b runs with its full context, and a 70B dense model at 8-bit. DeepSeek V4 Flash fits completely at 2-bit (86.7 GB), or at 4-bit with the experts in RAM. The same server also has room for fine-tuning on your own data.

If you’d rather have one of these models running on a server of your own without doing the sizing, that is what Private AI is for. We choose the size that fits, install the model and test it with your own questions before it goes live.