How a model loses its “no”: abliteration, Heretic and uncensored LLMs

An abliterated model is an open-weight language model whose refusals have been taken out of its weights, without retraining. Someone measured where the model represents “I should decline this”, removed that one direction and saved the result as a new model. The edit is called abliteration, a blend of ablation and obliterate. Heretic, the open-source tool that automates it, has already been used for well over 5,000 published models.

Why an LLM refuses: the “no” is trained in

A chat model does not refuse because a filter sits in front of it. It learns to refuse during the training that turns a raw language model into an assistant, and every vendor draws the line in a different place. Some refusals protect people. Others block a penetration tester, a doctor asking about overdose thresholds or a novelist writing a crime scene. Abliteration moves that decision from the vendor to whoever runs the model.

The refusal direction: one vector carries the “no”

In 2024, Arditi and colleagues sent 128 harmful and 128 harmless instructions through 13 open chat models and recorded what happened inside them. The difference between the two averages works like a switch. Take it out, and the model stops refusing harmful requests. Add it, and the model starts refusing harmless ones. In every model they tested, this one direction was enough.

To make it permanent, the direction is removed from the weights once, which leaves an ordinary model file. In the authors’ tests, most edited models kept their scores on standard knowledge and maths benchmarks.

Heretic: abliteration, automated

In 2024, Maxime Labonne turned the method into a recipe anyone could follow. Heretic, an open-source tool by Philipp Emanuel Weidmann, goes a step further and does the fine work by itself. An optimizer tries different settings, layer by layer, and scores each try on two goals: as few refusals as possible on harmful prompts, and as little change as possible on harmless ones. That second number is the KL divergence: how far the model’s word choices drift from the original’s.

You install Heretic, point it at a model and wait. For a model with 4 billion parameters, a run takes 20 to 30 minutes on a single RTX 3090, a gaming graphics card.

Does abliteration make a model dumber?

Done crudely, it can. Done carefully, the model keeps most or all of what it knows. The clearest example is Gemma 3 12B. Heretic’s documentation compares three edits that all cut Gemma’s refusals from 97 in 100 to 3. On the knowledge score of the UGI leaderboard (NatInt), they land in exactly the order of their drift: the edit with the most drift lost more than half of the original’s 18.72, the Heretic edit kept most of it.

gpt-oss-120b shows the same split. A plain abliteration fell from 33.68 to 16.72, the Heretic version kept 33.41. Newer methods go further still: a norm-preserving edit of Gemma 3 12B even scores above the original.

The rule that follows is simple: judge the finished model, not the name of the method.

The uncensored models in our catalogue

Our Private AI catalogue lists open-weight models whose licences allow commercial use and fine-tuning, most of them as uncensored variants. On the UGI leaderboard, the Heretic versions answer far more willingly and keep their knowledge:

Original model → Heretic variant. Which of them fits a 20, 24 or 96 GB graphics card is in our VRAM guide.

Abliterated vs uncensored: four ways to remove refusals

“Uncensored” is the umbrella term, and abliteration is one way to get there. The older way is training: Eric Hartford removed the refusals from an instruction dataset and trained an uncensored version of WizardLM on the rest.

Does abliteration work in other languages?

Yes. A team at LMU Munich showed that a refusal direction taken from English prompts also switches off refusals in the other languages the models were safety-trained in, German and Spanish included. To abliterate a model for German or Spanish speakers, you do not need separate prompt sets in those languages.

How to read the UGI leaderboard

UGI stands for Uncensored General Intelligence. The leaderboard is a Hugging Face Space run by DontPlanToEnd, and its test questions stay private so that nobody can train on them. Two columns matter most. W/10 is willingness on a scale from 0 to 10: how far a model can be pushed before it refuses or dodges. NatInt measures general knowledge.

Read the two together, and always next to the original model’s row. High willingness with a collapsed NatInt means a compliant model that got worse at everything else.

Uncensored means the rules are yours

Without built-in refusals, the rules move from the model to the way you run it: the system prompt, who can reach the model, what gets logged. A model free of a vendor’s policy still needs your own.

With Private AI we run such models for you: an open-weight model, uncensored if you want it, on a dedicated GPU server in Germany, with prompts and answers processed on your server. If a model still refuses too much for your work, we remove the refusals by abliteration. Before it goes live, it answers your own questions and edge cases side by side with the original.