Heretic 2.0: No Training, 30 Minutes—Get Rid of the Model's Preachiness

Heretic 2.0 has been released. The project has 37K stars on GitHub, and its models have been downloaded 88 million times on Hugging Face. Those numbers make it clear that this is not a toy.

Previously, when people mentioned Heretic, they thought of de-censorship. Give it a model, and it erases the tendency to refuse to answer. Version 2.0 expands the scope: it is now a general-purpose tool for modifying language models. The official description is a shift from a specialized de-censorship tool to a flexible behavior modification framework.

What exactly was added?

Arbitrary-Rank Ablation, or ARA for short, replaces the original directional ablation. Directional ablation finds a refusal direction and projects it out, while ARA is more flexible and no longer assumes that a behavior changes along only one direction. The author says this was developed specifically for Heretic, and the preprint has not yet been published.

Plugin architecture. A model's optimization objectives are determined by scorer plugins; by default, these are refusal rate and KL divergence. You can write your own scorers or combine multiple ones. Behavior modification is handled by modifier plugins, with two built in: ARA and traditional directional ablation.

There are more controls. In addition to de-censorship, the official release provides several ready-made configurations: config.noslop.toml reduces slop, config.nohumor.toml lowers the sense of humor, config.nosyco.toml suppresses sycophancy, and config.emotions.toml adjusts emotional alignment. If you want to change something, just switch configuration files.

Compatibility has also been improved. Support is better for thinking models and models with hyper-connections. Automated testing has kept pace as well; non-interactive execution can run full end-to-end tests.

Usage has not become more complicated:

pip install -U heretic-llm
heretic Qwen/Qwen3-4B-Instruct-2507

The default action is de-censorship. To do something else, specify a configuration file. On an RTX 3090, the default configuration processes Qwen3-4B-Instruct-2507 in about 30 to 40 minutes. After it runs, you can choose to save, upload to Hugging Face, test by chatting directly, or run benchmarks.

One thing to note: 1.x configuration files are not compatible with 2.x. The migration guide is on GitHub. You need to delete the research feature settings, move the abliteration-related configuration to the [modifier.Abliteration] section, and move the refusal evaluation settings to the [scorer.KeywordRate] section. 2.x uses ARA by default; if you want to keep using the old directional ablation, you have to manually change the modifiers setting.

There is one more detail worth mentioning. The AI world currently likes to use coding agents to write code, and the author, Philipp Emanuel Weidmann, specifically wrote a passage in the README: Heretic is not vibe-coded; every line of code he submits is handwritten, and every merged PR has been reviewed line by line. He gives an example: one PR accumulated nearly 350 human comments and was discussed for more than 7 months. That remark is a bit pointed today, but the project's code is indeed clean and simply structured.

Heretic also implements reproducibility. When uploading a model, you can choose to include a reproduce folder containing a machine-readable manifest. Others can take the manifest and run heretic --reproduce to reproduce results with identical weights in just a few minutes, without rerunning those hours of computation. These manifests are collected in a repository called Heretic Grimoire and can be backed up locally.

Back to the judgment at the beginning. Model modification used to be a privilege of companies with large compute resources: training, fine-tuning, RLHF—every step burned money. Heretic takes another path: no training; instead, it uses contrastive examples and scorers to perform a mathematical shift. There is of course an upper limit to the effect, but it lowers the barrier to the point where anyone with a gaming GPU can try it. 37K stars and 88 million downloads probably mean that quite a few people think this is worth tinkering with.

GitHub: https://github.com/p-e-w/heretic

Migration guide: https://github.com/p-e-w/heretic/blob/master/docs/migration_v1_to_v2.md