Three LoRAs Combined into One: Qwen3.8-27B Coding Benchmarks Gain Up to Nearly 20 Points
The Hugging Face team trained three LoRA adapters for Qwen3.8-27B, one each on pi, OpenCode, and Hermes Agent, then combined the three directly into a super adapter. No extra training was required, performance improved, and tool calls dropped by 53% to 73%. It also includes an interesting phenomenon about “reasoning leakage.”
One model, three coding agents, one super adapter. The Hugging Face team built three LoRA adapters for Qwen3.8-27B, corresponding to pi, OpenCode, and Hermes Agent, then directly combined the three adapters into one. The resulting model gained up to nearly 20 points on SWE-bench Pro and cut tool calls by more than half.
An agent harness is the execution framework for coding agents. The same model performs differently under different harnesses, and a general-purpose model will not be optimized for a particular framework. The team’s approach: freeze the model weights and use reinforcement learning to train a small LoRA adapter inside the harness. What the adapter learns is “how to work more efficiently within this framework.”
Each adapter on its own shows clear gains. pi went from 39.2% to 64.7% on SWE-bench Pro hard 51, Hermes from 29.4% to 43.1%, and OpenCode from 35.3% to 51.0%. On Terminal-Bench 2.1, the gains were +16.3, +14.1, and +6.0, respectively. More important is efficiency: on tasks that both the base model and the adapter can solve, tool calls dropped by 56% to 72%.

The merge is even more interesting. The three adapters’ weight updates are nearly orthogonal, with cosine similarity of only 0.01 to 0.02. Each adapter pushes the model in a different direction without interfering with the others. The team concatenated the three LoRAs along the rank dimension, with weights set to 1/2. With no additional training, this produced a rank-96 super adapter.
With thinking off, this super adapter beat the base model on all six harness-benchmark combinations, raising average pass@1 from 39.5% to 49.0% while using 53% to 73% fewer tool calls. The most surprising part was with thinking on: although thinking mode was never enabled during training, it became the strongest model. Its average rose from 55.5% to 63.5%, with +15.7 and +19.6 under Hermes and OpenCode on SWE-bench Pro, respectively; pi was already at 74% and stayed flat.

The task data came from SWE-smith, SWE-Gym, and Tasksmith, was validated with openenv validate, and then the base model attempted the tasks; only tasks that were “sometimes solvable and sometimes not” were kept. This ensures GRPO has a learning signal.
The team noted a phenomenon: in non-thinking mode, the adapter model generates a short piece of text before a tool call, sometimes looking like reasoning traces, such as “..wait? ..let me see.” The base model usually calls tools directly. They call this “reasoning leakage.” One commenter said: “Combining three adapters into one rank-96 LoRA feels like cheating.” The team responded: the results are valid, and it really is stronger under the same conditions. However, they also acknowledged that training purely thinking models needs dedicated study.
Some developers also think the adapter layer is more interesting than the model itself: if harnesses share the same interface, teams can test the same agent behavior without rewriting workflows.
The full recipe is on the Hugging Face blog, and all models, GGUF files, and tasks are open sourced. If you have your own model and harness, you can train one following the same approach. Key point: let the model run in a real harness, not reimplement one inside a framework.
发布时间: 2026-11-21 07:09