Wink Pings

One LoRA Boosts Qwen3.8's Coding Pass Rate from 39% to 65%

Hugging Face has released the Apache-2.0 licensed Qwen3.8-27B-pi-adapted LoRA adapter, trained via reinforcement learning for pi coding agents. It lifts SWE-bench Pro hard-51 pass@1 rate from 39.2% to 64.7%, and Terminal-Bench 2.1 performance from 39.5% to 55.8%.

Hugging Face has released Qwen3.8-27B-pi-adapted, an Apache-2.0 licensed LoRA adapter trained with reinforcement learning, built for pi coding agent scenarios.

The performance metrics it delivers are very straightforward:

EvaluationBaseline ResultResult With Adapter
SWE-bench Pro hard-51 pass@139.2%64.7%
Terminal-Bench 2.139.5%55.8%

SWE-bench Pro hard-51 pass@1 sees a 25.5 percentage point increase, while Terminal-Bench 2.1 posts a 16.3 percentage point uplift. Both benchmarks measure single-attempt success rate: the former focuses on real-world software engineering tasks, while the latter centers on terminal environment operations and troubleshooting. For coding agents, these metrics are far more aligned with actual production workflows than standard code completion benchmarks.

The value of this LoRA adapter lies in its lightweight design. It does not modify all parameters of the base model, but adds a layer of loadable adaptation weights on top of Qwen3.8-27B. For engineering teams, deployment and iteration costs are far lower than fully retraining a large language model. The Apache-2.0 license also lowers the usage barrier for the adapter itself, though you still need to separately confirm the licenses of the base model, pi coding agent and all related dependencies.

While these results are exciting, there are several key constraints to note. The hard-51 subset is a small collection of extremely challenging tasks, and pass@1 only reflects the success rate of a single attempt. Messy unstandardized code, missing dependencies and vague requirements in real code repositories are not equivalent to curated evaluation datasets. While the performance uplift is worth attention, it is still recommended to re-run tests on your own project's task set before production rollout, to observe failure modes and overall stability.

If you want to try it out, the access point is on the Hugging Face model page. Read the model card first to confirm the base model version, loading method and integration instructions for the pi coding agent, then run validation on a small set of tasks. The original article was published by Asif Razzaq on MarkTechPost, the link is here.

发布时间: 2026-11-19 00:07