4B Vance hits 69.4%, and it wants to write Verilog on a laptop

Vance is an LLM tuned for SystemVerilog with 4B parameters. Author Patrick Ostler says it scored 69.4% on VerilogEval V2 Spec-to-RTL, beating all local LLMs in the figure. The model is already on Hugging Face under the repo name ostlerai/vance-4b-instruct and can run on a laptop.

SystemVerilog is a hardware description language used to write chip and FPGA logic. VerilogEval V2 Spec-to-RTL tests generating RTL code from specifications—that is, turning natural-language requirements into synthesizable hardware descriptions. Hardware development has high correctness requirements, and local LLMs have long struggled with this task. A 4B model scoring 69.4% is somewhat counterintuitive.

The image is a comparison compiled by ATaylorFPGA, on a 128GB AMD Strix Halo platform. The table lists model name, size, quantization method, runtime, CVDP pass rate, VerilogEval S2R, VerilogEval CC, and tokens per second. From 125B-A6B MoE Q3 (about 90GB) Qwen3.8 Flash-Next to 9B Q4 Qwen-FPGA, CVDP pass rates range from 9.6% to 40.7%. At the bottom, Claude 3.7 Sonnet, GPT-4.1, and Llama 3.1 405B are included as references. The chart makes the reality of local hardware LLMs clear: parameters, quantization, speed, and pass rate all pull against one another.

Vance takes a different route: tuning a 4B model on SystemVerilog domain data for a specific task, rather than relying on scaling parameters. The tweet doesn't elaborate on training details, but mentions three sources. A post by Brian Roemmele about sovereign AI led the author to start the project. 《Deep Learning for Coders with Fastai and Pytorch》 by Jeremy Howard and Sylvain Gugger was his technical reference when getting started. The benchmark results were compiled by ATaylorFPGA.

For people who want to do RTL generation locally and don't want to send code to the cloud, Vance is an option they can try directly. It's a 4B instruct model, runs on a laptop, and can be downloaded by searching the repo name on Hugging Face. 69.4% is not a number that can go straight into production, but it shows that specialized small models have room in local hardware workflows. General-purpose large models often confidently get Verilog wrong; specialized tuning at least pushes the error rate down somewhat.

Original tweet: https://x.com/PatrickOstler/status/2117707109873897731