Voice AI that listens while it speaks goes open source: Haru-4B runs locally on 12GB VRAM
Japan's higurashi AI has open-sourced Haru-4B, a full-duplex speech model based on Qwen3.5-4B. It can listen and speak at the same time, handle interruptions and overlapping speech, and supports custom character voices. It can run locally on an NVIDIA GPU with about 12GB of VRAM, and the weights and inference code are under Apache 2.0. The model topped Hugging Face's trending chart across all categories.
Japan's higurashi AI has open-sourced Haru-4B, a 4B-parameter full-duplex speech model. It took first place on Hugging Face's trending chart across all categories.
Traditional voice assistants wait until you finish speaking before responding. Haru-4B is different. It can listen and speak at the same time, allows interruptions, will proactively interrupt you, and can make feedback sounds like “mm” and “ah,” making conversation feel closer to talking with a real person. The company says it supports 9 major languages.
The team's performance data from an Apple M1 Max MacBook: median processing latency of 65.2ms, generating one audio frame every 133ms, using its own Metal inference kernel.
The model architecture is based on Qwen3.5-4B plus audio components. Training data includes 40M+ hours of real speech and 13M+ hours of full-duplex dialogue, covering 160+ languages.
Local deployment requirements:
- NVIDIA GPU: about 12GB VRAM
- Apple Silicon: native MLX implementation
- GPU serving: vLLM-Omni
- Weights and inference code: Apache 2.0
- BF16 checkpoint: about 10.5GB
It also supports customizing characters and voices with reference audio.
Developers added real-world details in the comments. Model Test Lab mentioned that the 4-bit GGUF available to Mac users is only 4.5GB, but it can only run in speech.cpp; llama.cpp, Ollama, and LM Studio can't load it. The 8-bit MLX version is 6.6GB. Another developer, Sean Uddin, thinks 12GB is still too large and hopes to get it down to the 1GB level so it can coexist with a thinking LLM or serve as a front-end module for an existing LLM.
Another developer shared a different approach: not every scenario needs to be full-duplex. His Pipecat pipeline uses local STT (Nemotron ASR on MLX), local TTS (Pocket TTS + voice samples), and a Gemma 4 12B in the middle (running on Ollama). Swapping models only requires changing one variable; character settings and pronunciation are fully controllable, and data never leaves the machine. He built a demo called Lilly Beyond Talk, where you video-chat with an AI character in the browser. The code is on GitHub: https://github.com/Jeuners/lilly-beyond-talk


The characters in this project are clearly labeled as AI-generated, and the app also sets limits on call duration and number of calls.
Haru-4B lowers the barrier to full-duplex speech to consumer-grade graphics cards, and the open-source license is permissive enough. The more interesting question next is: once you run it locally, what do you do with it? The next step in voice interaction may not lie in the model alone.
发布时间: 2026-11-20 18:47