FP8 Attention on AMD MI355X: Zyphra Claims 1.76x Speedup Over AITER
Zyphra open-sources ZYFA, a set of FP8 and BF16 prefill attention kernels optimized for AMD Instinct MI355X. Tested on real model activations, the FP8 variant runs 1.16 to 1.76x faster than AMD AITER, with 62% to 78% lower error at head dimension 128.
Zyphra has released ZYFA, an open-source collection of FP8 and BF16 prefill attention kernels built for the AMD Instinct MI355X. As reported by Michal Sutter from MarkTechPost, when tested on real-world model activations, its FP8 kernel delivers 1.16 to 1.76x higher performance than AMD's AITER, while reducing error rates by 62% to 78% at a head dimension of 128.
Why This Is Worth Noticing
Large language model inference typically operates in two distinct phases: prefill and decode. The prefill phase processes user input prompts, and attention is the unavoidable core computation for all Transformer architectures. Optimizing the speed of attention kernels in this phase directly improves first-token latency and overall system throughput.
AMD's software stack has long been catching up in the LLM acceleration space. AITER is AMD's official operator library, and ZYFA chose to benchmark against it on the MI355X alongside publicly available error metrics, to prove that FP8 not only runs on the platform, but also maintains reliable precision. FP8 8-bit floating point arithmetic cuts down VRAM bandwidth consumption and compute overhead, with the biggest challenge being effective error control. Head dimension 128 is a widely adopted production configuration, so the data measured at this setting carries high reference value.
How To Interpret The Performance Numbers
The 1.16 to 1.76x speedup is a result measured under specific test constraints: real model activations, native FP8 kernel implementation, and direct head-to-head comparison with AITER. The original article does not cover all possible model sizes, sequence lengths, and full end-to-end inference scenarios, so this figure is a kernel-level performance signal, not an equivalent speedup guarantee for complete whole-system inference.
The core value of open-sourcing this work is full reproducibility. With the code released publicly, developers can run their own benchmarks, verify error rates and performance, and examine how the implementation makes use of the unique hardware features of the MI355X. For people following the AMD LLM ecosystem, this release is far more valuable than a standalone technical blog post.
Original article link: Zyphra ZYFA: FP8 FlashAttention for AMD MI355X, Up to 1.76x Faster, authored by Michal Sutter.
发布时间: 2026-11-07 01:28