Generating a string of tokens at once: 5.7× faster with no drop in quality

Autoregressive language models have a default assumption: each forward pass emits only one token. To generate a response, the model has to call itself repeatedly, and latency grows with model scale. In a paper submitted on October 19, 2026 by Felix Draxler and 15 coauthors (16 authors in total), they propose Parallel Language Models, or PLM, aiming to let the model generate a string of mutually consistent tokens in a single forward pass. The tweet says, This will change how LLMs be trained.

The tweet's video slows down the generation process 60×: blue tokens are proposed together by a single PLM call, green tokens are accepted by the verifier, and gray tokens are the steps the autoregressive model still has to take.

Methodologically, PLM views parallel text generation as a variational autoencoder. At inference time, a parallel decoder maps a set of random auxiliary variables into a coherent sequence in a single call. The auxiliary variables encode dependencies among tokens. During training, PLM pairs this with an autoregressive encoder that maps each training token to auxiliary variables that can be decoded back to it. Maximizing the ELBO of the data likelihood ultimately reduces to a standard autoregressive loss plus a parallel reconstruction loss. The encoder and decoder share all Transformer weights; only the input embeddings differ.

The auxiliary variables are a fixed set of unit vectors in the embedding space. Each auxiliary variable is assigned to the most similar token embedding, with a per-token bias so that the share of auxiliary variables belonging to a token equals the probability of that token under the autoregressive encoder. The project page calls this a weighted Voronoi partition. This design exploits the similarity between token embeddings.

At inference time, auxiliary variables are sampled at random and fed to the decoder, and the decoder proposes a block of tokens at once. In the next call, the encoder verifies these proposals while the decoder simultaneously proposes the next block; both happen in a single forward pass, which the project page says is adapted Quadratic Decoding. In addition to exact verification, they use a relaxed acceptance rule that keeps proposed tokens with sufficiently high probability under the encoder. At comparable quality, this accepts on average 46.5% more tokens.

Results: During pretraining, from 80M to 3B parameters, PLM approaches the autoregressive pretraining loss, and the speedup follows a power law with model scale. After pretraining, PLM is better than autoregressive models at the same inference speed on text continuation, while using only 25B to 300B training tokens, whereas the compared models used trillions of tokens; it also outperforms the discrete diffusion models tested, including few-step models. After post-training with SFT and DPO, the 1.3B PLM matches or exceeds similarly post-trained autoregressive models on math, code, and instruction-following benchmarks while inferring 2.3 to 5.7 times faster. The 3B model has even higher quality. As scale increases, the number of correctly reconstructed tokens also grows according to a power law, and the model becomes more parallel. β controls the trade-off between quality and parallelism.

Assessment: Parallel generation is often invoked to talk about latency, but many approaches compromise on quality. What makes PLM worth watching is that the parallel decoding loss is part of the pretraining objective, and verification and proposal can happen in the same forward pass. It does not bypass autoregression: the encoder still carries the autoregressive loss during training. The cost is a more complex training objective, and both the auxiliary-variable assignment and the acceptance rule need tuning. Code is available on GitHub. For those building inference services, the 5.7× speedup is a direct draw; for those doing pretraining, the more critical question is whether this curve can continue as parallelism scales up under a power law with model size.

Paper: https://arxiv.org/abs/2610.26693

Project page: https://www.justuswill.com/plm/

Code: https://github.com/mandt-lab/plm