【论文】循环循环Transformer

arXiv 2610.07591 · 20 人推荐 · 附代码 yifanzhang-pro/recurrent-looped-tranformer（908 stars）。State tracking requires an update at every input, but the depth a Transformer applies to each token is fixed regardless of sequence length. We introduce the Recurrent Looped Transformer (RLT), which splits its layers between a parallel causal encoder and a recurrent decoder. At each token, the decoder merges the encoder output with the previous token's final decoder state, so the computation path grows with sequence length at a fixed per-token cost. On six algorithmic tasks, we compare five spli