The author of this Hacker News post, who admits limited expertise in model development, responds to readers asking why strands-decider-2B is built as a modified LLM rather than an encoder like BERT. The intuition is that bidirectional encoders, where each token is represented using both earlier and later tokens, should outperform causal models that only look backward. The author concedes that the project is essentially using a decoder as an encoder, so the question is fair.
To test this, five designs were compared: v19, the final Qwen3.5-based strands-decider; two encoder-decoder models based on T5Gemma 2 at 2B and 8B total parameters; and two encoder-only variants derived from T5Gemma, one of which uses a masked state cache so multiple questions can be evaluated in parallel. The 8B encoder-decoder model beats v19 on accuracy and Brier scores, but it is roughly four times larger, so the comparison is not apples-to-apples. The more meaningful comparison is v19 against the latency-optimized encoder e1b, which slightly beats v19 on one benchmark accuracy measure, though within run-to-run variance, and performs slightly worse on calibration.
On latency, the encoder-only models win clearly, with a median near 35ms versus 65ms for v19. However, the advantage shrinks as prompts grow longer. The encoder uses full attention across all layers, which scales quadratically, while most of v19's layers use Gated Delta Net with linear cost. These layers have a higher fixed cost but scale better, creating a higher latency floor. Since the benchmark workloads skew toward short prompts, the encoder benefits in practice.
The unmasked encoder e1a beats v19 on seven of nine tasks but performs worse on two key benchmarks, suggesting that allowing questions to attend to the state may be a weakness without masking. The author concludes that the encoder approach shows promise but is not a clear win across the board.
Source: Hacker News · Summarized by HeadlinesBriefing