These startups are chasing the next big thing in LLMs

Why it matters

A wave of startups is building alternatives to the transformer, using sparse attention, retention mechanisms and diffusion approaches. The shared goal is cutting the compute cost of inference, with a secondary bet that new architectures enable reasoning that is not bound to language. Useful context for anyone assuming the current model shape is settled.

The source