Attention Mechanisms

Multi-head, grouped-query, multi-query, and multi-head latent attention variants and their tradeoffs.