What is FlashAttention?

FlashAttention is an exact attention algorithm introduced by Tri Dao et al. in a 2022 paper that dramatically accelerates self-attention by tiling computations to keep intermediate tensors in fast GPU SRAM rather than slow HBM memory. The platform strengthens enterpri