Damai Dai

Native Sparse Attention: Hardware-Aligned and Natively Trainable Sparse Attention Jingyang Yuan, Huazuo Gao, Damai Dai, Junyu Luo, Liang Zhao, Zhengyan Zhang, Zhenda Xie, Yuxing Wei, Lean Wang, Zhiping Xiao, Yuqing Wang, Chong Ruan, Ming Zhang, Wenfeng Liang, Wangding Zeng Important: The Anthology treat PDFs as authoritative. Please use this form only to correct data that is out of line with the PDF. See our corrections guidelines if you need to change the PDF. Title Adjust the title.

Papers Published on Feb 16 ยท Submitted by HelloJiang on Feb 18 #1 Paper of the day Abstract Long-context modeling is crucial for next-generation language models, yet the high computational cost of standard attention mechanisms poses significant computational challenges. Sparse attention offers a promising direction for improving efficiency while maintaining model capabilities.

[Submitted on 14 Dec 2023 ( v1 ), last revised 19 Feb 2024 (this version, v3)] Title:Math-Shepherd: Verify and Reinforce LLMs Step-by-step without Human Annotations Download a PDF of the paper titled Math-Shepherd: Verify and Reinforce LLMs Step-by-step without Human Annotations, by Peiyi Wang and Lei Li and Zhihong Shao and R.X. Xu and Damai Dai and Yifei Li and Deli Chen and Y.Wu and Zhifang Sui Download PDF Abstract:In this paper, we present an innovative process-oriented math process...