关键词: 长短期记忆网络, 并行流水, 硬件加速, NLP, MAC, 运算簇, 存储阵列

Abstract: Compared to traditional recurrent neural networks (RNNs), long short-term memory networks(LSTM) increases multiple gating units and memory cells, effectively overcoming the issues of gradient vanishing and gradient explosion encountered by traditional RNNs. Due to the advantage in handling complex sequential dependencies, LSTM networks have been widely applied in natural language processing tasks such as machine translation, sentiment analysis, and text classification.With the increasing complexity of intelligent applications and the increase in the number of layers and hidden layer nodes in LSTM networks, the requirements for storage capacity, memory access bandwidth, and processing performance of end side processing devices have also dramatically increased. This paper analyzes the characteristics of the LSTM algorithm and designs a highly parallel pipeline computation unit. A multi-level shared data path method is proposed, and optimization and control of the hardware implementation process of the LSTM algorithm are carried out. A hardware accelerator for LSTM is designed, achieving a peak computing power of 2.144 TOPS. The accelerator is physically implemented based onFinFET technology, and chip-level test results after tape-out demonstrate that the LSTM hardware acceleratorachieves an operational efficiency exceeding 95%, with processing performance per TOPS reaching more than 2.8 times that of the NVIDIA GTX 1080 Ti GPU.

Key words: long short-term memory, parallel pipeline, hardware acceleration, NLP, MAC, computational cluster, storage array