关键词: GPU集群, 并行计算, 人工智能, 扩展互连, 网络结构

Abstract: Driven by the exponential growth of parameter scale and training data in artificial intelligence applications and large-scale neural networks, we design a novel general-purpose GPU cluster parallel computing system. Distinct from traditional Clos general distributed and supernode rack-mounted GPU clusters, we propose a new parallel computing system based on wafer-level substrate. Based on the analysis of large model parallelism technologies, an extended interconnection technique for GPU clusters is presented, the overall network architecture is designed, and the basic modules and system construction of the parallel computing system are completed. Finally, a comparative analysis is conducted on the advantages and disadvantages between the proposed system and current mainstream GPU multi-card modules. The evaluation results demonstrate that the computational density and power density of this parallel computing system reach 1.34 times and 4.59 times of those of mainstream commercial multi-GPU modules, respectively, with the energy efficiency ratio increased by 16%. The proposed system is capable of satisfying the continuously growing demands for data processing and computing power of artificial intelligence applications in the next three to five years.

Key words: GPU cluster, parallel computing, artificial intelligence, extended interconnection, network architecture