3.1 The AI Wave Boosts Computing Power Demand, and Achieving Scalability of Intelligent Technology R
Last updated
Last updated
Machine learning has entered the era of large models, with the training and iteration of general large models like ChatGPT greatly increasing the demand for intelligent computing power. After successful deployment of models, a large amount of intelligent computing power is also needed for inference. From the perspective of model training, the computational power for machine learning training can be roughly divided into three periods. The first period was before 2012, when training computing power roughly followed Moore's Law, doubling approximately every 20 months. With the advent of the deep learning era, the rate of computing power doubling accelerated to 56 months. Around 2015-2016, the era of large models began, during which the growth of computational volume slowed down, with a doubling time of about 10 months. However, the overall training computational volume was 2 to 3 orders of magnitude (OOM) larger than the systems of the deep learning era. By the end of 2022, with the success of ChatGPT leading a new wave of AI, general large models such as Bert, GPT4, and Wenxin Yiyan have been released domestically and internationally. These large models require trillions, or even quadrillions, of parameters, as well as thousands of GB of high-quality data, significantly increasing the demand for intelligent computing power. In addition, as models mature and are promoted, the intelligent computing power required for model inference will gradually increase, with its proportion continually rising.