METHODS FOR REDUCING THE COMPUTATIONAL COMPLEXITY OF LIGHTWEIGHT NEURAL NETWORKS USING DATABASES FOR MOBILE AND EDGE PLATFORMS

Abstract

In addition to the use of lightweight architectures, a key optimization direction involves applying methods to reduce the computational complexity of existing machine learning models. To this end, this paper proposes a combination of techniques: quantization, pruning, knowledge distillation, low-rank decomposition, computation graph optimization, and operator fusion. Another important optimization avenue-beyond lightweight architectures-involves reducing computational complexity by utilizing databases to store and efficiently retrieve compressed machine learning model parameters. Each of these approaches has its own advantages, disadvantages, and practical implementation nuances. These optimization methods can be implemented either independently or in combination and interaction with one another. For instance, machine learning models often combine quantization with pruning or employ knowledge distillation followed by execution graph optimization. Such a combined strategy yields greater efficiency than the use of isolated approaches. It has been demonstrated that employing these methods significantly reduces machine learning model size, lowers memory usage, and accelerates inference. However, it has also been observed that most of these approaches establish a static model configuration prior to deployment; once optimization is complete, the model's structure and behavior remain virtually unchanged. Consequently, it is evident that optimization efficiency may vary depending on the device's current state and execution conditions.