심층신경망 모델 압축을 위한 확률적 양자화 기법의 성능 평가 (Performance Evaluation of Stochastic Quantization Methods for Compressing the Deep Neural Network Model)

서지정보

· 발행기관 : 제어·로봇·시스템학회
· 수록지 정보 : 제어.로봇.시스템학회 논문지 / 25권 / 9호 / 775 ~ 781페이지
· 저자명 : 최준영, 유준혁

초록

Discretizing the Deep Neural Network (DNN) is enforced by a rounding function of weights and activations, which canseverely degrade the performance of DNN. This paper categorizes the available quantization methods to be uniform or non-uniformaccording to the distribution of its quantized interval, and proposes a noble stochastic rounding method to reduce the accuracy loss ofthe quantized DNN model. Two stochastic rounding methods of linear or nonlinear probability distribution are quantitativelyevaluated and analyzed to give an important insight to employ them for designing the compressed DNN model, by comparing themwith the previous deterministic nearest rounding. Experimental results show that the stochastic rounding methods are not alwayssuperior to the deterministic one and the optimal performance is obtained via a hybrid rounding scheme when applying the stochasticrounding to the input adjacent to the intermediate value between the neighboring quantized values and still using the deterministicone near to the quantized value.

영어초록

Discretizing the Deep Neural Network (DNN) is enforced by a rounding function of weights and activations, which canseverely degrade the performance of DNN. This paper categorizes the available quantization methods to be uniform or non-uniformaccording to the distribution of its quantized interval, and proposes a noble stochastic rounding method to reduce the accuracy loss ofthe quantized DNN model. Two stochastic rounding methods of linear or nonlinear probability distribution are quantitativelyevaluated and analyzed to give an important insight to employ them for designing the compressed DNN model, by comparing themwith the previous deterministic nearest rounding. Experimental results show that the stochastic rounding methods are not alwayssuperior to the deterministic one and the optimal performance is obtained via a hybrid rounding scheme when applying the stochasticrounding to the input adjacent to the intermediate value between the neighboring quantized values and still using the deterministicone near to the quantized value.

참고자료

· 없음