| IEICE Technical Committee Submission System Conference Paper's Information |
Online Proceedings [Sign in] Tech. Rep. Archives |
| Go Top Page | Go Previous | [Japanese] / [English] |
| Paper Abstract and Keywords | ||
| Presentation | 2021-10-28 15:30 A Deep Neural Network Model Compression with Spherical Clustering of Neurons Shin Sakamoto, Masao Okita, Fumihiko Ino (Osaka Univ)  NC2021-22 |
|
| Abstract | (in Japanese) | (See Japanese page) |
| (in English) | In this paper, we propose weight matrix compression with spherical clustering of neurons , aiming at reducing memory usage and computational complexity while maintaining the accuracy of deep neural network (DNN) inference models. Compared with existing methods that use general k-means clustering, the proposed method reduces the information loss with compression by unit spherizing the weight vector space in advance. Furthermore, the proposed method embeds the vector norm separated by spherization into the weight matrix of the previous layer, in order to obtain the identical calculation results without additional computation and space. We conducted experiments with AlexNet of the DNN model to compare the predictability of compressed DNN models with the proposed method and an existing method. Experimental results show that the proposed method can improve the accuracy of prediction by up to 20 at the same column reduction ratio in the case of high sparsity of the weight matrix before compression. | |
| Keyword | (in Japanese) | (See Japanese page) |
| (in English) | information loss / pruning / k-means / fully connected layer / / / / | |
| Reference Info. | IEICE Tech. Rep., vol. 121, no. 223, NC2021-22, pp. 22-27, Oct. 2021. | |
| Paper # | NC2021-22 | |
| Date of Issue | 2021-10-21 (NC) | |
| ISSN | Online edition: ISSN 2432-6380 | |
| Copyright and reproduction |
All rights are reserved and no part of this publication may be reproduced or transmitted in any form or by any means, electronic or mechanical, including photocopy, recording, or any information storage and retrieval system, without permission in writing from the publisher. Notwithstanding, instructors are permitted to photocopy isolated articles for noncommercial classroom use without fee. (License No.: 10GA0019/12GB0052/13GB0056/17GB0034/18GB0034) | |
| Download PDF | NC2021-22 |
|
| Conference Information | |
| Committee | MBE NC |
| Conference Date | 2021-10-28 - 2021-10-29 |
| Place (in Japanese) | (See Japanese page) |
| Place (in English) | Online |
| Topics (in Japanese) | (See Japanese page) |
| Topics (in English) | |
| Paper Information | |
| Registration To | NC |
| Conference Code | 2021-10-MBE-NC |
| Language | Japanese |
| Title (in Japanese) | (See Japanese page) |
| Sub Title (in Japanese) | (See Japanese page) |
| Title (in English) | A Deep Neural Network Model Compression with Spherical Clustering of Neurons |
| Sub Title (in English) | |
| Keyword(1) | information loss |
| Keyword(2) | pruning |
| Keyword(3) | k-means |
| Keyword(4) | fully connected layer |
| Keyword(5) | |
| Keyword(6) | |
| Keyword(7) | |
| Keyword(8) | |
| 1st Author's Name | Shin Sakamoto |
| 1st Author's Affiliation | Osaka University (Osaka Univ) |
| 2nd Author's Name | Masao Okita |
| 2nd Author's Affiliation | Osaka University (Osaka Univ) |
| 3rd Author's Name | Fumihiko Ino |
| 3rd Author's Affiliation | Osaka University (Osaka Univ) |
| 4th Author's Name | |
| 4th Author's Affiliation | () |
| 5th Author's Name | |
| 5th Author's Affiliation | () |
| 6th Author's Name | |
| 6th Author's Affiliation | () |
| 7th Author's Name | |
| 7th Author's Affiliation | () |
| 8th Author's Name | |
| 8th Author's Affiliation | () |
| 9th Author's Name | |
| 9th Author's Affiliation | () |
| 10th Author's Name | |
| 10th Author's Affiliation | () |
| 11th Author's Name | |
| 11th Author's Affiliation | () |
| 12th Author's Name | |
| 12th Author's Affiliation | () |
| 13th Author's Name | |
| 13th Author's Affiliation | () |
| 14th Author's Name | |
| 14th Author's Affiliation | () |
| 15th Author's Name | |
| 15th Author's Affiliation | () |
| 16th Author's Name | |
| 16th Author's Affiliation | () |
| 17th Author's Name | |
| 17th Author's Affiliation | () |
| 18th Author's Name | |
| 18th Author's Affiliation | () |
| Speaker | Author-1 |
| Date Time | 2021-10-28 15:30:00 |
| Presentation Time | 25 minutes |
| Registration for | NC |
| Paper # | NC2021-22 |
| Volume (vol) | vol.121 |
| Number (no) | no.223 |
| Page | pp.22-27 |
| #Pages | 6 |
| Date of Issue | 2021-10-21 (NC) |
[Return to Top Page]