| IEICE Technical Committee Submission System Conference Paper's Information |
Online Proceedings [Sign in] Tech. Rep. Archives |
| Go Top Page | Go Previous | [Japanese] / [English] |
| Paper Abstract and Keywords | ||
| Presentation | 2023-10-05 10:10 [Encouragement Talk] Communication Scheduling Based on Heuristic Algorithm in Distributed Deep Learning Ryudai Fukuda, Takuji Tachibana (Univ. Fukui)  NS2023-86 |
|
| Abstract | (in Japanese) | (See Japanese page) |
| (in English) | In distributed deep learning, where multiple processors are used, the learning time can be significantly reduced by executing the learning process on each processor. However, distributed deep learning requires communication of gradient data between processors during learning, and the communication processing time significantly affects the learning time. To reduce the communication processing time, pipelining methods based on pipeline processing and gradient compression methods based on data compression are used. In this paper, we propose a communication scheduling based on a heuristic algorithm that considers multiple data compression techniques and communication methods. The proposed method can quickly derive a unique solution from multiple compression techniques and communication schemes at each layer using a heuristic algorithm, enabling distributed processing on multiple processors. The performance of the proposed method is evaluated by simulation. Numerical examples show that the heuristic algorithm used in the proposed method is comparable to the genetic algorithm. It also shows that the solution derivation time of the heuristic algorithm is considerably faster than the solution derivation time of the genetic algorithm. | |
| Keyword | (in Japanese) | (See Japanese page) |
| (in English) | Distributed deep learning / Back propagation / Data compression / Communication scheduling / / / / | |
| Reference Info. | IEICE Tech. Rep., vol. 123, no. 198, NS2023-86, pp. 83-88, Oct. 2023. | |
| Paper # | NS2023-86 | |
| Date of Issue | 2023-09-27 (NS) | |
| ISSN | Online edition: ISSN 2432-6380 | |
| Copyright and reproduction |
All rights are reserved and no part of this publication may be reproduced or transmitted in any form or by any means, electronic or mechanical, including photocopy, recording, or any information storage and retrieval system, without permission in writing from the publisher. Notwithstanding, instructors are permitted to photocopy isolated articles for noncommercial classroom use without fee. (License No.: 10GA0019/12GB0052/13GB0056/17GB0034/18GB0034) | |
| Download PDF | NS2023-86 |
|
| Conference Information | |
| Committee | NS |
| Conference Date | 2023-10-04 - 2023-10-06 |
| Place (in Japanese) | (See Japanese page) |
| Place (in English) | Hokkaidou University + Online |
| Topics (in Japanese) | (See Japanese page) |
| Topics (in English) | Network architecture (5G, Local 5G, Beyond5G, Mobile networks, Ad-hoc and sensor networks, Overlay and P2P networks, Programmable networks, SDN/NFV, IoT, Network slicing), Next generation packet transport (High speed Ethernet, IP over WDM, Multi-service package technology, MPLS), Grid, etc. |
| Paper Information | |
| Registration To | NS |
| Conference Code | 2023-10-NS |
| Language | Japanese |
| Title (in Japanese) | (See Japanese page) |
| Sub Title (in Japanese) | (See Japanese page) |
| Title (in English) | Communication Scheduling Based on Heuristic Algorithm in Distributed Deep Learning |
| Sub Title (in English) | |
| Keyword(1) | Distributed deep learning |
| Keyword(2) | Back propagation |
| Keyword(3) | Data compression |
| Keyword(4) | Communication scheduling |
| Keyword(5) | |
| Keyword(6) | |
| Keyword(7) | |
| Keyword(8) | |
| 1st Author's Name | Ryudai Fukuda |
| 1st Author's Affiliation | University of Fukui (Univ. Fukui) |
| 2nd Author's Name | Takuji Tachibana |
| 2nd Author's Affiliation | University of Fukui (Univ. Fukui) |
| 3rd Author's Name | |
| 3rd Author's Affiliation | () |
| 4th Author's Name | |
| 4th Author's Affiliation | () |
| 5th Author's Name | |
| 5th Author's Affiliation | () |
| 6th Author's Name | |
| 6th Author's Affiliation | () |
| 7th Author's Name | |
| 7th Author's Affiliation | () |
| 8th Author's Name | |
| 8th Author's Affiliation | () |
| 9th Author's Name | |
| 9th Author's Affiliation | () |
| 10th Author's Name | |
| 10th Author's Affiliation | () |
| 11th Author's Name | |
| 11th Author's Affiliation | () |
| 12th Author's Name | |
| 12th Author's Affiliation | () |
| 13th Author's Name | |
| 13th Author's Affiliation | () |
| 14th Author's Name | |
| 14th Author's Affiliation | () |
| 15th Author's Name | |
| 15th Author's Affiliation | () |
| 16th Author's Name | |
| 16th Author's Affiliation | () |
| 17th Author's Name | |
| 17th Author's Affiliation | () |
| 18th Author's Name | |
| 18th Author's Affiliation | () |
| Speaker | Author-1 |
| Date Time | 2023-10-05 10:10:00 |
| Presentation Time | 25 minutes |
| Registration for | NS |
| Paper # | NS2023-86 |
| Volume (vol) | vol.123 |
| Number (no) | no.198 |
| Page | pp.83-88 |
| #Pages | 6 |
| Date of Issue | 2023-09-27 (NS) |
[Return to Top Page]