| 講演抄録/キーワード |
| 講演名 |
2026-08-07 11:30
Forgetting-Resilient Distributed Learning via Importance-Aware Sparse Parameter Retention ○Gebreegziabher Hagos Berhe・Takao Marukame・Tetsuya Asai・Kota Ando(Hokkaido Univ.) CCS2026-31 |
| 抄録 |
(和) |
Distributed learning over non-stationary tasks must adapt to new tasks while preserving previously acquired knowledge. Existing approaches suffer from catastrophic forgetting, as continual updates progressively overwrite the representations learned from earlier tasks. We propose a continual distributed learning approach that unifies objective-similarity-aware partial model fusion with B-SCOR² (Benefit-optimal Sparse COordinate Retention and Regularization), which identifies and preserves the parameters most critical to prior tasks through a benefit-optimal selection criterion while leaving the remaining parameter space free to adapt to evolving data. B-SCOR² stores the retained values compactly through a tensor-train singular value decomposition (TT-SVD) and protects them with a selective regularization penalty, preserving old-task knowledge at a fraction of the memory cost of dense methods. On CIFAR-10, B-SCOR² improves prior-knowledge retention rate, reduces forgetting, and lowers the retention memory footprint over established continual learning baselines, providing an effective basis for non-stationary distributed settings. |
| (英) |
Distributed learning over non-stationary tasks must adapt to new tasks while preserving previously acquired knowledge. Existing approaches suffer from catastrophic forgetting, as continual updates progressively overwrite the representations learned from earlier tasks. We propose a continual distributed learning approach that unifies objective-similarity-aware partial model fusion with B-SCOR² (Benefit-optimal Sparse COordinate Retention and Regularization), which identifies and preserves the parameters most critical to prior tasks through a benefit-optimal selection criterion while leaving the remaining parameter space free to adapt to evolving data. B-SCOR² stores the retained values compactly through a tensor-train singular value decomposition (TT-SVD) and protects them with a selective regularization penalty, preserving old-task knowledge at a fraction of the memory cost of dense methods. On CIFAR-10, B-SCOR² improves prior-knowledge retention rate, reduces forgetting, and lowers the retention memory footprint over established continual learning baselines, providing an effective basis for non-stationary distributed settings. |
| キーワード |
(和) |
Continual learning / Catastrophic forgetting / Sparse coordinate retention / TT-SVD compression / Partial model fusion / Distributed learning / / |
| (英) |
Continual learning / Catastrophic forgetting / Sparse coordinate retention / TT-SVD compression / Partial model fusion / Distributed learning / / |
| 文献情報 |
信学技報, vol. 126, no. 143, CCS2026-31, pp. 30-35, 2026年8月. |
| 資料番号 |
CCS2026-31 |
| 発行日 |
2026-07-30 (CCS) |
| ISSN |
Online edition: ISSN 2432-6380 |
著作権に ついて |
技術研究報告に掲載された論文の著作権は電子情報通信学会に帰属します.(許諾番号:10GA0019/12GB0052/13GB0056/17GB0034/18GB0034) |
| PDFダウンロード |
CCS2026-31 |
|