| 電子情報通信学会 研究会発表申込システム 講演論文 詳細 |
技報閲覧サービス [ログイン] 技報アーカイブ |
| トップに戻る | 前のページに戻る | [Japanese] / [English] |
| 講演抄録／キーワード | ||
| 講演名 | 2026-06-12 14:25 On the Impact of Residual Connections on Hierarchical Feature Learning in CNNs ○Jirayus Lapamnuaypol・Kenya Jin'no（TCU Univ.）  NLP2026-23 CCS2026-23 |
|
| 抄録 | （和） | Residual connections are a fundamental component of modern convolutional neural networks, enabling the successful optimization of deep architectures. While their benefits for gradient propagation and optimization are well established, their influence on hierarchical feature learning remains less explored. This study investigates how residual learning affects internal representation development in convolutional networks from two complementary perspectives. First, t-SNE visualization is applied to intermediate feature representations in ResNet-50 to examine layer-wise feature evolution and residual block behavior. The analysis reveals that while hierarchical abstraction is preserved globally, residual addition partially restores earlier feature manifold structures, indicating that skip connections modify strictly sequential refinement. Second, a dynamic activation function, Activity-Modulated ReLU (AMReLU), is introduced to probe layer-wise activity propagation through adaptive threshold modulation based on preceding-layer activations. Experiments on plain CNNs and ResNet-20 models trained on CIFAR-10 show that AMReLU produces structured sparsity dynamics in sequential architectures, while these dynamics become attenuated in residual networks due to feature mixing introduced by skip connections. These findings suggest that residual learning preserves intermediate representations across depth, resulting in a more iterative hierarchical refinement process compared to conventional feedforward CNNs. |
| （英） | Residual connections are a fundamental component of modern convolutional neural networks, enabling the successful optimization of deep architectures. While their benefits for gradient propagation and optimization are well established, their influence on hierarchical feature learning remains less explored. This study investigates how residual learning affects internal representation development in convolutional networks from two complementary perspectives. First, t-SNE visualization is applied to intermediate feature representations in ResNet-50 to examine layer-wise feature evolution and residual block behavior. The analysis reveals that while hierarchical abstraction is preserved globally, residual addition partially restores earlier feature manifold structures, indicating that skip connections modify strictly sequential refinement. Second, a dynamic activation function, Activity-Modulated ReLU (AMReLU), is introduced to probe layer-wise activity propagation through adaptive threshold modulation based on preceding-layer activations. Experiments on plain CNNs and ResNet-20 models trained on CIFAR-10 show that AMReLU produces structured sparsity dynamics in sequential architectures, while these dynamics become attenuated in residual networks due to feature mixing introduced by skip connections. These findings suggest that residual learning preserves intermediate representations across depth, resulting in a more iterative hierarchical refinement process compared to conventional feedforward CNNs. | |
| キーワード | （和） | Residual connection / t-SNE / feature Representation / / / / / |
| （英） | Residual connection / t-SNE / feature Representation / / / / / | |
| 文献情報 | 信学技報, vol. 126, no. 68, NLP2026-23, pp. 119-124, 2026年6月. | |
| 資料番号 | NLP2026-23 | |
| 発行日 | 2026-06-04 (NLP, CCS) | |
| ISSN | Online edition: ISSN 2432-6380 | |
| 著作権に ついて |
技術研究報告に掲載された論文の著作権は電子情報通信学会に帰属します．(許諾番号：10GA0019/12GB0052/13GB0056/17GB0034/18GB0034) | |
| PDFダウンロード | NLP2026-23 CCS2026-23 |
|
| 研究会情報 | |
| 研究会 | CCS NLP |
| 開催期間 | 2026-06-11 - 2026-06-12 |
| 開催地（和） | I-site なんば |
| 開催地（英） | I-site Namba |
| テーマ（和） | NLP，CCS，一般 |
| テーマ（英） | Nonlinear Problems, Complex Communication Sciences, etc. |
| 講演論文情報の詳細 | |
| 申込み研究会 | NLP |
| 会議コード | 2026-06-CCS-NLP |
| 本文の言語 | 英語 |
| タイトル（和） | |
| サブタイトル（和） | |
| タイトル（英） | On the Impact of Residual Connections on Hierarchical Feature Learning in CNNs |
| サブタイトル（英） | |
| キーワード(1)（和/英） | Residual connection / Residual connection |
| キーワード(2)（和/英） | t-SNE / t-SNE |
| キーワード(3)（和/英） | feature Representation / feature Representation |
| キーワード(4)（和/英） | / |
| キーワード(5)（和/英） | / |
| キーワード(6)（和/英） | / |
| キーワード(7)（和/英） | / |
| キーワード(8)（和/英） | / |
| 第1著者 氏名（和/英/ヨミ） | Jirayus Lapamnuaypol / Jirayus Lapamnuaypol / |
| 第1著者 所属（和/英） | Graduate School of Tokyo City University (略称： TCU Univ.) Graduate School of Tokyo City University (略称： TCU Univ.) |
| 第2著者 氏名（和/英/ヨミ） | Kenya Jin'no / Kenya Jin'no / |
| 第2著者 所属（和/英） | Graduate School of Tokyo City University (略称： TCU Univ.) Graduate School of Tokyo City University (略称： TCU Univ.) |
| 講演者 | 第1著者 |
| 発表日時 | 2026-06-12 14:25:00 |
| 発表時間 | 25分 |
| 申込先研究会 | NLP |
| 資料番号 | NLP2026-23, CCS2026-23 |
| 巻番号（vol） | vol.126 |
| 号番号（no） | no.68(NLP), no.69(CCS) |
| ページ範囲 | pp.119-124 |
| ページ数 | 6 |
| 発行日 | 2026-06-04 (NLP, CCS) |
[研究会発表申込システムのトップページに戻る]