| IEICE Technical Committee Submission System Conference Paper's Information |
Online Proceedings [Sign in] Tech. Rep. Archives |
| Go Top Page | Go Previous | [Japanese] / [English] |
| Paper Abstract and Keywords | ||
| Presentation | 2021-03-04 16:10 Estimation of imagined speech from electrocorticogram with an encoder-decoder model Kotaro Hayashi, Shuji Komeiji (TUAT), Takumi Mitsuhashi, Yasushi Iimura, Hiroharu Suzuki, Hidenori Sugano (Juntendo Univ.), Koichi Shinoda (TokyoTech), Toshihisa Tanaka (TUAT)  EA2020-87 SIP2020-118 SP2020-52 |
|
| Abstract | (in Japanese) | (See Japanese page) |
| (in English) | Recent advances in signal processing and machine learning technologies have made it possible to estimate and reconstruct speech or text during speaking and listening from invasive electrocorticogram (ECoG). Meanwhile, the estimation of imagined speech has not been successful due to the difficulty in synchronizing the ECoG with the target label. In this paper, we hypothesize that if imagined speech and ECoG are adequately synchronized, speech can be decoded from ECoG as in the case of ECoG during speaking and listening. We designed an experiment in which participants were presented with a screen on which short sentences were projected, and by highlighting the colors of the letters one by one, the timing and speed of imagination could be controlled. The ECoG was recorded during the three tasks of imagining speech, listening to speech, and speaking. Moreover, we recorded the speech presented in the listening task and the participants' speech in the speaking task. From the measured ECoG, we built an encoder-decoder model using the Mel cepstrum coefficients of the speech during the speaking and listening tasks to infer the imagined speech. The best character error rate of about 17% was achieved for sentences decoded from the imagined ECoG. | |
| Keyword | (in Japanese) | (See Japanese page) |
| (in English) | Electrocorticogram (ECoG) / Imaginary speech / Brain-computer interface (BCI) / Encoder-decoder model / / / / | |
| Reference Info. | IEICE Tech. Rep., vol. 120, no. 398, SIP2020-118, pp. 164-169, March 2021. | |
| Paper # | SIP2020-118 | |
| Date of Issue | 2021-02-24 (EA, SIP, SP) | |
| ISSN | Online edition: ISSN 2432-6380 | |
| Copyright and reproduction |
All rights are reserved and no part of this publication may be reproduced or transmitted in any form or by any means, electronic or mechanical, including photocopy, recording, or any information storage and retrieval system, without permission in writing from the publisher. Notwithstanding, instructors are permitted to photocopy isolated articles for noncommercial classroom use without fee. (License No.: 10GA0019/12GB0052/13GB0056/17GB0034/18GB0034) | |
| Download PDF | EA2020-87 SIP2020-118 SP2020-52 |
|
| Conference Information | |
| Committee | EA US SP SIP IPSJ-SLP |
| Conference Date | 2021-03-03 - 2021-03-04 |
| Place (in Japanese) | (See Japanese page) |
| Place (in English) | Online |
| Topics (in Japanese) | (See Japanese page) |
| Topics (in English) | Speech, Engineering/Electro Acoustics, Signal Processing, Ultrasonics, and Related Topics |
| Paper Information | |
| Registration To | SIP |
| Conference Code | 2021-03-EA-US-SP-SIP-SLP |
| Language | Japanese |
| Title (in Japanese) | (See Japanese page) |
| Sub Title (in Japanese) | (See Japanese page) |
| Title (in English) | Estimation of imagined speech from electrocorticogram with an encoder-decoder model |
| Sub Title (in English) | |
| Keyword(1) | Electrocorticogram (ECoG) |
| Keyword(2) | Imaginary speech |
| Keyword(3) | Brain-computer interface (BCI) |
| Keyword(4) | Encoder-decoder model |
| Keyword(5) | |
| Keyword(6) | |
| Keyword(7) | |
| Keyword(8) | |
| 1st Author's Name | Kotaro Hayashi |
| 1st Author's Affiliation | Tokyo University of Agriculture and Technology (TUAT) |
| 2nd Author's Name | Shuji Komeiji |
| 2nd Author's Affiliation | Tokyo University of Agriculture and Technology (TUAT) |
| 3rd Author's Name | Takumi Mitsuhashi |
| 3rd Author's Affiliation | Juntendo University (Juntendo Univ.) |
| 4th Author's Name | Yasushi Iimura |
| 4th Author's Affiliation | Juntendo University (Juntendo Univ.) |
| 5th Author's Name | Hiroharu Suzuki |
| 5th Author's Affiliation | Juntendo University (Juntendo Univ.) |
| 6th Author's Name | Hidenori Sugano |
| 6th Author's Affiliation | Juntendo University (Juntendo Univ.) |
| 7th Author's Name | Koichi Shinoda |
| 7th Author's Affiliation | Tokyo Institute of Technology (TokyoTech) |
| 8th Author's Name | Toshihisa Tanaka |
| 8th Author's Affiliation | Tokyo University of Agriculture and Technology (TUAT) |
| 9th Author's Name | |
| 9th Author's Affiliation | () |
| 10th Author's Name | |
| 10th Author's Affiliation | () |
| 11th Author's Name | |
| 11th Author's Affiliation | () |
| 12th Author's Name | |
| 12th Author's Affiliation | () |
| Speaker | Author-1 |
| Date Time | 2021-03-04 16:10:00 |
| Presentation Time | 25 minutes |
| Registration for | SIP |
| Paper # | EA2020-87, SIP2020-118, SP2020-52 |
| Volume (vol) | vol.120 |
| Number (no) | no.397(EA), no.398(SIP), no.399(SP) |
| Page | pp.164-169 |
| #Pages | 6 |
| Date of Issue | 2021-02-24 (EA, SIP, SP) |
[Return to Top Page]