IEICE Technical Committee Submission System
Conference Paper's Information
Online Proceedings
[Sign in]
Tech. Rep. Archives
 Go Top Page Go Previous   [Japanese] / [English] 

Paper Abstract and Keywords
Presentation 2024-03-13 10:00
Estimation of Perceived Difficulty when reading VR-based Educational Comics Using Gaze and Facial Movement, Heart Rate, and Electroencephalography
Hiroyuki Ishizuka (NAIST), Kenya Sakamoto, Shizuka Shirai, Jason Orlosky (OU), Yutaro Hirao, Monica Perusquia-Hernandez (NAIST), Naoya Isoyama (OWU), Hideaki Uchiyama, Kiyoshi Kiyokawa (NAIST) IMQ2023-13 IE2023-68 MVE2023-42
Abstract (in Japanese) (See Japanese page) 
(in English) This study explores significant biometric features that can be used to estimate subjective difficulty while reading educational cartoon materials. Although educational cartoons are effective teaching materials for learning a wide range of subjects, it is difficult to detect the learner's state using a log data-based approach. Therefore, we propose a subjective difficulty estimation method using eye gaze, face, electrocardiogram, and electroencephalogram to support learning when reading manga materials in VR(Virtual Reality) space. In addition to gaze features used in previous studies, the proposed method uses biometric information such as the amount of facial expression change, heart rate variability, and alpha and theta waves of EEG as features. To verify the effectiveness of the proposed method, we measured the learners' data during reading manga materials in a VR space, respectively, and performed subjective difficulty estimation using support vector machines and random forests. As a result, Accuracy and F1 scores of 0.965 and 0.940 were achieved by the Random Forest (RF) approach using facial expression and electrocardiogram features, respectively.
Keyword (in Japanese) (See Japanese page) 
(in English) Cognitive state recognition / eye tracking / facial expressions / ECG / EEG / Education / /  
Reference Info. IEICE Tech. Rep., vol. 123, no. 433, MVE2023-42, pp. 1-6, March 2024.
Paper # MVE2023-42 
Date of Issue 2024-03-06 (IMQ, IE, MVE) 
ISSN Online edition: ISSN 2432-6380
All rights are reserved and no part of this publication may be reproduced or transmitted in any form or by any means, electronic or mechanical, including photocopy, recording, or any information storage and retrieval system, without permission in writing from the publisher. Notwithstanding, instructors are permitted to photocopy isolated articles for noncommercial classroom use without fee. (License No.: 10GA0019/12GB0052/13GB0056/17GB0034/18GB0034)
Download PDF IMQ2023-13 IE2023-68 MVE2023-42

Conference Information
Committee IE MVE CQ IMQ  
Conference Date 2024-03-13 - 2024-03-15 
Place (in Japanese) (See Japanese page) 
Place (in English) Okinawa Sangyo Shien Center 
Topics (in Japanese) (See Japanese page) 
Topics (in English) Media of five senses, Multimedia, Media experience, Picture codinge, Image media quality, Network,quality and reliability, etc(AC) 
Paper Information
Registration To MVE 
Conference Code 2024-03-IE-MVE-CQ-IMQ 
Language Japanese 
Title (in Japanese) (See Japanese page) 
Sub Title (in Japanese) (See Japanese page) 
Title (in English) Estimation of Perceived Difficulty when reading VR-based Educational Comics Using Gaze and Facial Movement, Heart Rate, and Electroencephalography 
Sub Title (in English)  
Keyword(1) Cognitive state recognition  
Keyword(2) eye tracking  
Keyword(3) facial expressions  
Keyword(4) ECG  
Keyword(5) EEG  
Keyword(6) Education  
1st Author's Name Hiroyuki Ishizuka  
1st Author's Affiliation Nara Institute of Science and Technology (NAIST)
2nd Author's Name Kenya Sakamoto  
2nd Author's Affiliation Osaka University (OU)
3rd Author's Name Shizuka Shirai  
3rd Author's Affiliation Osaka University (OU)
4th Author's Name Jason Orlosky  
4th Author's Affiliation Osaka University (OU)
5th Author's Name Yutaro Hirao  
5th Author's Affiliation Nara Institute of Science and Technology (NAIST)
6th Author's Name Monica Perusquia-Hernandez  
6th Author's Affiliation Nara Institute of Science and Technology (NAIST)
7th Author's Name Naoya Isoyama  
7th Author's Affiliation Otsuma Women's University (OWU)
8th Author's Name Hideaki Uchiyama  
8th Author's Affiliation Nara Institute of Science and Technology (NAIST)
9th Author's Name Kiyoshi Kiyokawa  
9th Author's Affiliation Nara Institute of Science and Technology (NAIST)
10th Author's Name  
10th Author's Affiliation ()
11th Author's Name  
11th Author's Affiliation ()
12th Author's Name  
12th Author's Affiliation ()
13th Author's Name  
13th Author's Affiliation ()
14th Author's Name  
14th Author's Affiliation ()
15th Author's Name  
15th Author's Affiliation ()
16th Author's Name  
16th Author's Affiliation ()
17th Author's Name  
17th Author's Affiliation ()
18th Author's Name  
18th Author's Affiliation ()
19th Author's Name  
19th Author's Affiliation ()
20th Author's Name  
20th Author's Affiliation ()
Speaker Author-1 
Date Time 2024-03-13 10:00:00 
Presentation Time 20 minutes 
Registration for MVE 
Paper # IMQ2023-13, IE2023-68, MVE2023-42 
Volume (vol) vol.123 
Number (no) no.430(IMQ), no.432(IE), no.433(MVE) 
Page pp.1-6 
Date of Issue 2024-03-06 (IMQ, IE, MVE) 

[Return to Top Page]

[Return to IEICE Web Page]

The Institute of Electronics, Information and Communication Engineers (IEICE), Japan