| IEICE Technical Committee Submission System Conference Paper's Information |
Online Proceedings [Sign in] Tech. Rep. Archives |
| Go Top Page | Go Previous | [Japanese] / [English] |
| Paper Abstract and Keywords | ||
| Presentation | 2023-03-17 11:10 A Data Collection Protocol, Tool and Analysis for Avatar Facial Animation Ryosuke Miyawaki, Monica Perusquia-Hernandez, Naoya Isoyama, Hideaki Uchiyama, Kiyoshi Kiyokawa (NAIST)  IMQ2022-72 IE2022-149 MVE2022-102 |
|
| Abstract | (in Japanese) | (See Japanese page) |
| (in English) | Accurate facial displays are necessary to convey perceptual speech characteristics in avatar animations. Recently, data-driven methods using computer vision have been used to infer the relationship between facial movements and speech. We investigated the relationship between different speech volume levels and facial movements using blendshape-based tracking, which can be applied more directly and easily to an avatar animation. We found differences in facial movement around the nose, cheek, mouth, and head pitch at different speech volume levels, suggesting these are essential parts of the animation. We also present our experimental protocol for collecting speech audio, facial video, and blendshape-based facial movement data, as well as a WEB-based tool to support data collection and synchronization. In addition to this collection tool, we also provide a database of six Japanese participants reading emotion-related scripts at different volume levels. | |
| Keyword | (in Japanese) | (See Japanese page) |
| (in English) | Multimodal data collection / facial movement analysis / open-source tool / 3D avatar animation / / / / | |
| Reference Info. | IEICE Tech. Rep., vol. 122, no. 440, MVE2022-102, pp. 266-271, March 2023. | |
| Paper # | MVE2022-102 | |
| Date of Issue | 2023-03-08 (IMQ, IE, MVE) | |
| ISSN | Online edition: ISSN 2432-6380 | |
| Copyright and reproduction |
All rights are reserved and no part of this publication may be reproduced or transmitted in any form or by any means, electronic or mechanical, including photocopy, recording, or any information storage and retrieval system, without permission in writing from the publisher. Notwithstanding, instructors are permitted to photocopy isolated articles for noncommercial classroom use without fee. (License No.: 10GA0019/12GB0052/13GB0056/17GB0034/18GB0034) | |
| Download PDF | IMQ2022-72 IE2022-149 MVE2022-102 |
|
| Conference Information | |
| Committee | IMQ IE MVE CQ |
| Conference Date | 2023-03-15 - 2023-03-17 |
| Place (in Japanese) | (See Japanese page) |
| Place (in English) | Okinawaken Seinenkaikan (Naha-shi) |
| Topics (in Japanese) | (See Japanese page) |
| Topics (in English) | Media of five senses, Multimedia, Media experience, Picture codinge, Image media quality, Network,quality and reliability, etc（AC） |
| Paper Information | |
| Registration To | MVE |
| Conference Code | 2023-03-IMQ-IE-MVE-CQ |
| Language | English (Japanese title is available) |
| Title (in Japanese) | (See Japanese page) |
| Sub Title (in Japanese) | (See Japanese page) |
| Title (in English) | A Data Collection Protocol, Tool and Analysis for Avatar Facial Animation |
| Sub Title (in English) | |
| Keyword(1) | Multimodal data collection |
| Keyword(2) | facial movement analysis |
| Keyword(3) | open-source tool |
| Keyword(4) | 3D avatar animation |
| Keyword(5) | |
| Keyword(6) | |
| Keyword(7) | |
| Keyword(8) | |
| 1st Author's Name | Ryosuke Miyawaki |
| 1st Author's Affiliation | Nara Institute of Science and Technology (NAIST) |
| 2nd Author's Name | Monica Perusquia-Hernandez |
| 2nd Author's Affiliation | Nara Institute of Science and Technology (NAIST) |
| 3rd Author's Name | Naoya Isoyama |
| 3rd Author's Affiliation | Nara Institute of Science and Technology (NAIST) |
| 4th Author's Name | Hideaki Uchiyama |
| 4th Author's Affiliation | Nara Institute of Science and Technology (NAIST) |
| 5th Author's Name | Kiyoshi Kiyokawa |
| 5th Author's Affiliation | Nara Institute of Science and Technology (NAIST) |
| 6th Author's Name | |
| 6th Author's Affiliation | () |
| 7th Author's Name | |
| 7th Author's Affiliation | () |
| 8th Author's Name | |
| 8th Author's Affiliation | () |
| 9th Author's Name | |
| 9th Author's Affiliation | () |
| 10th Author's Name | |
| 10th Author's Affiliation | () |
| 11th Author's Name | |
| 11th Author's Affiliation | () |
| 12th Author's Name | |
| 12th Author's Affiliation | () |
| 13th Author's Name | |
| 13th Author's Affiliation | () |
| 14th Author's Name | |
| 14th Author's Affiliation | () |
| 15th Author's Name | |
| 15th Author's Affiliation | () |
| 16th Author's Name | |
| 16th Author's Affiliation | () |
| 17th Author's Name | |
| 17th Author's Affiliation | () |
| 18th Author's Name | |
| 18th Author's Affiliation | () |
| Speaker | Author-1 |
| Date Time | 2023-03-17 11:10:00 |
| Presentation Time | 25 minutes |
| Registration for | MVE |
| Paper # | IMQ2022-72, IE2022-149, MVE2022-102 |
| Volume (vol) | vol.122 |
| Number (no) | no.437(IMQ), no.439(IE), no.440(MVE) |
| Page | pp.266-271 |
| #Pages | 6 |
| Date of Issue | 2023-03-08 (IMQ, IE, MVE) |
[Return to Top Page]