Paper Abstract and Keywords |
Presentation |
2023-09-12 15:55
Estimation of unmasked face images based on voice and 3DMM Tetsumaru Akatsuka, Ryohei Orihara, Yuichi Sei, Yasuyuki Tahara, Akihiko Ohsuga (UEC) AI2023-32 |
Abstract |
(in Japanese) |
(See Japanese page) |
(in English) |
Facemasks have become common due to the COVID-19 pandemic. They have begun to affect security and identification systems because they cover almost half of the face. Current state-of-the-art methods have been applied to estimate unmasked faces from masked face images. They are successful in improving the quality of the face texture by 3D Morphable Model (3DMM) as intermediate representations. However, their performance in restoring the face shapes is insufficient, and some of generated faces lack identities. In this study, we focus on voice, which has a particularly high correlation with the shape of the mouth and nose, which are obscured by masks. We propose a multimodal method using 3DMM and voice for face shape estimation under masks. Experimental results show that the proposed method qualitatively and quantitatively improves the quality of shape restoration of a face compared to the baseline method without considering voice. |
Keyword |
(in Japanese) |
(See Japanese page) |
(in English) |
Mask Removal / Inpainting / 3DMM / Voice Embedding / Multimodal / / / |
Reference Info. |
IEICE Tech. Rep., vol. 123, no. 190, AI2023-32, pp. 187-193, Sept. 2023. |
Paper # |
AI2023-32 |
Date of Issue |
2023-09-05 (AI) |
ISSN |
Online edition: ISSN 2432-6380 |
Copyright and reproduction |
All rights are reserved and no part of this publication may be reproduced or transmitted in any form or by any means, electronic or mechanical, including photocopy, recording, or any information storage and retrieval system, without permission in writing from the publisher. Notwithstanding, instructors are permitted to photocopy isolated articles for noncommercial classroom use without fee. (License No.: 10GA0019/12GB0052/13GB0056/17GB0034/18GB0034) |
Download PDF |
AI2023-32 |
|