| IEICE Technical Committee Submission System Conference Paper's Information |
Online Proceedings [Sign in] Tech. Rep. Archives |
| Go Top Page | Go Previous | [Japanese] / [English] |
| Paper Abstract and Keywords | ||
| Presentation | 2023-02-22 13:15 Semantic-Consistent Style Transfer with Visual Transformers Jianbo Wang (UTokyo), Huan Yang, Jianlong Fu (MSR), Toshihiko Yamasaki (UTokyo), Baining Guo (MSR)  ITS2022-68 IE2022-85 |
|
| Abstract | (in Japanese) | (See Japanese page) |
| (in English) | Image style transfer has drawn increasing attention recently. This task takes a content image and a style image as inputs and generates an output image with the desired style while preserving the structure from the content image. However, most existing approaches adopt a global feature transformation to transfer style patterns into content images (e.g., AdaIN and WCT). Such a design usually destroys the spatial information of the input images and fails to keep semantic consistency between the content image and the stylized results. To solve this problem, we propose a novel STyle TRansformer (STTR) network which breaks both content and style images into visual tokens to achieve a fine-grained style transformation. Specifically, two attention mechanisms are adopted in our STTR. We first propose to use self-attention to encode content and style tokens such that similar tokens can be grouped and learned together. We then adopt cross-attention between content and style tokens that encourages fine-grained style transformations. As similar patches lean to match with the same style code, semantic-consistent style transfer can be obtained. To compare STTR with existing approaches, we conduct user studies on Amazon Mechanical Turk (AMT), which are carried out with 50 human subjects with 1,000 votes in total. Extensive evaluations demonstrate the effectiveness and efficiency of the proposed STTR in generating visually pleasing style transfer results | |
| Keyword | (in Japanese) | (See Japanese page) |
| (in English) | image style transfer / visual transformer / / / / / / | |
| Reference Info. | IEICE Tech. Rep., vol. 122, no. 385, IE2022-85, pp. 147-152, Feb. 2023. | |
| Paper # | IE2022-85 | |
| Date of Issue | 2023-02-14 (ITS, IE) | |
| ISSN | Online edition: ISSN 2432-6380 | |
| Copyright and reproduction |
All rights are reserved and no part of this publication may be reproduced or transmitted in any form or by any means, electronic or mechanical, including photocopy, recording, or any information storage and retrieval system, without permission in writing from the publisher. Notwithstanding, instructors are permitted to photocopy isolated articles for noncommercial classroom use without fee. (License No.: 10GA0019/12GB0052/13GB0056/17GB0034/18GB0034) | |
| Download PDF | ITS2022-68 IE2022-85 |
|
| Conference Information | |
| Committee | IE ITS ITE-MMS ITE-ME ITE-AIT |
| Conference Date | 2023-02-21 - 2023-02-22 |
| Place (in Japanese) | (See Japanese page) |
| Place (in English) | Hokkaido Univ. |
| Topics (in Japanese) | (See Japanese page) |
| Topics (in English) | Image Processing, etc. |
| Paper Information | |
| Registration To | IE |
| Conference Code | 2023-02-IE-ITS-MMS-ME-AIT |
| Language | English |
| Title (in Japanese) | (See Japanese page) |
| Sub Title (in Japanese) | (See Japanese page) |
| Title (in English) | Semantic-Consistent Style Transfer with Visual Transformers |
| Sub Title (in English) | |
| Keyword(1) | image style transfer |
| Keyword(2) | visual transformer |
| Keyword(3) | |
| Keyword(4) | |
| Keyword(5) | |
| Keyword(6) | |
| Keyword(7) | |
| Keyword(8) | |
| 1st Author's Name | Jianbo Wang |
| 1st Author's Affiliation | The University of Tokyo (UTokyo) |
| 2nd Author's Name | Huan Yang |
| 2nd Author's Affiliation | Mircrosoft Research (MSR) |
| 3rd Author's Name | Jianlong Fu |
| 3rd Author's Affiliation | Mircrosoft Research (MSR) |
| 4th Author's Name | Toshihiko Yamasaki |
| 4th Author's Affiliation | The University of Tokyo (UTokyo) |
| 5th Author's Name | Baining Guo |
| 5th Author's Affiliation | Mircrosoft Research (MSR) |
| 6th Author's Name | |
| 6th Author's Affiliation | () |
| 7th Author's Name | |
| 7th Author's Affiliation | () |
| 8th Author's Name | |
| 8th Author's Affiliation | () |
| 9th Author's Name | |
| 9th Author's Affiliation | () |
| 10th Author's Name | |
| 10th Author's Affiliation | () |
| 11th Author's Name | |
| 11th Author's Affiliation | () |
| 12th Author's Name | |
| 12th Author's Affiliation | () |
| 13th Author's Name | |
| 13th Author's Affiliation | () |
| 14th Author's Name | |
| 14th Author's Affiliation | () |
| 15th Author's Name | |
| 15th Author's Affiliation | () |
| 16th Author's Name | |
| 16th Author's Affiliation | () |
| 17th Author's Name | |
| 17th Author's Affiliation | () |
| 18th Author's Name | |
| 18th Author's Affiliation | () |
| Speaker | Author-1 |
| Date Time | 2023-02-22 13:15:00 |
| Presentation Time | 15 minutes |
| Registration for | IE |
| Paper # | ITS2022-68, IE2022-85 |
| Volume (vol) | vol.122 |
| Number (no) | no.384(ITS), no.385(IE) |
| Page | pp.147-152 |
| #Pages | 6 |
| Date of Issue | 2023-02-14 (ITS, IE) |
[Return to Top Page]