| Paper Abstract and Keywords |
| Presentation |
2023-02-21 11:00
A note on text prompt tuning in cross-modal image retrieval for a specific database
Huaying Zhang, Rintaro Yanagi, Ren Togo, Takahiro Ogawa, Miki Haseyama (Hokkaido Univ.) |
| Abstract |
(in Japanese) |
(See Japanese page) |
| (in English) |
With the development of storage devices and the Internet, the number of users creating personal image databases has increased. To effectively retrieve images from these databases, cross-modal image retrieval methods which allow users to retrieve images by entering a simple text query have been widely researched. Among these methods, several large cross-modal models pre-trained on huge amounts of image data have been proposed. However, to adapt these models to personal databases, it is required to store and update millions of parameters. This can be inefficient in terms of data storage in practice. To solve this problem, in this paper, we propose a novel image retrieval method that uses text prompt tuning to efficiently adapt the cross-modal model to a specific database without fine-tuning the model parameters. In the proposed method, we first construct a vector of several dimensions (prompt) and combine the prompt with the query vectorized to the same dimension. Then the combined vector and the candidate images are fed into the pre-trained cross-modal models. Finally, the prompt is optimized to achieve better retrieval accuracy. At the end of this paper, we have verified the effectiveness of the proposed method through experiments on the open dataset. |
| Keyword |
(in Japanese) |
(See Japanese page) |
| (in English) |
Image retrieval / Cross-modal retrieval / Prompt learning / / / / / |
| Reference Info. |
IEICE Tech. Rep. |
| Paper # |
|
| Date of Issue |
|
| ISSN |
Online edition: ISSN 2432-6380 |
| Download PDF |
|
| Conference Information |
| Committee |
IE ITS ITE-MMS ITE-ME ITE-AIT |
| Conference Date |
2023-02-21 - 2023-02-22 |
| Place (in Japanese) |
(See Japanese page) |
| Place (in English) |
Hokkaido Univ. |
| Topics (in Japanese) |
(See Japanese page) |
| Topics (in English) |
Image Processing, etc. |
| Paper Information |
| Registration To |
ITE-ME |
| Conference Code |
2023-02-MMS-ME-AIT-IE-ITS |
| Language |
Japanese |
| Title (in Japanese) |
(See Japanese page) |
| Sub Title (in Japanese) |
(See Japanese page) |
| Title (in English) |
A note on text prompt tuning in cross-modal image retrieval for a specific database |
| Sub Title (in English) |
|
| Keyword(1) |
Image retrieval |
| Keyword(2) |
Cross-modal retrieval |
| Keyword(3) |
Prompt learning |
| Keyword(4) |
|
| Keyword(5) |
|
| Keyword(6) |
|
| Keyword(7) |
|
| Keyword(8) |
|
| 1st Author's Name |
Huaying Zhang |
| 1st Author's Affiliation |
Hokkaido University (Hokkaido Univ.) |
| 2nd Author's Name |
Rintaro Yanagi |
| 2nd Author's Affiliation |
Hokkaido University (Hokkaido Univ.) |
| 3rd Author's Name |
Ren Togo |
| 3rd Author's Affiliation |
Hokkaido University (Hokkaido Univ.) |
| 4th Author's Name |
Takahiro Ogawa |
| 4th Author's Affiliation |
Hokkaido University (Hokkaido Univ.) |
| 5th Author's Name |
Miki Haseyama |
| 5th Author's Affiliation |
Hokkaido University (Hokkaido Univ.) |
| 6th Author's Name |
|
| 6th Author's Affiliation |
() |
| 7th Author's Name |
|
| 7th Author's Affiliation |
() |
| 8th Author's Name |
|
| 8th Author's Affiliation |
() |
| 9th Author's Name |
|
| 9th Author's Affiliation |
() |
| 10th Author's Name |
|
| 10th Author's Affiliation |
() |
| 11th Author's Name |
|
| 11th Author's Affiliation |
() |
| 12th Author's Name |
|
| 12th Author's Affiliation |
() |
| 13th Author's Name |
|
| 13th Author's Affiliation |
() |
| 14th Author's Name |
|
| 14th Author's Affiliation |
() |
| 15th Author's Name |
|
| 15th Author's Affiliation |
() |
| 16th Author's Name |
|
| 16th Author's Affiliation |
() |
| 17th Author's Name |
|
| 17th Author's Affiliation |
() |
| 18th Author's Name |
|
| 18th Author's Affiliation |
() |
| 19th Author's Name |
|
| 19th Author's Affiliation |
() |
| 20th Author's Name |
|
| 20th Author's Affiliation |
() |
| Speaker |
Author-1 |
| Date Time |
2023-02-21 11:00:00 |
| Presentation Time |
15 minutes |
| Registration for |
ITE-ME |
| Paper # |
|
| Volume (vol) |
vol.122 |
| Number (no) |
|
| Page |
|
| #Pages |
|
| Date of Issue |
|