| IEICE Technical Committee Submission System Conference Paper's Information |
Online Proceedings [Sign in] Tech. Rep. Archives |
| Go Top Page | Go Previous | [Japanese] / [English] |
| Paper Abstract and Keywords | ||
| Presentation | 2009-03-12 15:15 Semiparametric Statistical Approach to Value Function Estimation Tsuyoshi Ueno (Kyoto Univ.), Motoaki Kawanabe (Fraunhofer First), Takeshi Mori, Shin-ichi Maeda, Shin Ishii (Kyoto Univ.)  NC2008-146 |
|
| Abstract | (in Japanese) | (See Japanese page) |
| (in English) | Recently least-squares temporal difference (LSTD) learning has been developed for the model-free value function estimation and has shown good practical performance. However, the statistical efficiency of its estimation has not been well elucidated. In this study, we discuss LSTD-based policy evaluation from a viewpoint of semiparametric statistical inference. When the true value function can be represented by a parametric model, a consistent estimator of the value function is obtained by using a particular estimating function without knowing the whole data distribution, such as state transition probability or reward probability. Especially, we 1) analyze the asymptotic variance of the LSTD-based estimator, 2) derive the optimal estimating function with the minimum asymptotic estimation variance, and 3) derive the suboptimal estimator to reduce the computational cost when obtaining the optimal estimating function. |
|
| Keyword | (in Japanese) | (See Japanese page) |
| (in English) | reinforcement learning / instrumental variable method / least square temporal difference learning / estimating function / semiparametric model / policy evaluation / / | |
| Reference Info. | IEICE Tech. Rep., vol. 108, no. 480, NC2008-146, pp. 255-260, March 2009. | |
| Paper # | NC2008-146 | |
| Date of Issue | 2009-03-04 (NC) | |
| ISSN | Print edition: ISSN 0913-5685 Online edition: ISSN 2432-6380 | |
| Copyright and reproduction |
All rights are reserved and no part of this publication may be reproduced or transmitted in any form or by any means, electronic or mechanical, including photocopy, recording, or any information storage and retrieval system, without permission in writing from the publisher. Notwithstanding, instructors are permitted to photocopy isolated articles for noncommercial classroom use without fee. (License No.: 10GA0019/12GB0052/13GB0056/17GB0034/18GB0034) | |
| Download PDF | NC2008-146 |
|
| Conference Information | |
| Committee | NC MBE |
| Conference Date | 2009-03-11 - 2009-03-13 |
| Place (in Japanese) | (See Japanese page) |
| Place (in English) | Tamagawa Univ. |
| Topics (in Japanese) | (See Japanese page) |
| Topics (in English) | |
| Paper Information | |
| Registration To | NC |
| Conference Code | 2009-03-NC-MBE |
| Language | Japanese |
| Title (in Japanese) | (See Japanese page) |
| Sub Title (in Japanese) | (See Japanese page) |
| Title (in English) | Semiparametric Statistical Approach to Value Function Estimation |
| Sub Title (in English) | |
| Keyword(1) | reinforcement learning |
| Keyword(2) | instrumental variable method |
| Keyword(3) | least square temporal difference learning |
| Keyword(4) | estimating function |
| Keyword(5) | semiparametric model |
| Keyword(6) | policy evaluation |
| Keyword(7) | |
| Keyword(8) | |
| 1st Author's Name | Tsuyoshi Ueno |
| 1st Author's Affiliation | Kyoto University (Kyoto Univ.) |
| 2nd Author's Name | Motoaki Kawanabe |
| 2nd Author's Affiliation | Fraunhofer First (Fraunhofer First) |
| 3rd Author's Name | Takeshi Mori |
| 3rd Author's Affiliation | Kyoto University (Kyoto Univ.) |
| 4th Author's Name | Shin-ichi Maeda |
| 4th Author's Affiliation | Kyoto University (Kyoto Univ.) |
| 5th Author's Name | Shin Ishii |
| 5th Author's Affiliation | Kyoto University (Kyoto Univ.) |
| 6th Author's Name | |
| 6th Author's Affiliation | () |
| 7th Author's Name | |
| 7th Author's Affiliation | () |
| 8th Author's Name | |
| 8th Author's Affiliation | () |
| 9th Author's Name | |
| 9th Author's Affiliation | () |
| 10th Author's Name | |
| 10th Author's Affiliation | () |
| 11th Author's Name | |
| 11th Author's Affiliation | () |
| 12th Author's Name | |
| 12th Author's Affiliation | () |
| 13th Author's Name | |
| 13th Author's Affiliation | () |
| 14th Author's Name | |
| 14th Author's Affiliation | () |
| 15th Author's Name | |
| 15th Author's Affiliation | () |
| 16th Author's Name | |
| 16th Author's Affiliation | () |
| 17th Author's Name | |
| 17th Author's Affiliation | () |
| 18th Author's Name | |
| 18th Author's Affiliation | () |
| Speaker | Author-1 |
| Date Time | 2009-03-12 15:15:00 |
| Presentation Time | 25 minutes |
| Registration for | NC |
| Paper # | NC2008-146 |
| Volume (vol) | vol.108 |
| Number (no) | no.480 |
| Page | pp.255-260 |
| #Pages | 6 |
| Date of Issue | 2009-03-04 (NC) |
[Return to Top Page]