| IEICE Technical Committee Submission System Conference Paper's Information |
Online Proceedings [Sign in] Tech. Rep. Archives |
| Go Top Page | Go Previous | [Japanese] / [English] |
| Paper Abstract and Keywords | ||
| Presentation | 2021-09-16 16:15 Construction of Japanese Essay Data for Evaluating Automatic Essay Scoring System Koichi Takeuchi, Masahiro Taguchi, Yoshihiko Inada, Masaya Iizuka, Tasuhiko Abo, Hitoshi Ueda (Okayama Univ.)  NLC2021-15 |
|
| Abstract | (in Japanese) | (See Japanese page) |
| (in English) | We construct Japanese essay data for evaluating automatic essay scoring system. The essays are written by Japanese native speakers for the wide range prompts and all the essays are graded by annotators. In this paper, we give an overview of the essay data and discuss what factors were considered in manual evaluation to clarify the issues involved in creating scored essay data. In manual evaluation, we set four evaluation criteria: comprehension, logic, validity, and writing ability. With a rubric based on the criteria, two raters assign scores to the essays. With evaluation results of inter-rater agreement for prompts, we discuss the stability of human rating on each evaluation criteria depending on the characteristics of prompts. | |
| Keyword | (in Japanese) | (See Japanese page) |
| (in English) | automatic essay scoring / Japanese native speakers / evaluation criteria / / / / / | |
| Reference Info. | IEICE Tech. Rep., vol. 121, no. 178, NLC2021-15, pp. 40-44, Sept. 2021. | |
| Paper # | NLC2021-15 | |
| Date of Issue | 2021-09-09 (NLC) | |
| ISSN | Online edition: ISSN 2432-6380 | |
| Copyright and reproduction |
All rights are reserved and no part of this publication may be reproduced or transmitted in any form or by any means, electronic or mechanical, including photocopy, recording, or any information storage and retrieval system, without permission in writing from the publisher. Notwithstanding, instructors are permitted to photocopy isolated articles for noncommercial classroom use without fee. (License No.: 10GA0019/12GB0052/13GB0056/17GB0034/18GB0034) | |
| Download PDF | NLC2021-15 |
|
| Conference Information | |
| Committee | NLC |
| Conference Date | 2021-09-16 - 2021-09-16 |
| Place (in Japanese) | (See Japanese page) |
| Place (in English) | Online |
| Topics (in Japanese) | (See Japanese page) |
| Topics (in English) | The 18th Text Analytics Symposium |
| Paper Information | |
| Registration To | NLC |
| Conference Code | 2021-09-NLC |
| Language | Japanese |
| Title (in Japanese) | (See Japanese page) |
| Sub Title (in Japanese) | (See Japanese page) |
| Title (in English) | Construction of Japanese Essay Data for Evaluating Automatic Essay Scoring System |
| Sub Title (in English) | |
| Keyword(1) | automatic essay scoring |
| Keyword(2) | Japanese native speakers |
| Keyword(3) | evaluation criteria |
| Keyword(4) | |
| Keyword(5) | |
| Keyword(6) | |
| Keyword(7) | |
| Keyword(8) | |
| 1st Author's Name | Koichi Takeuchi |
| 1st Author's Affiliation | Okayama University (Okayama Univ.) |
| 2nd Author's Name | Masahiro Taguchi |
| 2nd Author's Affiliation | Okayama University (Okayama Univ.) |
| 3rd Author's Name | Yoshihiko Inada |
| 3rd Author's Affiliation | Okayama University (Okayama Univ.) |
| 4th Author's Name | Masaya Iizuka |
| 4th Author's Affiliation | Okayama University (Okayama Univ.) |
| 5th Author's Name | Tasuhiko Abo |
| 5th Author's Affiliation | Okayama University (Okayama Univ.) |
| 6th Author's Name | Hitoshi Ueda |
| 6th Author's Affiliation | Okayama University (Okayama Univ.) |
| Speaker | Author-1 |
| Date Time | 2021-09-16 16:15:00 |
| Presentation Time | 25 minutes |
| Registration for | NLC |
| Paper # | NLC2021-15 |
| Volume (vol) | vol.121 |
| Number (no) | no.178 |
| Page | pp.40-44 |
| #Pages | 5 |
| Date of Issue | 2021-09-09 (NLC) |
[Return to Top Page]