Paper Abstract and Keywords |
Presentation |
2013-06-17 14:35
High Performance of Big Data Processing via NICT Science Cloud Fusako Isoda (Science Service), Ken T. Murata, Hidenobu Watanabe (NICT), Keiichiro Fukazawa (Kyushu Univ.), Kazunori Yamamoto (NICT), Osamu Tatebe, Masahiro Tanaka (Tsukuba Univ.), Eizen Kimura (Ehime Univ.) SC2013-4 |
Abstract |
(in Japanese) |
(See Japanese page) |
(in English) |
Science cloud is a cloud system designed for scientific researches, and expected as a new infrastructure for big data sciences. Not only parallelization of CPU as in super-computers, but I/O and network throughput parallelization are crucial for the big data science. One of the typical structures of science cloud is a scalable cluster in which multiple clusters in a cloud are connected with high-speed network. In the present study, we study a performance of parallelization of both CPU and I/O inside a cloud as a first step to the high performance scalable clusters. In case with few processes executed on each computational node (server), parallelization efficiency is almost 100%. This high efficiency is expected to maintain in larger-scale cluster systems such as those with 100 servers. On the other hand, under the condition of multi-processes on each node, the present parallelization does not show good performance due to the congestions of I/O. Parallelization efficiency is as low as 15.6. New techniques of decentralization of I/O within each node are required in the next step. |
Keyword |
(in Japanese) |
(See Japanese page) |
(in English) |
ScienceCloud / Gfarm / Pwrake / Big data / Magnetspheric MHD simulation / / / |
Reference Info. |
IEICE Tech. Rep., vol. 113, no. 86, SC2013-4, pp. 19-24, June 2013. |
Paper # |
SC2013-4 |
Date of Issue |
2013-06-10 (SC) |
ISSN |
Print edition: ISSN 0913-5685 Online edition: ISSN 2432-6380 |
Copyright and reproduction |
All rights are reserved and no part of this publication may be reproduced or transmitted in any form or by any means, electronic or mechanical, including photocopy, recording, or any information storage and retrieval system, without permission in writing from the publisher. Notwithstanding, instructors are permitted to photocopy isolated articles for noncommercial classroom use without fee. (License No.: 10GA0019/12GB0052/13GB0056/17GB0034/18GB0034) |
Download PDF |
SC2013-4 |
|