講演抄録/キーワード |
講演名 |
2013-12-02 17:10
NICTサイエンスクラウドによる衛星観測データの高速データ処理技技法 ○村田健史・渡邉英伸(NICT)・笠原禎也・矢木大介(金沢大)・笠井康子・石井昌憲・山本和憲(NICT)・木村映善(愛媛大)・建部修見・田中昌宏(筑波大)・鵜川健太郎・村永和哉・鈴木 豊(セック)・小嶋浩嗣(京大) SANE2013-94 |
抄録 |
(和) |
A variety of satellite missions, Earth observations and environments, Space physics and Astrophysics and Commercial uses, are carried out every year. Most of the satellites yield a large-scale data, and high-performance data processing technologies are expected. We have been developing a cloud system (the NICT Science Cloud) for big data analyses of Earth and Space observations via spacecraft. In the present study we propose a new technique to process large-scale data with paying attention to the fact that high-speed I/O (data file read and write) is important compared with data processing itself. We also adopt a task scheduler, the Pwrake, for easy management of parallel data processing. Using a long-time scientific satellite observation data (GEOTAIL satellite), we examine performance of the system on the NICT Science Cloud. We successfully archive high-speed data processing more than 100 times faster than those on traditional data processing environments. Since the design of the present system is for versatile use, we also discuss how to apply for other satellite missions such as GOSAT satellite, KAGUYA satellite and ISOSIM-L (a radar simulator). |
(英) |
A variety of satellite missions, Earth observations and environments, Space physics and Astrophysics and Commercial uses, are carried out every year. Most of the satellites yield a large-scale data, and high-performance data processing technologies are expected. We have been developing a cloud system (the NICT Science Cloud) for big data analyses of Earth and Space observations via spacecraft. In the present study we propose a new technique to process large-scale data with paying attention to the fact that high-speed I/O (data file read and write) is important compared with data processing itself. We also adopt a task scheduler, the Pwrake, for easy management of parallel data processing. Using a long-time scientific satellite observation data (GEOTAIL satellite), we examine performance of the system on the NICT Science Cloud. We successfully archive high-speed data processing more than 100 times faster than those on traditional data processing environments. Since the design of the present system is for versatile use, we also discuss how to apply for other satellite missions such as GOSAT satellite, KAGUYA satellite and ISOSIM-L (a radar simulator). |
キーワード |
(和) |
/ / / / / / / |
(英) |
Satellite data / Earth and space observation / data processing / NICT Science Cloud / parallel file system / Gfarm / Pwrake / |
文献情報 |
信学技報, vol. 113, no. 335, SANE2013-94, pp. 133-138, 2013年12月. |
資料番号 |
SANE2013-94 |
発行日 |
2013-11-25 (SANE) |
ISSN |
Print edition: ISSN 0913-5685 Online edition: ISSN 2432-6380 |
著作権に ついて |
技術研究報告に掲載された論文の著作権は電子情報通信学会に帰属します.(許諾番号:10GA0019/12GB0052/13GB0056/17GB0034/18GB0034) |
PDFダウンロード |
SANE2013-94 |