In order to find a method of estimating the video viewers’ emotion, we analyze video viewers’ emotions using pupil size, background activity of brain, and R-R interval. We apply principal component analysis to these data, and classify them based on the principal component score by using k-means clustering. The result of the classification shows that each cluster has some relationship corresponding to specific questionnaire responses, which suggests the possibility of emotion classification and emotion estimation by the proposed method.