This paper examined the psychological and physiological influences which two-color combinations of a blouse-skirt outfit had on the wearer, and investigated the impact of this psychological reaction on the physiological mechanism. The images of 15 subjects wearing different color blouse-skirt combinations were stored in a computer. Those same 15 subjects then viewed the images of the 15 different color combinations worn. While they viewed the images one by one, their brainwaves (α wave content) and fluctuations in heartbeat (HF component, LF/HF component) were measured, in order to examine the physiological influences. The SD method and the principal component analysis were employed for psychological evaluation. In the case of two-color combination of a blouse-skirt outfit, results showed that the higher the α wave content, the higher the evaluation. However, when the LF/HF components derived from the fluctuations of the heartbeat were low, psychologically active emotions arose. In the frontal region, thought to be related to emotions and aesthetic sensitivity, the α wave content decreased as a result of active emotions, and the evaluation decreased. The 1/f fluctuation also influenced the active emotions. Moreover, when the HF component was high, the evaluation decreased.