In this paper, we present a virtual mirror system tracking a 3D position and orientation of a user's head in real-time. This system can exhibit a high-realistic color change of the human face. The “virtual mirror” is a system that can reproduce a mirror image of user's face onto the display by using a video camera and a half mirror. Unlike a normal mirror, virtual mirror has an advantage that various image processing can be performed to the video stream. Therefore, it is thought that the virtual mirror can be applied to various applications, for example, video chatting and electronic cosmetic and so on. Our goal is to create the electronic cosmetic system with the virtual mirror. The 3D head position and orientation are tracked to control the skin color on appropriate position in the face.