This paper proposes a nonverbal interface system using bio-potential signals, such as EOG and EMG, measured by a brain-computer interface and investigates its possibility of application to control of a hands-free manipulation system. A simple gesture recognition algorithm is presented to estimate the user's thinking from the EOG and EMG signals. To evaluate the feasibility and the characteristics of the interface system for hands-free manipulation, moving control experiments in 3D virtual space are carried out and the effectiveness of the proposed interface system is confirmed.