出版社：The Institute of Image Information and Television Engineers
摘要：Relating audio-visual events is important for constructing an artificial intelligent system, which can acquire the audio-visual knowledge of moving objects through active observation without a supervisor. This paper proposes a method for relating multiple audio-visual events observed by a camera and two microphones in the scene that a moving object is not in the view of the camera. As corresponding cues, we use Gestalt's grouping law : simultaneity of sound onsets and changes in movement, similarity of repetition between sound and movement, similarity of direction between sound and movement. Based on the correlation coefficient between sound onset series and STI time series, the component of frequency at sound onset is related to the spatiotemporal invariant sequence (STI sequence) of movement. We experimented in the real environment and obtained satisfactory results showing the effectiveness of the proposed method.
