A human tracking system based on mobile agent technologies has been proposed for achieving automatic human tracking function. In this system, target persons are tracked automatically by a mobile agent moving among sensors in which a person is detected. The current system utilizes an algorithm to predict which sensor detects the target person. However, in order to keep tracking without missing person, it will be necessary to deal with the uncertainty of sensors. In this paper, we propose a method which corresponds to the uncertainty of sensors by finding hidden neighbor relations based on the detection result of sensors.