Recently, the user interface (UI) has become more important because devices have become complex and give us excessive information. In this study, we investigated “what type of knowledge users use” for information search. We tested the hypothesis that graphical UI users would utilize perceptual knowledge (the knowledge of the visual cues on the display) rather than structural knowledge (the knowledge of how information is categorized). The results of the experiment showed that participants acquired little structural knowledge. Further, the computer simulation results indicated that the model with only structural knowledge was inadequate to explain the search patterns observed in the experiment. The model that had perceptual knowledge was better able to explain the results of the experiment.