Early processing stages in the perception of familiar and unfamiliar faces were studied in four experiments by varying the type of available facial information in a four alternative forced choice recognition task. Both reaction time and recognition accuracy served as dependent measures. The observed data revealed an asymmetry in processing familiar and unfamiliar faces. A markedly weak inversion effect and strong blurring effect suggest a limited usage of spatial relations within early processing stages of unfamiliar faces. Recognition performance for whole familiar faces did not deteriorate due to blurring or the presentation of isolated internal features, suggesting a low level of representation for featural properties of familiar faces. Based on the data we propose that recognition of familiar faces relies much more on spatial relations among features, particularly internal features, than on featural characteristics. In contrast, recognition of unfamiliar faces resorts mainly to featural information.