We propose a reliable approach to detect skin regions that can be used in various human-related image processing applications. We use a color distance map, which itself is a grayscale image making the process simple, but still containing color information. Based on this map, we generate some skin as well as nonskin seed pixels, and then grow them to capture the appropriate regions. This approach outperforms the existing approaches in terms of segmenting solid and perfect skin regions. It does not generate much noisy segments. Moreover, it does not need any prior training session and can adapt to detect skin regions from images taken at different imaging conditions.