This paper presents a sound source (talker) localization method using only a single microphone, where a Gaussian Mixture Model (GMM) of clean speech is introduced to estimate the acoustic transfer function from a user’s position. The new method is able to carry out this estimation without measuring impulse responses. The frame sequence of the acoustic transfer function is estimated by maximizing the likelihood of training data uttered from a given position, where the cepstral parameters are used to effectively represent useful clean speech. Using the estimated frame sequence data, the GMM of the acoustic transfer function is created to deal with the influence of a room impulse response. Then, for each test dataset, we find a maximum-likelihood (ML) GMM from among the estimated GMMs corresponding to each position. The effectiveness of this method has been confirmed by talker localization experiments performed in a room environment.