Among image restoration approaches, image deconvolution has been considered a powerful solution. In image deconvolution, a point spread function (PSF), which describes the blur of the image, needs to be determined. Therefore, in this paper, we propose an iterative PSF estimation algorithm which is able to estimate an accurate PSF. In real-world motion-blurred images, a simple parametric model of the PSF fails when a camera moves in an arbitrary direction with an inconsistent speed during an exposure time. Moreover, the PSF normally changes with spatial location. In order to accurately estimate the complex PSF of a real motion blurred image, we iteratively update the PSF by using a directional spreading operator. The directional spreading is applied to the PSF when it reduces the amount of the blur and the restoration artifacts. Then, to generalize the proposed technique to the linear shift variant (LSV) model, a piecewise invariant approach is adopted by the proposed image segmentation method. Experimental results show that the proposed method effectively estimates the PSF and restores the degraded images.