The persistent development of large-scale teaching video highlights the insufficiency of the current teaching video retrieval. Now, a teaching video retrieval system based on key frame has been designed and achieved. The system uses mutual information extraction for the video key frame extraction. The feature of key frame uses colour information; the similarity judgment uses simplified histogram for computation. In case of inquiry and retrieval, such a video retrieval method as combines key word and key frame is applied. The system had been tested and verified, and the result proves that the system meets the application requirements.