Frame memory compression (FMC) is a technique to reduce memory bandwidth by compressing the video data to be stored in the frame memory. This paper proposes a new FMC algorithm integrated into an H.264 encoder that compresses a 4×4 block by differential pulse code modulation (DPCM) followed by Golomb-Rice coding. For DPCM, eight scan orders are predefined and the best scan order is selected using the results of H.264 intra prediction. FMC can also be used for other systems that require a frame memory to store images in RGB color space. In the proposed FMC, RGB color space is transformed into another color space, such as YCbCr or G, R-G, B-G color space. The best scan order for DPCM is selected by comparing the efficiency of all scan orders. Experimental results show that the new FMC algorithm in an H.264 encoder achieves 1.34 dB better image quality than a previous MHT-based FMC for HD-size sequences. For systems using RGB color space, the transform to G, R-G, B-G color space makes most efficient compression. The average PSNR values of R, G, and B colors are 46.70 dB, 50.80 dB, and 44.90 dB, respectively, for 768×512-size images.