This paper presents a proposed method to compress images using two polynomials with different models based on the value of block pixels variance. These two polynomials are chosen from different set of models, which give low number of coefficients and preserve the quality of image as much as possible. This procedure of adaptive fitting ensures that the number of coefficients for each block is as the minimum as possible depending on the value of block variance. After applying multi-level of scalar quantization and Huffman encoding to polynomials coefficients for each block of image and testing different variance thresholds; mean square error (MSE), peak signal to noise ratio (PSNR), processing time, and compression ratio (CR) are evaluated for two types of images (color and gray scales) and for different block sizes (4x4 and 8x8 pixels). Computer results showed that the proposed method gives an acceptable compression ratio and image quality compared with non-adaptive fitting. For 4x4-block size, there is an improvement in PSNR (25.19 dB) compared with nonlinear polynomial case (25.08 dB). In addition, CR (7.45) is better than both cases (7.11 for linear and 5.56 for nonlinear polynomial case). The results showed that the suggested method of adaptive polynomial fitting is more suitable for gray scale images (including handwriting images).