Fast block-wise motion estimation algorithm based on translational model solves the high computational complexity issue to some extent, but it sacrifices the motion compensation quality, whilst the higher-order motion model still exhibits the problems of computationally inefficiency and unstable convergence. Through a number of experiments, it is found that about 56.21% of the video blocks contain zoom motion, thus a conclusion is drawn that zoom motion is one of the most important motion forms in video except for the translational motion. Therefore, a zoom coefficient is introduced into the conventional block-wise translational model by bilinear interpolation, and model the motion-compensated error into a quadratic function with regard to the zoom coefficient. Subsequently, the approach is derived to compute the optimal zoom coefficient under the condition of 1D zoom motion through Vieta’s theorem, which is further extended to the condition of 2D zoom motion with equal proportion. Based on the above, a fast block-matching motion estimation algorithm is presented and is optimized by the adaptive zoom coefficient. It first uses the diamond search (DS) to compute the translational motion vector, and then determines an optimal matching block for the block to be predicted with the adaptive zoom coefficient. Experimental results carried out on 33 standard test video sequences showed that the proposed algorithm gains separately 0.11 dB and 0.64 dB higher motion-compensated peak signal-to-noise ratio (PSNR) than those of the full search (FS) and the DS based on block-wise translational model. And its computational complexity is 96.02% lower than that of the FS, slightly higher than that of the DS. Compared with the motion estimation based on the zoom model, the average PSNR of the proposed algorithm is 0.62 dB lower than that of 3D full search, but 0.008 dB higher than that of fast 3D diamond search. And the computational complexity only amounts to 0.11% and 3.86% of the 3D full search and the 3D diamond search, respectively. Meanwhile, the proposed algorithm can realize the self- synchronization between the encoder and decoder without transmitting the zoom vectors, so it does not increase the overhead of the side information. Additionally, the proposed adaptive zoom coefficient computation can also be combined with state-of-art fast block-wise motion estimation algorithms other than the diamond search, improving their motion-compensation quality.