A good resistance against mechanical stress is essential for the utilization of metal-organic frameworks (MOFs) in practical applications such as gas sorption, separation, catalysis or energy conversion. Here, we report on the successful modification of the mechanical properties of zeolitic imidazolate frameworks (ZIFs) achieved through a substitution of the terminal group. The mechanical modulus of SALEM-2 was found to significantly improve when the -H groups at position 2 of the imidazole linkers were replaced with electron withdrawing groups (-CHO, -Cl, or -Br). The charge distribution and electron density were analyzed to reveal the mechanism behind the observed variation of the elastic stiffness. Furthermore, ZIF-I with a -I group at position 2 of the imidazole linkers was predicted to exhibit an excellent mechanical strength in our study and then prepared experimentally. The results indicate that an inconspicuous change of the structure of ZIFs, i.e., additional groups strengthening the ZnN4 tetrahedron, will lead to a stiffer framework.