The stereo-digital image correlation technique using two synchronized industrial-grade cameras has been extensively used for full-field 3D shape, displacement and deformation measurements. However, its use in resource-limited institutions and field settings is inhibited by the need for relatively expensive, bulky and complicated experimental set-ups. To mitigate this problem, we established a cost-effective and ultra-portable smartphone-based stereo-digital image correlation system, which only uses a smartphone and an optical attachment. This optical attachment is composed of four planar mirrors and a 3D-printed mirror support, and can split the incoming scene into two sub-images, simulating a stereovision system using two virtual smartphones. Although such a mirror-based system has already been used for stereo-image correlation, this is the first time it has been combined with a commercial smartphone. This publication explores the potential and limitations of such a configuration. We first verified the effectiveness and accuracy of this system in 3D shape and displacement measurement through shape measurement and in-plane and out-of-plane translation tests. Severe thermal-induced virtual strains (up to 15,000 με) were found in the measured results due to the smartphone heating. The mechanism for the generation of the temperature-dependent errors in this system was clearly and reasonably explained. After a simple preheating process, the smartphone-based system was demonstrated to be accurate in measuring the strain on the surface of a loaded composite specimen, with comparable accuracy to a strain gauge. Measurements of 3D deformation are illustrated by tracking the deformation on the surface of a deflating ball. This cost-effective and ultra-portable smartphone-based system not only greatly decreases the hardware investment in the system construction, but also increases convenience and efficiency of 3D deformation measurements, thus demonstrating a large potential in resource-limited and field settings.