2D-3D fusion for layer decomposition of urban facades

Yangyan Li*, Qian Zheng, Andrei Sharf, Daniel Cohen-Or, Baoquan Chen, Niloy Mitra

*Corresponding author for this work

Research output: Chapter in Book/Report/Conference proceedingConference contributionpeer-review

61 Scopus citations

Abstract

We present a method for fusing two acquisition modes, 2D photographs and 3D LiDAR scans, for depth-layer decomposition of urban facades. The two modes have complementary characteristics: point cloud scans are coherent and inherently 3D, but are often sparse, noisy, and incomplete; photographs, on the other hand, are of high resolution, easy to acquire, and dense, but view-dependent and inherently 2D, lacking critical depth information. In this paper we use photographs to enhance the acquired LiDAR data. Our key observation is that with an initial registration of the 2D and 3D datasets we can decompose the input photographs into rectified depth layers. We decompose the input photographs into rectangular planar fragments and diffuse depth information from the corresponding 3D scan onto the fragments by solving a multi-label assignment problem. Our layer decomposition enables accurate repetition detection in each planar layer, using which we propagate geometry, remove outliers and enhance the 3D scan. Finally, the algorithm produces an enhanced, layered, textured model. We evaluate our algorithm on complex multi-planar building facades, where direct autocorrelation methods for repetition detection fail. We demonstrate how 2D photographs help improve the 3D scans by exploiting data redundancy, and transferring high level structural information to (plausibly) complete large missing regions.

Original languageEnglish (US)
Title of host publication2011 International Conference on Computer Vision, ICCV 2011
Pages882-889
Number of pages8
DOIs
StatePublished - Dec 1 2011
Event2011 IEEE International Conference on Computer Vision, ICCV 2011 - Barcelona, Spain
Duration: Nov 6 2011Nov 13 2011

Publication series

NameProceedings of the IEEE International Conference on Computer Vision

Other

Other2011 IEEE International Conference on Computer Vision, ICCV 2011
CountrySpain
CityBarcelona
Period11/6/1111/13/11

ASJC Scopus subject areas

  • Software
  • Computer Vision and Pattern Recognition

Fingerprint Dive into the research topics of '2D-3D fusion for layer decomposition of urban facades'. Together they form a unique fingerprint.

Cite this