In this small work, a plane sweep method was implemented for depth map generation. The main idea was to compare census based plane-sweep (in-house) with novel image intensity based cost metric. Calibrated views were given as an input along with high-resolution images from iPhone. As initial step, the 3D poses had to be estimated using 5point algorithm from the extracted feature points. Also bundle adjustment step was implemented using ceres library. A plane-sweep algorithm was developed which scans a given depth range and produces photometric consistency cost for each depth candidate. Depth consistency curve (illustrated in screenshots below) measures quality of each depth candidate. It is used to pick the best depth for each pixel in selected reference view.
Case1: Top-left: reference image, top-mid: top view on 3D poses and the reference pose (on green) which spans the depth range for plane sweep, top-right: depth-consistency curve. bottom-left: current image (pose on red), bottom-mid: depth error image for selected depth, bottom-right: depth map after the sweep.
Case2: Top-left: reference image, top-mid: top view on 3D poses and the reference pose (on green) which spans the depth range for plane sweep, top-right: depth-consistency curve. bottom-left: current image (pose on red), bottom-mid: depth error image for selected depth, bottom-right: depth map after the sweep.