Local Occlusion Detection Under Deformations
Using Topological Invariants

 

Occlusions provide critical cues about the 3D structure of man-made and natural scenes. We present a mathematical framework and algorithm to detect and localize occlusions in image sequences of scenes that include deforming objects. Our occlusion detector works under far weaker assumptions than other detectors. We prove that occlusions in deforming scenes occur when certain well-defined local topological invariants are not preserved. Our framework employs these invariants to detect occlusions with a zero false positive rate under assumptions of bounded deformations and color variation. The novelty and strength of this methodology is that it does not rely on spatio-temporal derivatives or matching, which can be problematic in scenes including deforming objects, but is instead based on a mathematical representation of the underlying cause of occlusions in a deforming 3D scene. We demonstrate the effectiveness of the occlusion detector using image sequences of natural scenes, including deforming cloth and hand motions.


  • E. Lobaton, R. Vasudevan, R. Bajcsy, and R. Alterovitz, "Local Occlusion Detection Under Deformations Using Topological Invariants," European Conference on Computer Vision (ECCV), Greece, 2010. [PDF]

Animations (Click on Images):

The animations illustrate the detections between consecutive frames in a video sequence. The detections are centered at frame t. The pictures at times t-1, t and t+1 can be observed on the top row. The bottom-left plot shows the occlusion detections between the frames. The red areas correspond to detections between t-1 and t, and the blue areas correspond to detections between t and t+1. Detections are marked by highlighting the neighborhood Er in which the Image Homeomorphism Criterion was not satisfied. Several values of r are used in the analysis. The bottom-right plot shows the detections in black and white over the frame at time t. For these sequences we used Kc = 10, Kd = 2, Kt = 10, Nt = 5, and the color bins were set to partition the range from 0 to 255 into 20 bins (see Section 4 for definitions). The widths of the pictures in the sequence are 400, 250 and 550 pixels respectively.


Folding a Colored Macbeth Board. Detections for r = 1 and 2 are shown. Note that the motion is not consistent on this sequence (the board moves back and forth between frames) making it difficult to perform a motion consistency analysis. [Slow Motion] - [Dataset]

   
   

Synthetic Sphere Moving Through Tiled Room. Detections for r = 1, 2, 3 and 5 are shown. Each detection region provides samples from the foreground and background objects which are used to obtain the segmentation in Fig 7 of paper. [Slow Motion] - [Dataset]

Person Walking. Detections for r = 1 and 2. [Slow Motion] - [Dataset]