Dynamic Removing and Reconstruction of Partially Occluded Region of Objects

Abstract

In this paper I proposed a three dimensional camera sensor technology which is capable of rotating and moving in three axis instantaneously to solve the partial occlusion problem dynamically. When the subject will partially have occluded by moving or static objects a particular camera sensor will move as accordance to the occlusion parts. The main camera will then combine the whole data and reconstruct the occluded part from other sensor image. Occlusion is a great problem in many applications nowadays.

Introduction

In object tracking or motion detection applications occlusion is an inevitable problem nowadays. In the majority of photos or video parts or even a tracking object are not visible in all the frames of the sequence, either due to the existence of static objects (e.g., walls, trees, obstacles) that occlude it or due to the existence some moving objects (e.g., two people walking and crossing each other). In the tracking literature many algorithms have ignored occlusions and handled them simply as noise in the matching process. Even in face recognition applications occlusion is one of the strongest barrier. Occlusion is the cause of one object in a space blocking another object from current view. In the computer vision perspective, occlusion means a blockage to the camera sensor which is the vision agent. However, there are many approaches and methods to overcome the occlusion problem but it is still improving and enhancing. On the other hand, object reconstruction means the process of recovering 3D information from a sequence of 2D images.

In this paper I propose a new method with the capability of rotating and moving in three axis instantaneously with multiple wide angle camera sensors. When the subject will be partially occluded, any of the camera sensors will be triggered and it will move or rotate as necessary. So this method is mainly new hardware configuration change rather than computation algorithm and 3D sensor.

Related work

Partial and full occlusion problem is a great concern nowadays as it’s creating barriers in such number and varieties of application. In the field of computer vision and pattern recognition massive and numerous methods have been proposed in recent years along with some other research domain such as brain research, neural networking, quantum computation.

Multiview object reconstruction Qian et al presented the first approach for simultaneously recovering the 3D shape of both a wavy water surface and a moving underwater scene. It’s a different type configuration of a portable camera array system that captures the scene from multiple viewpoints above the water. An optical flow method is to estimate the correspondences of the water and are used to infer the shape of the water surface and the underwater scene. Shen introduced a patch-based stereo matching process wich merged a depth map for reconstructing objects in large-scale scenes. An efficient patched based stereo matching process is used to generate depth-map at each image with acceptable errors, followed by a depth-map refinement process to enforce consistency over neighboring views.

Camera array systems Zhang and Chen developed a reconfigure camera positions to achieve better rendering quality and a self-reconfigurable camera array system to capture video sequences from an array of mobile cameras. Gasparovich and Gajski proposed a two-step process for calibrating ultrawide-angle cameras. Based on the camera-calibration method using images taken from an unmanned aerial vehicle. Structure from motion Paul Beardsley, Phil Torr and Andrew Zisserman proposed a method based on a minimum number of token correspondences across an image triplet which includes a trifocal sensor. The incremental strategy for SFM has become the most commonly used 3D reconstruction approach at present and emerged as mainstream research.

Research goal

Our first goal is to move and rotate all the camera sensors comfortably and instantly. Then the segmentation part is very important to identify the occluded region and trigger the perfect sensor which can get the different view from other angle and perspective. After getting the images, merging them properly, reconstruct and generate the part is our very main goal.

Solution method

In this paper I propose a new multiple wide angle camera based technology with the capability of rotating and moving instantly in three axis to get the occluded part from different angle of perspective. The camera sensors will be wide angle as they got bigger perspective and area of the view. So the probability of getting the desired part in the view gets higher. As part of getting the exact part which is occluded I need to detect it first and there I am proposing hybrid segmentation process which uses both structural and stochastic segmentation process. It uses discrete pixel and structural information together. A simple algorithm will compute and choose the sensor which will move and how much it should be, where it can be changed dynamically as the subject or occlusion object can change its position. The algorithm should be focused on speed and as much accurate as possible. After getting the partial part which is occluded, the regenerating of the full image starts. The different perspective will be realigned and corrected with various approaches depended on the environment and image.

Results and discussion

In the proposed method, wide angle camera sensors are used because they can take more area as a view and that lessen the probability of losing the occluded part. Three axis motion is needed as the perfect view from different perspective for the exact occluded part can be from right shifting or left shifting or adjusting forward or backward and maybe even just rotating up or down any of the camera sensors. Multiple camera sensor gives higher probability of precise and perfect positioning. Partially occluded part should be detected by the segmentation and there, structural and stochastic both segmentation process will be needed. Because, from edge detecting segmentation the structure of the subject will be detected and the occluded part will need discrete pixel information while segmenting and selecting final area which is occluded. The moving algorithm is focused on speed as the system should be instantaneous and dynamic. Perspective correction is needed as it is changed and chance of distortion while moving or rotating and wide angle lens has some perspective issues. All the necessary parts above have been discussed and included in my method which gives a higher probability of success. Conclusion Partial or full occlusion is a day to day problem in many applications like face recognition, vehicle recognition, motion analysis. Many methods and approaches has been developed though they should be enhanced and amended. My method is a hardware based approach which is in its initial stage of formulating so it must be refined with new approaches added and removing unnecessary steps. The occlusion object can be tracked to improve sensor moving algorithm. However, many refinement is left for further investigation.