Face Occlusion Detection and Recovery using Fuzzy C-Means

This paper presents a framework to detect and r ecover t he occluded fa cial region. We based on fact that any face has symmetric and not sym metric facial features and all these symmetric facial features a re consistent wi th the shape of the face. So that, if there is an occlus ion in one half of th e input face image, then the second half is used to recover the occlusion. Usi ng symmetry feature of the face makes the recovered face very close to the original face image in terms of pixel values and in general appearance. In other side when features do not symmetric, the occl usion can not be recovered using the symmetry feature of the face as the case the mouth region is occluded, so t he images database is used to select from it the most similar face images to the occluded face image to use it to select similar face to recover occlusion. In current work, we first dete ct the occluded fac e image by using pixel based skin color segmentation and eye templa te matching. Then, using fuzz y c-means to detect occlusion. Finally, the procedure for rec overy is implemented. The results shows that the proposed algorithm provides an effective solution to solve the problem of face occ lusion. This work can used in many applications as in repair the i mportant persons historic image and archive image which we get results reaches to 73% of identical to original image which has 40% occluded.


Introduction
In recent years, face images has attracted much attention and its research has rapidly expanded, since it has many potential applications in computer vision, communication, and automatic access control system.Especially, face detection is an important part of face recognition and many applications.However, face detection is effected by many factors that modify face image appearance such as, viewing angle (front, non front), occlusion, image orientation, illumination, facial expression and individual factors [1].
Face occlusion represents big problem in many applications like face recognition, image archive system, history image …etc.So a technique that can automatically recover the deterioration in the face images is needed.
Recently, some algorithms are proposed to recover partially deteriorated face images.Hwang and Lee presented a method to recover occluded faces by linear combination of faces.Initially, a face image is represented with two eigen bases, one for the shape correspondence between the particular face and reference face, and a second for the shape-normalized face texture.Given a face with occluded region, the shape and texture of known regions are fit with a linear combination of their respective eigen vectors and the same combination is then used to recover the unknown (occluded) region [2].Mo et al presented a method to recover occluded face image using a positive only-mixture of training faces .Negative least sequares algorithm is used for the positive mixture representation of a given face .Given a target face with occluded region, a subset of the training faces that are relatively similar to the target face are determined firstly .Then, positive representation weights are chosen to evaluate the match in the occluded region [3].Dahua and Xiaoo presented a framework to automatically detect and recover the occluded facial region.Initially, a Bayesian framework unifies the occlusion and recovery stages .Then a quality assessment model is developed to drive both the detection and recovery process, which captures the face priorities in both global and local patterns [4].

Face Image and Occlusion Types
Human face represents one of the most common biometric patterns that our visual system encounters daily.Faces to be useful biometric, facial features should remain invariant to factors unrelated to person identity that modify face image appearance [1].Deterioration has large effect on face image since it may be damage the face image , lose part of the face image , or cause high noise .theface deterioration has many shape and many affection , so we can take the following types of deterioration like , 1-Face deterioration by text.

2-Face deterioration by stain 3. Skin Color Segmentation and Eye Template Matching
To detect face region, a combination of skin color segmentation and template matching is used here.Skin color segmentation has proven to be useful and robust cue for face detection and tracking.There are many feature-based face detection methods, the most important once using skin color as detection cue.Skin color allows fast processing and is highly robust to geometric variations of the face pattern.There are many color spaces; one of these color spaces is the (red, green, blue) RGB color PDF created with pdfFactory Pro trial version www.pdffactory.comspace that is used by our algorithm for skin segmentation purpose.It is one of the most widely used color spaces for processing and storing of digital image data [5].
Face image can be separated into skin and non skin regions based on the knowledge that in RGB space the skin has a higher red content than other component.So (R, G, and B) is classified as skin if [6]: R> 95 and G > 40 and B > 20 and Max(R, G, B)-min(R, G, B) > 15 and R-G > 20 and R-B >20, …….( 1) Figure (1) shows an example of applying pixel based skin color segmentation.When the results of RGB based skin color segmentation produce more than one skin color regions, then eye template is used as verification process to further distinguishes face from other non-face regions that have the same skin color.Eye template shown in Figure (2), is generic and captures the general properties of the eye and its surrounding area.
The basic method to perform template matching is to loop through all pixels in the search image and capture them using the following steps: Step -1 Resize the width of the skin regions to the width of the eye template which is equal to 45 pixels.
Step -2 Convert the skin region to binary levels.
Step -3 Perform tracking vertically on the search region (skin region) and then compute the similarity measure for each block.
Step -4 Select the block with maximum value and set the corresponding skin region as the face is detected.
Figure (3) shows the results of applying eye template matching on the skin regions.At this point, the occluded face region is detected.

Occlusion Detection and Fuzzy cmeans
In many image processing applications, we employ prior knowledge to interpret an input scene.Examples are pattern recognition, region segmentation, scene description, and so on.Fuzzy logic offer us powerful tool to represent process of human knowledge in the form of fuzzy if-then-else rule [7].
The most popular fuzzy clustering algorithm, known as fuzzy c-means algorithm (FCM).FCM algorithm partition an input image of size M×N specified by m-dimensional vector of K data points (k = 1, 2, . . .,K) into C fuzzy clusters, in which each point have a membership associated with it.This membership value represents the extentent to which a pixel belongs to a class, having a specific attributes.It finds a cluster centre in each, minimizing an objective function.Fuzzy c-means is different from hard c-means, mainly because it employs a fuzzy partitioning, that is, a point can belong to several clusters with degrees of membership.To accommodate the fuzzy partitioning, the membership matrix M has elements in the interval [0, 1].A point's total membership of all clusters, however, must always be equal to unity to maintain the properties of the M matrix.The objective function is:

…………………………………(2)
Where item is the membership value in the interval [0, 1] of data point k of cluster i, vector is the centre of fuzzy cluster i, scalar 5747 = | | -|| is the Euclidean distance between the ith cluster centre and the kth data point, and scalar q is the fuzziness exponent.The commercial tools usually recommend a value q = 2.There are two necessary conditions for J to reach a minimum, ….. (3) and …… (4 The algorithm is, in essence, iteration through the preceding two conditions [7].
The algorithm for occlusion detection is based on the fact that the detected face share similar properties in Y color and RGB component.So that; occlusion can be extracted from the detected face region as a different region.This different region should be determined firstly by one of the segmentation algorithm to recover it in the next processing .FCM clustering algorithm is one of the most successful image segmentation algorithm, is used here for segment the detected face into two regions occluded and non-occluded regions.The proposed algorithm for occlusion detection is performed using the following steps: 1-Convert the occluded face image from RGB to Y components using the following eq.[8] ……………………………….( 5

Proposed System
The main component of our system is shown in figure (5): The proposed system does not work with face image that have 2 on the fact that any face has symmetric facial features and all these symmetric facial features are consistent with the shape of the face, so that, if the occluded region of the face is the left eye, then right eye can be used to recover left eye.But, when the occluded region is not one of the symmetric facial features, such as nose and mouth, then the recovery process is not based on the face itself, but based on another similar face that is built from face data base according to specific similarity measures.

Face Database
The face data base used here contains about 100 face images with frontal view.
Figure (6) shows a sample set of face images stored in the data base.All face images in the face data base are of size 96×96 px with mask shape.The face images are stored in the data base with several information related to it such as : person sex(male, female), person age (young , old), Standard deviation value for the main parts of the face which include left eye, right eye, nose, and mouth using the division in Figure (7).Table (1) shows the fields of the face database.

Steps of Proposed System
Step-1 input occluded face image, with any size.
Step-2 Detect face region using skin color segmentation and template matching.
Step-3 segments the occluded region inside face mask using fuzzy c-means algorithm.
Step-5 test the location of the occlusion region , if the location is one of the symmetric region then replace the occlusion region with its symmetric region in the occlusion face image , else select from the face data base the set of faces which are most similar to the occluded face and then built mean face.These faces which are used to built mean face are selected using the following parameters , • Skin standard deviation.
• Person sex (male, female).Skin standard deviation is computed over visible regions of the occluded face.
Step-6 performs occlusion recovery by using mean face to recover occluded region.
Step-7 performs an enhancement filter on the resulted image to smooth the variation in the colors.We apply enhancement filter on the surrounding area of the deterioration region only to obtain not blurred face image.
Step-8 exit. Figure (8) shows the results of face occlusion recover.

Results
In order to test the efficiency of the proposed system that is implemented using Programming language Visual Basic, a series of experiments was performed using many different sets of images.Initially, three face data bases are used for testing the accuracy of the proposed face detection algorithm as follows : PDF created with pdfFactory Pro trial version www.pdffactory.com

Experiment-1
The results of applying Experiment-1 to the proposed face detection algorithm using three types of face database with different properties are shown in Table (2).The face detection ratio is computed using the following eq.[12]………………………………….(8) This experiment is performed using three standard face data base .The results are displayed in Figure (12).It can be seen that the proposed face detection algorithm achieves 84% detection ratio for face database which make it robust to partial occlusion and pure illumination , and also achieves 80% detection ratio for face database which make it is flexible to detect face images with complex background .
At a result, it can be noted from Figure(11) that the face detection ratio increases when the face images are considered with clean background.

Experiment-2
The results of applying Experiment-2 to the proposed face recovery algorithm using 100 occluded face images are shown in figure (13) using the following eq.At a result, it can be noted from figure (13) that the difference between recovered face image and the original face image using symmetry feature of the face is very low in compression with recovery quality using mean face only.So that, it is very easy to obtain very similar face image to original face image by using symmetry feature of the face.

Evaluation
To evaluate the whole system performance ,some measures are used like: 1-The distance between the original face image A and the occluded face image B using eq.( 9).
PDF created with pdfFactory Pro trial version www.pdffactory.com

1 - 3 -
The first face data base [9] is a collection of 100 frontal face images with clean background.Figure (9) shows a sample set of face images from .2-The second face database [10] is a collection of 100 face images of 100 persons with frontal views and different illumination conditions and partial occlusion.Figure(10) shows a sample set of face images from .The third face database [11] is a collection of 100 face images with frontal views and generic background.

Figure
against the ground truth in the case that occluded face is recovered using symmetry feature of the face only, while Figure (12),b shows the recovered occluded face using mean face.

Figure ( 5 )
Figure (5) Outline of the proposed face occlusion detection and recovery system.

Figure ( 8 )Face
Figure (8) The results of face occlusion recovery.

Figure ( 12 )Face
Figure (12) The results of face detection algorithm .