3D FACE MODELING: COMPREHENSIVE DESCRIPTION

Sushma Jaiswal; Dr. Sarita Singh Bhadauria; Dr.Rakesh Singh Jadon

3D FACE MODELING: COMPREHENSIVE DESCRIPTION

Sushma Jaiswal¹, Dr. Sarita Singh Bhadauria², Dr.Rakesh Singh Jadon³

Lecturer, S.O.S. in Computer Science, Pt.Ravishankar Shukla University,Raipur(C.G.)
Professor & Head,Department of Electornics Engineering, Madhav Institute of Technology & Science,Gwalior(M.P.)
Professor & Head,Department of Computer Applications, Madhav Institute of Technology & Science,Gwalior(M.P.)

Related article at Pubmed, Scholar Google

Visit for more related articles at Journal of Global Research in Computer Sciences

Abstract

To provide a comprehensive survey, we not only categorize existing modeling techniques but also present detailed descriptions of representative methods within each category. In addition, relevant topics such as biometric modalities, system evaluation, and issues of illumination and pose variation are covered. 3D models hold more information of the face, like surface information, that can be used for face recognition or subject discrimination. This paper, gives the survey based techniques or methods for 3D face modeling, in this paper first step namely Model Based Face Reconstruction, secondly Methods of 3d Face models divided into three parts Holistic matching methods, Feature-based (structural) matching methods, Hybrid methods thirdly Other methods categorized into again three parts 2D based class, 3D Based class and 2D+3D based class are discussed. There are two underlying motivations for us to write this survey paper: the first is to provide an up-to-date review of the existing literature, and the second is to offer some insights into the studies of machine recognition of faces

Keywords

Biometric modalities, 3D face model, bootstrapping algorithm, morphable model, 3D Shape based methods.

INTRODUCTION

Reliable personal recognition techniques play a critical role in our everyday and social activities. In access control, authorized users should be allowed for entrance with high accuracy while unauthorized users should be denied. In welfare benefit disbursement, people not only should verify whether the identity of a person is whom he/she claimed to be, but also should avoid the occurrence that one person claims to be another person to receive the welfare benefit twice (double dipping).

Traditionally, there are two categories of personal recognition approaches, token-based and knowledge-based (Miller, 1994). In the token-based approach, the identity of a person is verified according to what they have. Anyone possessed a certain physical object (token), e.g., keys or ID cards, is authorized to receive the associated service. The knowledge-based approaches authenticate the identity of an individual according to what they know. Any individuals with certain secret knowledge, such as passwords and answers to questions, would receive the associated service. Both the token-based and the knowledge-based approaches, however, have some inherent limitations. In the token-based approach, the “token” could be stolen or lost. In the knowledge-based approach, the “secret knowledge” could be guessed, forgotten, or shared. Biometric recognition is an emerging personal recognition technology developed to overcome the inherent limitations of the traditional personal recognition approaches (Jain, Bolle, & Pankanti, 1999a; Zhang, 2000, 2002, & 2004; Wayman, 2005; Bolle, 2004). The term biometrics, which comes from the Greek words bios (life) and metrikos (measure), refers to a number of technologies to authenticate persons using their physical traits such as fingerprints, iris, retina, speech, face and palm print or behavior traits such as gait, handwritten signature and keystrokes. In other words, biometric recognition recognizes the identity of an individual according to who He/she is. Compared with the token-based and the knowledge-based methods, biometric identifiers cannot be easily forged, shared, forgotten, or lost, and thus can provide better security, higher efficiency, and increased user convenience.

Biometric recognition lays the foundation for an extensive array of highly secure authentication and reliable personal verification (or identification) solutions. The first commercial biometric system, Identimat, was developed in 1970s, as part of an employee time clock at Shearson Hamill, a Wall Street investment firm (Miller, 1994). It measured the shape of the hand and the lengths of the fingers. At the same time, fingerprint-based automatic personal authentication systems were widely used in law enforcement by the FBI and by US government departments. Subsequently, advances in hardware such as faster processing power and greater memory capacity made biometrics more feasible and effective. Since the 1990s, iris, retina, face, voice, palm print, signature and DNA technologies have joined the biometric family (Jain et al., 1999a; Zhang, 2000).

With the increasing demand for reliable and automatic solutions to security systems, biometric recognition is becoming ever more widely deployed in many commercial, government, and forensic applications. After the 911 terrorist attacks, the interest in biometrics-based security solutions and applications increased dramatically, especially in the need to identify individuals in crowds. Some airlines have implemented iris recognition technology in airplane control rooms to prevent any entry by unauthorized persons. In 2004, all Australian international airports implemented passports using face recognition technology for airline crews and this will eventually became available to all Australian passport holders (Jain et al., 1999a). Several governments are now using or will soon be using biometric recognition technology. The U.S. INSPASS immigration card and the Hong Kong ID card, for example, both store biometric features for reliable and convenient personal authentication.

Generally speaking, any situation that allows an interaction between human and machine is capable of incorporating biometrics. Such situations may fall into a range of application areas. Biometrics is currently being used in areas such as computer desktops, networks, banking, immigration, law enforcement, telecommunication networks and monitoring the time and attendance of staff. Governments across the globe are tremendously involved in using and developing biometrics. National identity schemes, voting registration and benefit entitlement programs involve the management of millions of people and are rapidly incorporating biometric solutions. Fraud is an everincreasing problem and security is becoming a necessity in many walks of life. Biometric applications can be simply categorized as follows (Zhang, 2000):

LAW ENFORCEMENT

The law enforcement community is perhaps the largest user of biometrics. Police forces throughout the world use Automated Fingerprint Identification System (AFIS) technology to process suspects, match finger images and to process accused individuals. A number of biometric vendors are earning significant revenues in this area, primarily using AFIS and palm-based technologies.

BANKING

Banks have been evaluating a range of biometric technologies for many years. Automated Teller Machines (ATMs) and transactions at the point of sale are particularly vulnerable to fraud and can be secured by biometrics. Other emerging markets such as telephone banking and Internet banking must also be totally secure for bank customers and bankers alike. A variety of biometric technologies are now striving to prove themselves throughout this range of diverse market opportunities.

COMPUTER SYSTEMS (ALSO KNOWN AS LOGICAL ACCESS CONTROL)-

Biometric technologies are proving to be more than capable of securing computer networks. This market area has phenomenal potential, especially if the biometric industry can migrate to large-scale Internet applications. As banking data, business intelligence, credit card number, medical information and other personal data become the target of attack, the opportunities for biometric vendors are rapidly escalating.

PHYSICAL ACCESS

Schools, nuclear power stations, military facilities, theme parks, hospitals, offices and supermarkets, across the globe employ biometrics to minimize security threats. As security becomes more and more important for parents, employers, governments and other groups - biometrics will be seen as a more acceptable and therefore essential tool. The potential applications are infinite. Cars IGI Global, distributing in print or electronic forms without written permission of IGI Global is prohibited. For example, the sanctuary of the ordinary citizen, are under constant threat of theft. Biometrics - if appropriately priced and marketed - could offer the perfect security solution.

BENEFIT SYSTEMS

Benefit systems like welfare especially need biometrics to struggle with fraud. Biometrics is well placed to capitalize on this phenomenal market opportunity and vendors are building on the strong relationship currently enjoyed with the benefits community.

IMMIGRATION

Terrorism, drug running, illegal immigration and an increasing throughput of legitimate travellers is putting a strain on immigration authorities throughout the world. It is essential that these authorities can quickly and automatically process law-abiding travellers and identify law-breakers. Biometric technologies are being employed in a number of diverse applications to make this possible. The US Immigration and Naturalization Service is a major user and evaluator of a number of state-of-the-art biometric systems. Systems are currently in place throughout the US to automate the flow of legitimate travellers and deter illegal immigrants. Elsewhere biometrics is capturing the imagination of countries such as Australia, Bermuda, Germany, Malaysia and Taiwan.

NATIONAL IDENTITY

Biometric technologies are beginning to assist governments as they record population growth, identify citizens and prevent fraud from occurring during local and national elections. Often this involves storing a biometric template on a card that in turn acts as a national identity document. Finger scanning is particularly strong in this area and schemes are already under way in Jamaica, Lebanon, The Philippines and South Africa.

TELEPHONE SYSTEMS

Global communication has truly opened up over the past decade, while telephone companies are under attack from fraud. Once again, biometrics is being called upon to defend against this onslaught. Speaker ID is obviously well suited to the telephone environment and is making in-roads into these markets.

TIME, ATTENDANCE AND MONITORING: Recording and monitoring the movement of employees as they arrive at work, have breaks and leave for the day were traditionally performed by time-card machines. Replacing the manual process with biometrics prevents any abuses of the system and can be incorporated with time management software to produce management accounting and personnel reports.

BIOMETRIC RECOGNITION TECHNOLOGIES/ MODALITIES

A biometric system can be regarded as a pattern recognition system, where a feature set is first extracted from the acquired data, and then is compared with the stored template set to make a decision on the identity of an individual. A biometric system can be operated in two modes, biometric verification and biometric identification. In biometric verification mode, the decision is whether a person is “who he/she claims to be?” In biometric identification mode, the decision is “whose biometric data is this?” Thus a biometric system can be formalized into a two-class or multi-class pattern recognition system.

A general distinction is drawn between behavioural and physiological biometric characteristics. Physiological characteristics are usually determined by the genes (like face or vein pattern), in some cases (fingerprint, iris) they are also influenced by extra-genetic or environmental factors and can in theory be used to distinguish between identical twins (with identical genome). Behavioural modalities are affected by the human genome as well, but their occurrence can be changed deliberately. The process of capturing behavioural modalities is a measurement of an activity. Therefore these modalities are called active modalities.

A biometric system usually includes four major modules: data acquisition, feature extraction, matching, and system database (Jain, Ross, & Prabhakar, 2004). In the data acquisition module, the biometric data of an individual is acquired using a capture sensor. In the feature extraction module, the acquired data is processed to extract a set of discriminative features. In the matching module, the features are compared with the stored template set to make a decision on the identity of an individual. In the system database module, a database is built and maintained to store the biometric templates of the enrolled users. Feature extraction and matching are two of the most challenging problems in biometric recognition research, and have attracted researchers from different backgrounds: biometrics, computer vision, pattern recognition, signal processing, and neural networks.

Advances in sensor technology and increasing diverse demand of biometric systems cause the persistent progress on developing novel acquisition sensors and novel biometric technologies. Before 1980s, the offline “ink-technique” is the dominant approach to acquire fingerprint images. Nowadays, a number of online live-scan fingerprint sensors, e.g., optical, solid-state, and ultrasound, have been designed for fingerprint acquisition.

Although research on the issues of common biometric technologies have drawn considerable attention, and have been studied extensively over the last 25 years, there are still some limitations to varieties of existing applications. For example, some people have their fingerprints worn-away due to the hard work they do with their hands and some people are born with unclear fingerprints. Face-based and voice-based identification systems are less accurate and easier to be attacked using a mimic. Efforts geared towards improving the current personal identification methods will continue, and meanwhile new biometric technologies are under investigation. Currently, the major biometric technologies involve face, fingerprint, iris, palm print, signature, and voice recognition, as well as multi-biometric recognition technologies (Zhang, 2002). The following provides a brief introduction of these biometric traits:

FINGERPRINT

Because the patterns of ridges and valleys on an individual’s fingertips are unique to that individual, fingerprints can be used for authenticating personal identity. For decades, law enforcement has been classifying and determining identity by matching key points of ridge endings and bifurcations. Fingerprints are so unique that even identical twins usually do not have the same fingerprint.

IRIS

The patterns of the iris, the colored area that surrounds the pupil, are thought to be unique. Iris patterns can be obtained through a video-based image acquisition system. Iris scanning devices have been used in personal authentication applications. It has been demonstrated that iris-based biometric system can work with individuals without regard to ethnicity or nationality.

PALM PRINT

palm print, the inner surface of the palm, carries several kinds of distinctive identification features for accurate and reliable personal recognition. Like fingerprints, palm print have permanent discriminative features including patterns of ridges and valleys, minutiae, and even pores in high resolution (>1000dpi) images. Except these quasi fingerprint features, palm print also carries other particular distinctive features including principal lines and wrinkles. Using a high resolution capture device, it is possible to extract all kinds of palm print features to construct a high accurate biometric system. In the early stage, palm print recognition techniques have been investigated to extract and match the singular points and minutia points from high resolution palm print images. High resolution palm print scanner, however, is expensive, and is time consuming to capture a palm print image, which restricts the potential applications of online palm print recognition systems. Subsequently, online capture device has been developed to collect real time low resolution palm print image, and low resolution palm print recognition has gradually received considerable recent interest in biometric community (Zhang, Kong, You, & Wong, 2003; Jain et al., 2004; Zhang, 2004).

FACE

Humans are specialists in recognising faces. The automation of this intentional process is not easy, but research is sophisticated in 2D face recognition. This modality has a very high user acceptance because of its frequent employment. After 30 years of research, 2D results are quite good [64].

Nevertheless, there are some issues that are hard to cope with taking into account only “flat” images of faces. With 3D depth information lighting conditions and pose variations can be handled more exact. This type of face recognition is an evolving domain that is not yet well investigated. Another advantage of 3D models is that they are harder to copy than 2D images. Several 2D face recognition systems could be fooled by simply holding a picture in front of a capture device. Recent systems include liveness detection mechanisms to prevent this kind of attack – a possible resolution are two camera systems, that are harder to fool.

SIGNATURE

Signature authentication involves the dynamic analysis of a signature to authenticate a person’s identity. A signaturebased system will measure the speed, pressure, and angle used by the person when producing a signature. This technology has potential applications in e-business, where signatures can be an accepted method of personal authentication.

SPEECH

Speech-based personal authentication, which has a history of about four decades, is regarded as a non-invasive biometric technology. Speech authentication uses the acoustic features of speech, which have been found to be different between individuals. These acoustic patterns reflect both anatomic (e.g., size and shape of the throat and mouth) and behavioral patterns (e.g., voice pitch, speaking style) of an individual. The incorporation of learned patterns into the voice templates (the latter called "voiceprints") has allowed speaker recognition to be recognized as a "behavioral biometric". Speech-based personal authentication systems employ three styles of spoken input: text-dependent, textprompted and text-independent. Most speech authentication applications use text-dependent input, which involves selection and enrollment of one or more voice passwords. Text-prompted input is used whenever there is concern about imposters.

RETINA

Blood vessel patterns in the back of the inner eye are taken as reference. This feature is very stable and does not alter. Sensors are expensive and must use visible light which may annoy users. Retinal images are used in the medical domain to diagnose diseases and are therefore known to be one of the few biometric modalities that carry sensitive information.

GAIT

Gait, the peculiar way one walks, is a complex spatiotemporal biometric trait. Note that gait is a behavioral trait and may not remain the same over a long period of time, due to some factors such as changes in body weight. It is commonly considered that a gait-based biometric system can be used in some low-security applications. Gait authentication is also not intrusive and the acquisition of gait is similar to acquiring a facial image. Usually a gait-based biometric system analyzes a video-sequence to obtain the gait trait and it is generally computationally expensive.

HAND AND FINGER GEOMETRY

A system may measure geometrical characteristics of either the hands or the fingers to perform personal authentication. These characteristics include length, width, thickness and surface area of the hand. Requiring only a small biometric sample of a few bytes is one interesting characteristic of this kind of biometric technology. The biometric system based on hand and finger geometry has been used in physical access control in commercial and residential applications, in time and attendance systems.

DNA

Feature extraction is very expensive and takes a lot of time (up to several days) but it is referred to as the ultimate biometric characteristic. Desoxyribonucleic acid is available in every cell of each organism, a drawback is the equality of identical twins. Although 99.5 percent of the human genome overlaps between individuals there is still enough information for exact identification. Alleles are alternate forms of the DNA that can be used for feature extraction.

DNA can be misused to derive other information (e.g. medical conditions, race or paternity can be extracted) and therefore is absolutely critical in respect of privacy.

EAR

There is evidence to show that the shape of the ear and the structure of the cartilaginous tissue of the pinna are distinctive. As a result, the ear-based biometric system can be used for authenticating personal identity. This uncommon modality can also be used for recognition. Employing thermograms instead of normal pictures improves system performance because hairstyle has no effect on it. Ear shape models are often combined with face recognition to improve overall performance. Human bodies provide many more attributes to be captured and taken for comparison. To name a few: Odour, sweat pores, vein patterns, lip motion or skin reflectance. Using multi-modal biometrics7 can improve the system’s performance.

ODOR

Each object, including people, spreads an odor that is characteristic of its chemical composition. This could be used for distinguishing various objects. This would be done with an array of chemical sensors, each sensitive to a certain group of compounds. However, deodorants and perfumes could compromise distinctiveness.

MULTI-BIOMETRICS

From an application standpoint, widespread deployment of a user authentication solution requires support for an enterprise’s heterogeneous environment. Often, this requires a multi-faceted approach to security, deploying security solutions in combination. An authentication solution should seamlessly extend the organization’s existing security technologies. We are now interested in understanding both how to build multi-biometric recognition systems and what possible improvements these combinations can produce. Currently there are several true multi-modal databases available for testing multi-biometric recognition algorithms. The most important resource available may be the extended M2VTS database, which is associated with the specific Lausanne protocol for measuring the performance of verification tasks. This database contains audio-visual material from 295 subjects (Poh & Bengio, 2006). To facilitate multi-biometric research, NIST presents an open resource of Biometric Scores Set -Release 1 (BSSR1), which includes true multimodel matching scores generated by face and fingerprint recognition algorithms (Grother & Tabassi, 2004).

MAIN PROBLEMS IN BIOMETRIC RECOGNITION

To enhance the recognition performance of the biometric system, this section suggests two advanced biometric recognition technologies, biometric data discrimination and multi-biometric technologies. In biometric data discrimination, we first introduce the fundamental of biometric data discrimination, and then suggest using a family of tensor discriminant analysis to deal with the diversity in forms of biometric data. In multi-biometrics, we introduce three categories of fusion strategies to enhance the performance and reliability of the biometric system. Besides recognition performance, security and privacy issues should also be taken in account. In terms of security, There are many attacks, such as overplay, database and brute-force attacks, on biometric applications. In terms of privacy, biometric traits may carry additional sensitive personal information. For example, genetic disorders might be inferred from the DNA data used for personal

BIOMETRIC DATA DISCRIMINATION

Generally, biometric data mainly exists in the following three forms: 1D waveform (e.g. voice, signature data), 2D images (e.g. face images, fingerprints, palm prints, or image sequences, i.e., video), and 3D geometric data (such as 3-D facial or hand geometric shapes). Since the diversity in biometric data and feature forms, it is hardly difficult to develop a universal recognition technology which is capable to process all kinds of biometric data. Fortunately, recent progress in discriminant analysis sheds some light on the possibility on this problem. Discriminant analysis, with the goal of dimensionality reduction and of retaining the statistical separation property between distinct classes, is a natural choice for biometric recognition. With the development of biometrics and its applications, many classical discriminant analysis technologies have been borrowed and applied to deal with biometric data. Among them, principal component analysis (PCA, or K-L transform) and Fisher linear discriminant analysis (LDA) have been very successful, in particular for face image recognition. These methods have themselves been greatly improved with respect to specific biometric data analyses and applications. Recently, nonlinear projection analysis technology represented by kernel principal component analysis (KPCA) and kernel Fisher discriminant (KFD) has also shown great potential for dealing with biometric recognition problems. In summary, discrimination technologies play an important role in the implementation of biometric systems. They provide methodologies for automated personal identification or verification. In turn, the applications in biometrics also facilitate the development of discrimination methodologies and technologies, making discrimination algorithms more suitable for image feature extraction and recognition.

Currently discriminant analysis has been widely applied to face, ear, fingerprint, gait recognition, and even multi-modal biometrics. Further, the increasing demand for reliable and convenient biometric system is also contributing to the development and improvement of linear/nonlinear discriminant analysis techniques.

A tensor is a higher order generalization of a vector or a matrix. In fact, a vector is a first-order tensor and a matrix is a tensor of order two. Furthermore speaking, tensors are multilinear mapping over a set of vector spaces. If we have data in three or more dimensions, then we mean to deal with a higher-order tensor. Tensor presents a generalized representation of biometric data. To deal with the diversity in biometric data forms, families of tensor discriminant analysis technologies have been investigated. Nowadays, tensor principal compoenent analysis, tensor discriminant analysis, tensor independent component analysis, and other tensor analysis approaches have been successfully applied to face, palm print, and gait recognition.

Biometric data discrimination technologies can be briefly defined as automated methods of feature extraction and recognition based on given biometric data. It should be stressed that the biometric data discrimination technologies are not the simple application of classical discrimination techniques to biometrics, but are in fact improved or reformed discrimination techniques that are more suitable for biometric applications, for example by having a more powerful recognition performance or by being computationally more efficient for feature extraction or classification. In other words, the biometric data discrimination technologies are designed for extracting features from biometrics data, which are characteristically high-dimensional, large scale, and offer only a small sample size. The following explains these characteristics more fully.

HIGH DIMENSIONALITY

In biometric recognition, high dimensional data usually are expected to be more powerful. The high-dimensionality of biometric data, however, would make direct classification (e.g. the so-called correlation method that uses a nearest neighbour classifier) in original data space almost impossible, firstly because the similarity (distance) calculation is very computationally expensive, secondly because it demands large amounts of storage. High dimensionality makes it necessary to use a dimension reduction technique prior to recognition.

LARGE SCALE

Real- world biometric applications are often large-scale, which means biometric systems should be operated in large population databases. Typical examples of this would include welfare-disbursement, national ID cards, border control, voter ID cards, driver’s licenses, criminal investigation, corpse identification, parenthood determination, and the identification of missing children. Given an input biometric sample, a large-scale biometric identification system determines whether the pattern is associated with any of a large number (e.g., millions) of enrolled identities. These large-scale biometric applications require high-quality and very generalizable biometric data discrimination technologies.

SAMPLE QUALITY

Biometric systems automatically capture, detect and recognizing biometric image, making it inevitable that biometric data will sometimes be noisy or partially corrupted. The capture and communication of biometric data itself may introduce noise; some accessories will cause the partial corruption of biometric data, for example a scarf may occlude a facial image. Because all these factors are inevitable, the development of biometric system should always address the robust feature extraction and recognition of noisy or partially corrupted biometric data.

SMALL SAMPLE SIZE

Unlike, for example, optical character recognition (OCR) problems, the training samples per class that are available in real-world biometric recognition problems are always very limited. Indeed, there may be only one sample available for each individual. Combined with high-dimensionality, small sample size creates the so-called small sample size (or under-sampled) problems. In these problems, the withinclass scatter matrix is always singular because the training sample size is generally less than the space dimension. As a result, the classical LDA algorithm becomes infeasible in image vector space.

MULTI-BIOMETRICS

Verification or identification accuracy is always the first-ofall objective for biometric systems. Unibiometric system, the biometric system using a single biometric characteristic, usually suffers from some limitations and cannot provide satisfactory recognition performance. For example, manual workers with damaged or dirty hands may not be able to provide high-quality fingerprint images, and thus failure to enrol would happen for single fingerprint recognition system.

Multi-biometric systems, which integrate information from multiple biometric traits, provide some effective means to enhance the performance and reliability of the biometric system. To combine information from individual biometric traits, there are three categories of fusion strategies, feature level fusion, matching score level fusion, and decision level fusion. In feature level fusion, the data obtained from each sensor is used to compute a feature vector. As the feature extracted from one biometric trait is independent of that extracted from the other, it is reasonable to concatenate the two vectors into a single new vector for performing multibio- metric based personal authentication. Note that the new feature vector now has a higher dimensionality than the original feature vector generated from each sensor. Feature reduction techniques may be employed to extract useful features from the set of the new feature vector. In matching score level fusion, each subsystem using one biometric trait of the multi-biometric system provides a matching score indicating the proximity of the feature vector with the template vector. These scores can be combined to assert the veracity of the claimed identity. In decision level fusion each sensor first acquire one of multiple biometric traits and the resulting feature vectors are individually classified into the two decisions---accept or reject the claimed identity.

Then a scheme that exploits the known decisions to make the final decision is used. In the field of multi-biometrics, a great number of studies of feature level fusion, matching score level fusion and decision level fusion have been made. Though fusion of multi-biometrics are generally recognized as three classes as described above, in real-world applications of multi-modal biometric it is possible that the “Fusion Process” may be simultaneously involved in different levels such as in both the matching score level and the decision level. Recent decades have witnessed the development and prosperity of biometric data discrimination technologies. Various unsupervised/supervised, linear/nonlinear, vector/tensor discrimination technologies have been investigated and successfully applied to biometric recognition. At the beginning, linear unsupervised method, principal component analysis (PCA), was used to extract the holistic feature vectors for facial image representation and recognition (Sirovich & Kirby, 1987; Kirby & Sirovich, 1990; Turk & Pentland, 1991a & 1991b). Since then, PCA has been widely investigated and has become one of the most successful approaches to face recognition (Pentland, Moghaddam, & Starner, 1994; Pentland, 2000; Zhao & Yang, 1999; Moghaddam, 2002; Zhang, 2002; Kim, H. C., Kim, D., Bang, & Lee, 2004) and palm print recognition (Lu, Plataniotis, & Venetsanopoulos, 2003b). Other popular unsupervised methods, such as independent component analysis (ICA) and non-negative matrix factorization (NMF), have been applied to biometric recognition (Bartlett et al., 2002; Yuen & Lai, 2002; Liu & Wechsler, 2003; Draper, Baek, Bartlett, & Beveridge, 2003; Petridis & Perantonis, 2004).

Since the unsupervised methods do not utilize the class label information in the training stage, it is generally believed that the supervised methods are more effective in dealing with recognition problems. Fisher linear discriminant analysis (LDA), which aims to find a set of the optimal discriminant projection vectors that map the original data into a lowdimensional feature space, is then gaining popularity in biometric recognition research. In 1986, Fisher linear discriminant analysis was first applied to image classification (Tian, Barbero, Gu, & Lee, 1986). Further, LDA was applied to face recognition, and subsequently was developed into one of the most famous face recognition approaches, Fisherfaces (Liu, K. Cheng, Yang, & Liu, X. 1992; Swets & Weng, 1996; Belhumeur, Hespanha, & Kriengman, 1997). In biometric recognition, the data dimensionality is much higher than the size of the training set, leading to the well-known small sample size (SSS) problem. Currently there are two popular strategies to solve the SSS problem, the transform-based and the algorithmbased (Yang & Yang, 2003; Jian, Yang, Hu, & Lou, 2001; Chen, Liao, Lin, Kao, & Yu, 2000; Yu & Yang, 2001; Lu et al., 2003a; Liu & Wechsler, 2000 & 2001; Zhao et al., 1998; Loog, Duin, & Haeb-Umbach, 2001; Duin & Loog, 2004; Ye, 2004; Howland & Park, 2004). The transform-based strategy first reduces the dimensions of the original image data and then uses LDA for feature extraction. Typical transform-based methods include PCA+LDA and uncorrected LDA. The algorithm-based strategy finds an algorithm for LDA that can circumvent the SSS problem. Some representative algorithm-based methods can avoid the SSS problem, but most algorithm-based methods are computationally expensive or lose parts of important discriminatory information.

Biometric recognition usually is highly complex and can not be regarded as a linear problem. In the last few years, a class of nonlinear discriminant analysis techniques named as kernel-based discriminant analysis has been widely investigated for biometric data discrimination. Kernel principal component analysis (KPCA) and kernel Fisher discriminant (KFD) are two of the most representative nonlinear methods and have received considerable interests in the fields of biometrics, pattern recognition, and machine learning. By far, a number of kernel-methods, such as KPCA, KFD, complete kernel Fisher discriminant (CKFD), and kernel direct discriminant analysis (KDDA), have been developed from biometric recognition (Schölkopf et al., 1998; Mika Rätsch, Schölkopf, Smola, Weston, & Müller,1999a & 1999b; Baudat & Anouar, 2000; Roth & Steinhage, 2000; Mika, Rätsch, & Müller, 2001a & 2001b; Mika et al., 2003; Yang, 2002; Lu et al., 2003b; Xu, Zhang, & Li, 2001; Billings & Lee, 2002; Gestel, Suykens, Lanckriet, Lambrechts, De Moor, & Vanderwalle, 2002; Cawley & Talbot, 2003; Lawrence & Schölkopf, 2001; Liu, 2004; Yang, Zhang, & Lu, 2004a & 2004b; Xu, Yang, J. Y, & Yang, J., 2004; Yang, Zhang, Yang, Zhong, & Frangi, 2005). Most recently, manifold learning methods, such as isometric feature mapping (ISOMAP), locally linear embedding (LLE), and Laplacian eigenmaps, have also shown great potential in biometric recognition (Tenenbaum, 2000; Roweis & Saul, 2000; Belkin & Niyogi, 2002).

As a generalization of vector-based methods, a number of tensor discrimination technologies have been proposed. The beginning of tensor discrimination technology can be traced back to 1993, where a 2D image matrix based algebraic feature extraction method is proposed for image recognition (Liu, Cheng, & Yang, 1993). As a new development of the 2D image matrix based straightforward projection technique, a two-dimensional PCA (2DPCA) and uncorrelated image projection analysis were suggested for face representation and recognition (Yang, Zhang, Frangi, & Yang, 2004c; Yang. J, Yang, J. Y., Frangi, A. F., & Zhang, 2003b). To reduce the computational cost of 2DPCA, researchers have developed several BDPCA and generalized low rank approximation of matrices (GLRAM) approaches (Zuo, Wang, & Zhang, 2005; Ye, 2004; Liang & Shi 2005; Liang, Zhang, & Shi, 2007). Motivated by multilinear generalization of singular vector decomposition, a number of alterative supervised and unsupervised tensor analysis methods have been proposed for image or image sequence feature extraction (Lathauwer, Moor, & Vndewalle, 2000; Vasilescu, & Terzopoulos 2003; Tao, Li, Hu, Maybank, & Wu, 2005; Yan, Xu, Yang, Zhang, Tang, & Zhang, 2007).

Multi-biometric system is designed to overcome the limitations of any single biometric systems by fusing information from multiple biometric traits. The fusion can be implemented in either of three levels, feature level, matching score level, and decision level. In feature level fusion, a new feature vector is constructed using the concatenation rule (Ross & Govindarajan, 2005), the parallel rule (Yang et al., 2003a; Yang, J., & Yang, J. Y., 2002), or the competitive rule (Kong, Zhang, & Kamel, 2006). In matching score level fusion, a number of transformation-based (Jain, Nandakumar, & Ross, 2005; Zuo, Wang, Zhang, D., & Zhang, H., 2007), clas-sifierbased (Brunelli & Falavigna, 1995; Jain, Prabhakar, & Chen, 1999b; Fierrez-Aguilar, Ortega-Garcia, Gonzalez- Rodriguez, & Bigun, 2005), and density-based (Ulery, Hicklin, Watson, Fellner, & Hallinan, 2006; Nandakumar, Chen, Jain, & Dass, 2006) score fusion methods have been used to combine scores of multiple scores. In decision level fusion, boolean conjunctions, weighted decision methods, classical inference, Bayesian inference, Dempster–Shafer method, and voting have been proposed to make the final recognition decision (Gokberk, Salah, & Akarun, 2003; Jing, Zhang, D., & Yang, 2003).

MODEL BASED FACE RECONSTRUCTION

RELATED WORK

In this section, we review methods for shape reconstruction. First, we describe methods which have been used for restoring (acquiring) geometrical data considering only shape. The following gives methods where reconstruction is done to acquire both shape and structure. With restored structure, the reconstruction model can be animated.

SHAPE RECONSTRUCTION

To get a detailed matched shape, we need time-consuming manual job, a sophisticated equipment, or complicated algorithm. Most of them need one more process to get structured shape for animation. In this section we focus on a few methods to get detailed range data for face.

Plaster Model Magnenat Thalmann et al. [1987] used plaster models in real world and selected facets and vertices marking on the models which are photographed from various angles to be digitized. Here the reconstruction approach requires a mesh drawn on the face and is time consuming, but can obtain high resolution in any interested area.

Laser Scanning In range image vision system some sensors, such as scanners, yield range images. For each pixel of the image, the range to the visible surface of the objects in the scene is known. Therefore, spatial location is determined for a large number of points on this surface. An example of commercial 3D digitizer based on laser-light scanning, is Cyberware Color DigitizerTM. Lee et al. [1996] digitized facial geometry through the use of scanning range sensors. However, the approach based on 3D digitization requires special high-cost hardware and a powerful workstation.

Recently, laser-based 3D range scanners have been commercially available. Examples include CyberwareTM [Cyberware, 2003] scanner, EyetronicsTM scanner [Eyetronics, 2003], and etc. CyberwareTM scanner shines a safe, low-intensity laser on a human face to create a lighted profile. A video sensor captures this profile from two viewpoints. The laser beam rotates around the face 360 degrees in less than 30 seconds so that the 3D shape of the face can captured by combining the profiles from every angle. Simultaneously, a second video sensor in the scanner acquires color information. EyetronicsTM scanner shines a laser grid onto the human facial surface. Based on the deformation of the grid, the geometry of the surface is computed. Comparing these two systems, EyetronicsTM is a “one shot” system which can output 3D face geometry based on the data of a single shot. In contrast, CyberwareTM scanner need to collect multiple profiles in a full circle which takes more time.

In post-processing stage, however, EyetronicsTM needs more manual adjustment to deal with noisy data. As for the captured texture of the 3D model, EyetronicsTM has higher resolution since it uses high resolution digital camera, while texture in CyberwareTM has lower resolution because it is derived from low resolution video sensor. In summary, these two ranger scanners have different features and can be used to capture 3D face data in different scenarios.

Based on the 3D measurement using these ranger scanners, many approaches have been proposed to generate 3D face models ready for animation. Ostermann et al. [Ostermann et al., 1998] developed a system to fit a 3D model using CyberwareTM scan data. Then the model is used for MPEG-4 face animation.

Lee et al. [Lee et al., 1993, Lee et al., 1995] developed techniques to clean up and register data generated from CyberwareTM laser scanners. The obtained model is then animated by using a physically based approach. Marschner et al. [Marschner et al., 2000] achieved the model fitting using a method built upon fitting subdivision surfaces.

Stripe Generator As an example of structured light camera range digitizer, a light striper with a camera and stripe pattern generator can be used for face reconstruction with relatively cheap equipment compared to laser scanners. Stripe pattern is projected on the 3D object surface and it is taken by a camera. With information of posit ions of projector and camera and stripe pattern, a 3D shape can be calculated. Proesmans et al. [1997] shows a good dynamic 3D shape using a slide projector, by a frame-by-frame reconstruction of a video.

Lighting Switch Photometry Lighting Switch Photometry uses three or more light sources for computing normal vectors for extracting shapes of static objects , Coleman EN et.al.[1982] or a moving human face , Hrroshi Saji et.al.[1992]. This method assumes that the reflectance map is lambertian. By Lighting Switch Photometry, the normal vector can be computed at the points where three incident light sources illuminate. It is difficult to compute the accurate normal vector at the point where the intensity of radiance is small, such as shadowed regions.

Stereoscopy A distance measurement method such as stereo can establish the correspondence at certain characteristic points. The method uses the geometric relation over stereo images to recover the surface depth. The method usually results in sparse spatial data. Fua and Leclerc [1996] used it mainly in textured areas by weighting the stereo component most strongly for textured image areas and the shading component most strongly for texture-less areas.

STRUCTURED SHAPE RECONSTRUCTION

Most of the above methods concentrate on recovering a good shape, but the biggest drawback is that they provide only the shape without structured information. To get a structured shape for animation, most typical way is to modify an available generic model with structural information such that eyes, lips, nose, hair and so on. We classify methods using range data1 and without using range data.

WITH RANGE DATA

The plaster marking method by Magnenat Thalmann et al. [1987] mentioned in previous section has structure for animation because each point has its own labeling corresponding to animation model. Except in this method, it is necessary to add a structural information to a set of 3D points to make the model suitable for animation.

Warping Kernels Williams [1990] reconstructed a head using CyberwareTM digitizer and apply warpware to animate the model. A set of warping kernels is distributed around the face, each of which is a Hanning (cosine) window, scaled to 1.0 in the center, and diminishing smoothly to 0.0 at the edge.

Mesh Adaptation Starting with a structured facial mesh, Lee et al. [1996] developed algorithms that automatically construct functional models of the heads of human subjects from laser-scanned range and reflection data. After getting the large arrays of data acquired by the scanner, they reduce it into a parsimonious geometric model of the face that can eventually be animated efficiently. They adapt a generic face mesh to the data. Once the mesh has been fitted by the feature based matching technique, the algorithm samples the range image at the location of the nodes of the face mesh to capture the facial geometry. The node positions also provide texture map coordinates that are used to map the full resolution color image onto the triangles.

While 3D face recognition research dates back to before 1990, algorithms that combine results from 3D and 2D data did not appear until about 2000. Most efforts to date in this area use relatively simplistic approaches to fusing results obtained independently from the 3D data and the 2D data. The single most common approach has been to use an eigenface type of approach on each of the 2D and 3D independently, and then combine the two matching scores. However, more recent works appear to take a variety of quite different approaches. Interestingly, several commercial face recognition companies already have capabilities for multi-modal 3D + 2D face recognition.

Lao et al. [2000] perform 3D face recognition using a sparse depth map constructed from stereo images. Iso-luminance contours are used for the stereo matching. Both 2D edges and iso-luminance contours are used in finding the irises. In this specific limited sense, this approach is multi-modal. However, there is no separate recognition result from 2D face recognition. Using the iris locations, other feature points are found so that poses standardization can be done. Recognition is performed by the closest average difference in corresponding points after the data are transformed to a canonical pose. Recognition rates of 87–96% are reported using a dataset of 10 persons, with four images taken at each of nine poses for each person. Beumier and Acheroy [2001] approach multi-modal recognition by using a weighted sum of 3D and 2D similarity measures. They use a central profile and a lateral profile, each in both 3D and 2D. Therefore they have a total of four classifiers, and an overall decision is made using a weighted sum of the similarity metrics. A data set representing over 100 persons imaged on multiple sessions, with multiple poses per session, is acquired. Portions of this data set have been used by several other researchers [C.Xu et.al. (2004), B.Gokberk et.al.(2005)]. In this paper, results are reported for experiments on a subset of the data, using a 27-person gallery and a 29-person probe set. An equal-error rate as low as 1.4% is reported for multimodal 3D + 2D recognition that merges multiple probe images per subject. In general, multi-modal 3D + 2D is found to perform better than either 3D or 2D alone.

Wang et al. [2002] use Gabor filter responses in 2D and ‘‘point signatures’’ in 3D to perform multi-modal face recognition. The 2D and 3D features together form a feature vector. Classification is done by support vector machines with a decision directed acyclic graph (DDAG).

Experiments are performed with images from 50 subjects, six images per subject, with pose and expression variations. Recognition rates exceeding 90% are reported.

Bronstein et al. [2003] use an isometric transformation approach to 3D face analysis in an attempt to better cope with variation due to facial expression. One method they propose is effectively multi-modal 3D + 2D recognition using eigen decomposition of flattened textures and canonical images. They show examples of correct and incorrect recognition by different algorithms, but do not report any overall quantitative performance results for any algorithm.

Tsalakanidou et al. [2003] report on multi-modal face recognition using 3D and color images. The use of color rather than simply gray-scale intensity appears to be unique among the multi-modal work surveyed here. Results of experiments using images of 40 persons from the XM2VTS dataset [1999] are reported for color images alone, 3D alone, and 3D + color. The recognition algorithm is PCA-style matching, followed by a combination of the results for the individual color planes and range image. Recognition rates as high as 99% are achieved for the multi-modal algorithm, and multi-modal performance is found to be higher than for either 3D or 2D alone.

Chang et al. [2003] report on PCA-based recognition experiments performed using 3D and 2D images from 200 persons. One experiment uses a single set of later images for each person as the probes. Another experiment uses a larger set of 676 probes taken in multiple acquisitions over a longer elapsed time. Results in both experiments are approximately 99% rank-one recognition for multi-modal 3D + 2D, 94% for 3D alone, and 89% for 2D alone. The multi-modal result was obtained using a weighted sum of the distances from the individual 3D and 2D face spaces.

Godil et al. [2005] present results of 3D + 2D face recognition using 200 persons worth of data taken from the CAESAR anthropometric database. They use PCA for matching both the 2D and the 3D, with the 3D represented as a range image. The 3D face data from this database may be rather coarse, with approximately 4000 points reported on the face. Multiple approaches to score-level fusion of the two results are explored. Performance as high as 82% rankone recognition is reported.

Papatheodorou and Rueckert [2004] perform multi-modal 3D + 2D face recognition using a generalization of ICP based on point distances in a 4D space (x, y, z, intensity).

This approach integrates shape and texture information at an early stage, rather than making a decision using each mode independently and combining decisions. They present results from experiments with 62 subjects in the gallery, and probe sets of varying pose and facial expression from the images in the gallery. They report 98–100% correct recognition in matching frontal, neutral-expression probes to frontal neutral-expression gallery images. Recognition drops when the expression and pose of the probe images is not matched to those of the gallery images, for example to the range of 73–94% for 45_ off-angle probes, and to the range of 69–89% for smiling expression probes. Tsalakanidou et.al. [2004] report on an approach to multi-modal face recognition based on an embedded hidden Markov model for each modality.

Their experimental data set represents a small number of different persons, but each has 12 images acquired in each of five different sessions. The 12 images represent varied pose and facial expression. Interestingly, they report a higher EER for 3D than for 2D in matching frontal neutralexpression probes to frontal neutral-expression gallery images, 19% versus 5%, respectively. They report that ‘‘depth data mainly suffers from pose variations and use of eyeglasses’’ [M.L. Koudelka et.al.(2005)]. This work is also unusual in that it is based on using five images to enroll a person in the gallery, and also generates additional synthetic images from those, so that a person is represented by a total of 25 gallery images. A longer version of this work appears in [F. Tsalakanidou et.al.(2005)].

Hu¨sken et al. [2005] describe the Viisage approach to multi- modal recognition. The 3D matching follows the style of hierarchical graph matching already used in Viisages 2D face recognition technology. This is felt to allow greater speed of matching in comparison to techniques based on ICP or similar iterative techniques. Fusion of the results from the two modalities is done at the score level. Multimodal performance on the FRGC version 2 data set is reported as 93% verification at 0.01 FAR. In addition, it is reported that performance of 2D alone is only slightly less than multi-modal performance, and that performance of 3D alone is substantially less than that of 2D alone. In this context, it may be interesting to note that results from a group (Geometrix) that originally focused on 3D face recognition show that 3D alone outperforms 2D alone, whereas results from a group (Viisage) that originally focused on 2D alone show that 2D alone outperforms 3D alone.

Lu et al. [2005] build on earlier work with ICP style matching of 3D shape [X. Lu et.al.(2004)] to create a 3D + 2D multi-modal system. They use a linear discriminant analysis approach for the 2D matching component. Their experimental data set consists of multiple scans of each of 100 persons. Five scans with a Minolta Vivid 910 system are taken in order to create a 3D face model for enrolling a person. Enrollment is done with neutral expression. Six scans are taken of each person, three with neutral expression, and three with smiling expression, to use as individual probes for testing. They report better performance with 3D matching alone than with 2D matching alone. They also report 98% rank-one recognition for 3D + 2D recognition on neutral expressions alone, and 91% on the larger set of neutral and smiling expressions.

Maurer et al. [2005] describe the Geometrix approach to multi-modal 3D + 2D face recognition. The 3D matching builds on the approach described by Medioni and Waupotitsch [2005], whereas the 2D matching uses the approach of Neven Vision. A weighted sum rule is used to fuse the two results, with the exception that ‘‘when the shape score is very high, we ignore the texture score’’ [T. Maurer et.al.(2005)].

Experimental results are presented for the FRGC version two data set. The facial expression variations in this dataset are categorized into ‘‘neutral,’’ ‘‘small,’’ and ‘‘large’’ and results are presented separately for these three categories. Multi-modal performance for the ‘‘all versus all’’ matching of the 4007 images reaches approximately 87% verification at 0.01 FAR. They also report that 3D + 2D outperforms 3D alone by a noticeable increment, and that the verification rates for 2D alone are below those for 3D alone.

WITHOUT RANGE DATA

The approach based on 3D digitization to get a range data often requires special purpose high-cost hardware. So a common way of creating 3D objects is reconstruction from 2D information which is accessible at low price. Two commonly used methods are an interactive deformation method which modifies or generates a surface employing deformation, and a reconstruction method with feature points which modifies a generic model after feature detection.

Interactive deformation Magnenat Thalmann et al. [1995] used an interactive tool to generate a polygon mesh surface for creating figures. The major operations performed include creation of primitives, selection, local deformations and global deformations. It is more tedious and time consuming. However, it may be only possible way to digitize a historical personage whose pictorial or other source is not available and is useful to invent new characters.

Reconstruction with feature points There are faster approaches to reconstruct a face shape from few pictures of a face [Horace H.S.et.al.1996, Takaaki Akimoto et.al.1993, Kurihara et.al.1991]. In this method, a generic model in 3D is provided in advance, and a limited number of feature points are detected either automatically or interactively on the two (or more) orthogonal pictures, and the other points on the generic model are modified by a special function. Then 3D points are calculated by just combining several 2D coordinates.

Kurihara and Arai [1991] used an interactive method to get a few points, and a Delaunay triangulation for the conformation of the face and texture mapping. The result seems nice, but a big drawback is that they use too few points to modify the generic model. So if the generic model has very different shape, the result may not be similar to the person and texture mapping may also not work well. To increase accuracy, one should increase input points for modification of generic model. Ip and Yin [1996] have very similar approach to the one of Akimoto et al. [1993]. These two approaches tried to detect feature points automatically using dynamic template matching or LMCT (Local Maximum-Curvature Tracking) checking concave and convex points on the side profile of a face and a very simple filtering method to get interior points. It was a trial for automation, but the method they use to detect points does not seem to be very robust. In addition LMCT was designed to calculate convex or concave points which works well only for Mongoloid looking people.

METHODS OF 3D FACE MODELS

Zhao et.al. (2003) describes, Many methods of face recognition have been proposed during the past many years. Face recognition is such a challenging yet interesting problem that it has attracted researchers who have different backgrounds: behavioral, psychology, pattern recognition, neural networks, computer vision, and computer graphics. It is due to this fact that the literature on face recognition is vast and diverse. Often, a single system involves techniques motivated by different principles. The usage of a mixture of techniques makes it difficult to classify these systems based purely on what types of techniques they use for feature representation or classification. To have a clear and highlevel categorization, we instead follow a guideline suggested by the psychological study of how humans use holistic and local features. Specifically, we have the following categorization:

HOLISTIC MATCHING METHODS:

These methods use the whole face region as the raw input to a recognition system. It covers Principal-component analysis (PCA), Eigenfaces, Probabilistic Eigenfaces, Fisherfaces/subspace LDA, SVM, Evolution pursuit, Feature lines, ICA, Other representations- LDA/FLD, PDBNN(Probabilistic decision based NN).

FEATURE-BASED (STRUCTURAL) MATCHING METHODS.

Typically, in these methods, local features such as the eyes, nose, and mouth are first extracted and their locations and local statistics (geometric and/or appearance) are fed into a structural classifier. It covers Pure geometry methods, Dynamic link architecture, Hidden Markov model, Convolution Neural Network.

HYBRID METHODS.

Just as the human perception system uses both local features and the whole face region to recognize a face, a machine recognition system should use both. One can argue that these methods could potentially offer the better of the two types of methods. It covers Modular Eigenfaces, Hybrid LFA, Shape-normalized, Component-based.

OTHER METHODS

Few papers on this topic have been published even if 3D face recognition research started in last eighties. Many criteria can be adopted to compare existing 3D face algorithms by taking into account the type of problems they address or their intrinsic properties. Indeed, some approaches perform very well only on faces with neutral expression, while some others try also to deal with expression changes. An additional parameter to measure 3D models based robustness is represented by how sensitiveness they are to size variation. In fact, sometimes the distance between the target and the camera can affect the size of the facial surface, as well as its height, depth, etc. Therefore, approaches exploiting a curvature-based representation cannot distinguish between two faces with similar shape, but different size.

In order to overcome this problem some methods are based on point-to-point comparison or on volume approximation. However, the absence of an appropriate standard dataset containing large number and variety of people, whose images were taken with a significant time delay and with meaningful changes in expression, pose and illumination, is one of the great limitations to empirical experimentation for existing algorithms.

In particular, 3D face recognition systems are tested on proprietary databases, with few models and with a limited number of variations per model. Consequently, comparing different algorithms performances often turns into a difficult task. Nevertheless, they can be classified based on the type of problems they address such as mesh alignment, morphing, etc.

The goal for this section is to present a terse description of most recent 3D based face recognition algorithms. Methods have been grouped in three main categories: 2D image based, 3D image based and multimodal systems. The first category includes methods based on comparisons among intensity images, but supported by a three-dimensional procedure that increases the system robustness. The second class groups approaches based on 3D facial representation, like range images or meshes. Finally, methods combining 2D image and 3D image information fall in the third category.

2D-BASED CLASS

Approaches based on 2D images supported by some 3D data are identified as 2D-based class methodologies. Generally, the idea is to use a 3D generic face model to improve robustness with respect to appearance variations such as hard pose, illumination and facial expression. An example of this approach is given by Blanz and Vetter (2003). They proposed to synthesize various facial variations by using a morphable model that augments the given training set containing only a single frontal 2D image for each subject. The morphable face is a parametric model based on a vector space representation of faces. This space is constructed so that any convex combination of shape and texture vectors belonging to the space describes a human face. Given a single face image, the algorithm automatically estimates 3D shape, texture, and all relevant 3D scene parameters like pose, illumination, etc. (see Fig. 7), while the recognition task is achieved measuring the Mahalanobis distance (Duda et al., 2001) between the shape and texture parameters of the models in the gallery and the fitting model. The identification has been tested on two publicly available databases of images: CMU-PIE (Sim et al., 2003) and FERET (Phillips et al., 2000). A recognition rate of 95% on CMUPIE dataset and 95.9% on FERET dataset is claimed.

Another interesting approach using a 3D model to generate various 2D facial images is given by Lu et al. (2004). They generated a 3D model of the face from a single frontal image. From this 3D model many views are synthesized to simulate new poses, illuminations and expressions. Tests are performed by measuring dissimilarities among affine subspaces according to a given distance measure. In particular, an affine subspace contains all the facial variations synthesized for a single subject. They performed experiments on a dataset of 10 subjects building 22 synthesized images per subject with different poses, facial expressions and illuminations. The method achieves a recognition rate of 85%, outperforming the PCA-based methods on this dataset. Nevertheless, very few people are in the database, making difficult to estimate accurately the real discriminating power of the method. On the contrary, Hu et al. (2004) show that linear methods such as PCA and LDA can be further extended to cope with changes in pose and illumination by using a Nearest Neighbor approach. The dataset is gathered on 68 subjects and 41.368 bi-dimensional images under various facial expression, illuminations and poses. Their results show that using virtual face for particular poses increase the recognition rate and the highest rate reached 95% when pose is approximately frontal and LDA is used.

Creating various 2D synthetic faces could be good way to overcome the classical problems of 2D face recognition, but two important considerations have to be carefully examined: ‘‘how much realistic is a synthesized face?’’ and ‘‘how precise can a 3D facial reconstruction taken by one single picture be?’’. First of all, we have to consider that modern 3D computer graphics technologies are able to reproduce synthetic images in an excellent realistic way and with an accurate geometric precision. Secondly, we have to consider that 3D facial reconstruction from a single view image can be considered good enough, only if the experimental results show a high discriminating power.

FACE MODELING USING 2D IMAGES

A number of researchers have proposed to create face models from 2D images. Some approaches use two orthogonal views so that the 3D information of facial surface points can be measured [Akimoto et al., 1993, Dariush et al., 1998, H.S.Ip and Yin, 1996]. They require two cameras which must be carefully set up so that their directions are orthogonal. [Zheng, 1994] developed a system to construct geometrical object models from image contours.

The system requires a turn-table setup. Pighin et al. [Pighin et al., 1998] developed a system to allow a user to manually specify correspondences across multiple images, and use computer vision techniques to compute 3D reconstructions of specified feature points. A 3D mesh model is then fitted to the reconstructed 3D points. With a manually intensive procedure, they were able to generate highly realistic face models. Fua and Miccio [Fua and Miccio, 1998] developed system which combine multiple image measurements, such as stereo data, silhouette edges and 2D feature points, to reconstruct 3D face models from images. Because the 3D reconstructions of face points from images are either noisy or require extensive manual work, researcher have tried to use prior knowledge as constraints to help the image-based 3D face modeling. One important type of constraints is the “linear classes” constraint. Under this constrain, it assumes that arbitrary 3D face geometry can be represented by a linear combination of certain basic face geometries. The advantage of using linear class of objects is that it eliminates most of the non-natural faces and significantly reduces the search space. Vetter and Poggio [Vetter and Poggio, 1997] represented an arbitrary face image as a linear combination of some number of prototypes and used this representation (called linear object class) for image recognition, coding, and image synthesis. In their representative work, Blanz and Vetter [Blanz and Vetter, 1999] obtain the basis of the linear classes by applying Principal Component Analysis (PCA) to a 3D face model database. The database contains models of 200 Caucasian adults, half of which are male. The 3D models are generated by cleaning up, registering the CyberwareTM scan data. Given a new face image, a fitting algorithm is used to estimate the coefficients of the linear combination. They have demonstrated that linear classes of face geometries and images are very powerful in generating convincing 3D human face models from images. For this approach to achieve convincing results, it requires that the novel is similar to faces in the database and the feature points of the initial 3D model is roughly aligned with the input face image.

Because it is difficult to obtain a comprehensive and high quality 3D face database, other approaches have been proposed using the idea of “linear classes of face geometries”. Kang and Jones [Kang and Jones, 1999] also use linear spaces of geometrical models to construct 3D face models from multiple images. But their approach requires manually aligning the generic mesh to one of the images, which is in general a tedious task for an average user. Instead of representing a face as a linear combination of real faces, Liu et al. [Liu et al., 2001b] represent it as a linear combination of a neutral face and some number of face metrics where a metric is a vector that linearly deforms a face. The metrics in their systems are meaningful face deformations, such as to make the head wider, make the nose bigger, etc. They are defined interactively by artists.

S.Jaiswal et.al.[2010] given a comprehensive literature on Image Based human and machine recognition of faces during 1987 to 2010. Machine recognition of faces has several applications. As one of the most successful applications of image analysis and understanding, face recognition has recently received significant attention, especially during the past several years. In addition, relevant topics such as Brief studies, system evaluation, and issues of illumination and pose variation are covered. In this paper numerous method which related to image based 3D face recognition are discussed. S.Jaiswal et.al.[2007] described an efficient method and algorithm to make individual faces for animation from possible inputs. Proposed algorithm reconstruct 3D facial model for animation from two projected pictures taken from front and side views or from range data obtained from any available resources. It is based on extracting features on a face in automatic way and modifying a generic model with detected feature points with conic section and pixalization. Then the fine modifications follow if range data is available. The reconstructed 3Dface can be animated immediately with given parameters. Several faces by one methodology applied to different input data to get a final Animatable face are illustrated.

S.Jaiswal et.al.[2007] the proposed study, 2D photographs image divided into two parts; one part is front view (x, y) and side view (y, z). Necessary condition of this method is that position or coordinate of both images should be equal. We combine both images according to the coordinate then we will get 3D Models (x, y, z) but this 3D model is not accurate in size or shape. In defining other words, we will get 3D animatable face, refinement of 3D animatable face through pixellization and smoothing process. Smoothing is performed to get the more realistic 3D face model for the person.

Security is the one of the main concern in today’s world. Whether it is the field of telecommunication, information, network, data security, airport or home security, national security or human security, there are various technique for the security. Biometric is one of the mode of it. A biometrics is, “Automated methods of recognizing an individual based on their unique physical or behavioral characteristics.” Face recognition is a task humans perform remarkably easily and successfully. This apparent simplicity was shown to be dangerously misleading as the automatic face recognition seems to be a problem that is still far from solved. In spite of more than 20 years of extensive research, large number of papers published in journals and conferences dedicated to this area, we still can not claim that artificial systems can measure to human performance. Automatic face recognition is intricate primarily because of difficult imaging conditions (lighting and viewpoint changes induced by body movement) and because of various other effects like aging, facial expressions, occlusions etc. Researchers from computer vision, image analysis and processing, pattern recognition, machine learning and other areas are working jointly, motivated largely by a number of possible practical applications. A general statement of the face recognition problem (in computer vision) can be formulated as follows: Given still or video images of a scene, identify or verify one or more persons in the scene using a stored database of faces. Face recognition is one of the most active and widely used techniques because of its reliability and accuracy in the process of recognizing and verifying a person’s identity. The need is becoming important since people are getting aware of security and privacy. For the Researchers Face Recognition is among the tedious work. It is all because the human face is very robust in nature; in fact, a person’s face can change very much during short periods of time (from one day to another) and because of long periods of time (a difference of months or years). One problem of face recognition is the fact that different faces could seem very similar; therefore, a discrimination task is needed. On the other hand, when we analyze the same face, many characteristics may have changed. These changes might be because of changes in the different parameters. The parameters are: illumination, variability in facial expressions, the presence of accessories (glasses, beards, etc); poses, age, finally background. We can divide face recognition techniques into two big groups, the applications that required face identification and the ones that need face verification. The difference is that the first one uses a face to match with other one on a database; on the other hand, the verification technique tries to verify a human face from a given sample of that face.

Principal components analysis (PCA) and linear discriminant analysis (LDA) are widely used in face recognition system. These methods can efficiently reduce the dimensions of biometric data and improve the robustness to disturbing factors like expression variance, wearing glasses, mimic, etc. Due to these advantages, they are popular with commercial face recognition system providers.

However, the strong dimension reduction of PCA-LDA algorithms limits its integration with in the template protection techniques. In our work, we selected three techniques for comparative study and evaluation, using a common face data base that contains overall 360 images. The three techniques are Principal Component Analysis (eigenface) , Regularized Linear Discriminant Analysis (R-LDA), and Morphological Method. These all are coupled with artificial neural networks for training and classification of extracted features. These techniques are having apparently promising performances and are representative of new trends in face recognition. All three techniques were reported to have recognition rates of more than 80–90% on data bases of moderate sizes (e.g., 16–50 persons). We believe this work would be a useful complement to, where the surveyed techniques were not evaluated on a common data base of relatively large size. Indeed, through a more focused and detailed comparative study of three important techniques, our goal is to gain more insights into their underlying principles, interrelations, advantages, limitations, and design tradeoffs and, more generally, into what the critical issues really are for an effective recognition algorithm. Basically we have used two different approaches for feature extraction of image:

MORPHOLOGICAL APPROACH

In morphological approach feature extraction methods can be distinguished into three types: (1) a Generic method is based on the analysis of edges, lines, and curves. (2) featuretemplate- based methods is based on the detection of the facial features such as eyes. (3) Structural matching methods that take into consideration geometrical constraints on the features. The technique we proposed here is independent of the aging factor, illumination and presence of accessories (glasses, beards, etc). Here in this technique we are considering the fiducial points. The points are the distance between eyes; eye and mouth. The distance between these facial points never changes. After drawing out the fiducial points we implement the Neural Network (NN) to the system for training and classification.

NEURAL NETWORK FOR TRAINING

The Back Propagation algorithm looks for the minimum of the error function in weight space using the method of gradient descent. Properly trained back propagation networks tend to give reasonable answers when presented with inputs that they have never seen. Typically, a new input leads to an output similar to the correct output for input vectors used in training that are similar to the new input being presented. This generalization property makes it possible to train a network on a representative set of input pairs and get good results without training the network on all possible input or output pairs. The RBF network performs similar function mapping with the BP, however its structure and function are much different. An RBF is a local network that is trained in a supervised manner contrasts with the BP network that is a global network. A BP performs a global mapping, meaning all inputs cause an output, while an RBF performs a local mapping, meaning only inputs near a receptive field produce activation. The LVQ network has two layers: a layer of input neurons, and a layer of output neurons. The weights of the connections to this neuron are then adapted, i.e. made closer if it correctly classifies the data point or made less similar if it incorrectly classifies it.

APPEARANCE BASED APPROACH

These approaches utilize the pixel intensity or intensityderived features. However, these methods may not perform well in many real-world situations, where the test face appearance is significantly different from the training face data, due to variations in pose, lighting and expression.

Usually a face image of size p × q pixels is represented by a vector in p.q dimensional space. In practice, however, these (p.q) -dimensional spaces are too large to allow robust and fast object recognition. A common way to attempt to resolve this problem is to use dimension reduction techniques. Two of the techniques for this purpose are Principal Component Analysis (PCA) and Regularized Linear Discriminant Analysis (R-LDA). In these approaches, the twodimensional face image is considered as a vector, by concatenating each row or column of the image. Each classifier has its own representation of basis vectors of a high dimensional face vector space. The dimension is reduced by projecting the face vector to the basis vectors, and is used as the feature representation of each face images.

3D-BASED CLASS

This subsection explores several methodologies that work directly on 3D datasets. The first problem concerning 3D face recognition is to set up a correct alignment between two face surfaces. One possible approach to gain a correct alignment is by using an acquisition system based on a morphable model, because it is pre-aligned within a given reference frame. The work presented by Ansari and Abdel- Mottaleb (2003) could be considered as an example of this kind of methods. Starting from one frontal and one profile view image, they use 3D coordinates of a set of facial feature points to deform a morphable model fitting the real facial surface. The deformation of the model is performed in two steps. At first a global deformation is carried out to scale and to align the morphable model to the feature points extracted from the pair images. Then a local deformation is applied to bring the vertices as close as possible to feature points. The recognition task is then performed calculating the Euclidean distance between 29 features points lying on 3D facial surface on mouth, nose and eyes. Their experimental results show a recognition rate of 96.2% on a database of 26 subjects with two pairs of images, one used for training and the other for testing.

The Iterative Closest Point (ICP) algorithm (Besl and McKay, 1992) is often used as an alternative approach aligning models. It could be used to reduce misalignment during the registration phase as well as to approximate the volume difference between two surfaces. Even though, it leads to problems with convergence when the initial misalignment of the data sets is too large, typically over; it is possible countered to this limitation with a coarse prealignment. An approach based on Iterative Closest Point algorithm is given by Cook et al. (2004).

They use ICP only to establish the correspondence between 3D surfaces in order to compensate problems due to nonrigid nature of faces. Then, once the registration is done, faces are compared by using a statistical model, namely Gaussian Mixture Model (GMM), and the distribution of the errors is then parameterized. They performed experiments on the 3D RMA database (Beumier and Acheroy, 2000) reaching a recognition rate of 97.33%. A quite similar ICPbased approach to find a point-to-point correspondence between landmarks features is given by Irfanoglu et al. (2004). They described a method to obtain a dense point-topoint matching by means of a mesh containing points that are present in all faces, so that the face alignment is trivially obtained. Then, once the dense correspondence is established, the Point Set Distance (PSD), that is a discrete approximation of the volume between facial surfaces, is used to compute the distance between two different clouds of points. In the experiments, they tested the algorithm on the 3D RMA database with a resulting recognition rate of 96.66%. Even if the ICP is a powerful tool in order to estimate the similarity between two faces, it has a serious lack. Indeed, ICP-based methods treat the 3D shape of the face as a rigid object so they are not able to handle changes in expression.

Medioni and Waupotitsch (2003) proposed an ICP-based approach that aligns two face surfaces and calculates a map of differences between the facial surfaces, then applying statistic measures in order to obtain a compact description of this map. They built 3D models of 100 subjects by using a stereo system; each subject has been acquired in 7 different poses within degrees with respect to the frontal view. The recognition rate on this dataset was 98%. As said before, a different use of the ICP algorithm is to approximate the surface difference between two faces. Indeed, the work of Lu et al. (2004) is headed in this direction. They describe both a procedure for constructing a database of 3D mesh models from several 2.5D images and a recognition method based on ICP algorithm. In order to build up the 3D meshes, features points are automatically detected on 2.5D images, searching for maximum and minimum local curvatures, so that ICP is run on these points aligning all the 2.5D images. Then, the recognition match between faces is carried out exploiting the local feature information correlated by the ICP. For the experiments, they report a recognition rate of 96.5% using a database of 113 range images for 18 subjects with different poses, facial expressions and illuminations. A further interesting aspect dealing with 3D face recognition concerns the analysis of the 3D facial surface in order to extrapolate information about the shape. Some approaches are based on a curvature-based segmentation detecting a set of fiducial regions.

Gordon (1991) presented a new method based on the idea that some facial descriptors, such as the shape of forehead, jaw line, eye corner cavities and cheeks, remain generally similar although they are taken by different range images for the same subject. This is not completely true when detection errors or changes in expression occur. His method consists in two different tasks: the former extracts a set of high level shape descriptors, for eyes, nose and head; the latter uses these descriptors to compute a set of basic scalar features corresponding to distance measurements. At last, each face images is projected in the feature space, while the Euclidean distance between feature vectors is used as a metric. The experiments of this method shows a recognition rate of 100% using a small training set of 8 subjects with three different view for each for a total of 24 faces.

Another interesting segmentation approach based on Gaussian curvature has been proposed by Moreno et al. (2003). For each 3D facial model, they detect a set of 86 different segmented regions by using an algorithm exploiting the signs of the median and Gaussian curvatures in order to isolate regions with significant curvatures (see Fig. 8). Then, this feature space is reduced in order to increase the efficiency. Finally, a feature vector is created for each subject. Experiments have been conducted on a dataset of 420 3D facial models belonging to 60 subjects, including images with light, rotation and facial expression variations, achieving a recognition rate of 78% for the best match and 92% for the five best matches. In addition, the segmentation process can be used to treat face recognition problem as a non-rigid object recognition problem to improve the robustness to facial expression variations.

Chua et al. (2000) observed that on there are regions on facial surfaces, such as nose, eye socket and forehead which undergo to much less deformation in case of expression changes (see Fig. 9). They find these ‘‘rigid’’ facial regions by using a Point Signature two-by-two comparison (Chua and Jarvis, 1997) among different facial expressions of the same person. Then, they store only the rigid parts in an indexed library, ranking models according to their similarity.

Their experiment shows a recognition rate of 100% on a dataset of 6 subjects and 4 facial expression variations. To model facial shape is also possible by creating a mathematical framework representing local/global curvatures. Another kind of approach to the analysis of facial shape is to create a mathematical model representative of local curvatures. This is a good way to account the 3D surface in a compact fashion using few features descriptors to characterize a face, without a wasteful complexity time. In addition, a local curvature-based representation better cope the non-rigid nature of face due to facial expressions because, though expressions changes the facial surface globally and the local curvature relations are preserved. Unluckily, this kind of representation is not able to handle information about the size of face, doing not possible to distinguish two similar faces but with different sizes.

Tanaka et al. (1998) proposed an example of these approaches performing a correlation-based face recognition based on analysis of minimum and maximum principal curvature and their directions, to describe the facial surface shape. Then, these descriptors are mapped on two unit spheres, the Extended Gaussian Images (EGI). The similarity match is performed by using the Fisher’s spherical approximation on the EGIs of faces. The method worked on 37 range images gathered by National Research Council of Canada (NRCC) (Rioux and Cournoyer, 1988), providing a recognition rate of 100%. On the contrary, Wang et al. (2004) presented a viewpoint-invariant technique based on a free-form representation, called Sphere-Spin-Images (SSI).

The SSIs are used to describe locally the shape of the facial surface. The SSIs of a point are constructed by mapping the 3D point coordinates lying into a sphere space, centered in that point, into a 2D space. The main aim of this mapping is to represent the local shape of points by means of a histogram. To describe a face, the method selects a small set of fixed points by means of a minimum principal curvature analysis and builds a single SSI series for each subject. Then, a simple correlation coefficient is used to compare the similarity between different SSI series. They performed tests on the SAMPL dataset (Range Imagery), with 31 models of 6 different subjects, reporting a recognition rate of 91.68%. Then, a simple correlation coefficient is used to compare the similarity between different SSI series. They performed tests on the SAMPL dataset (Range Imagery), with 31 models of 6 different subjects, reporting a recognition rate of 91.68%. the Principal Component Analysis (PCA) has been mentioned as a technique largely used in 2D face recognition in order to classify face images, reducing the dimensionality of image input space. In 3D face recognition is applied treating data as a cloud of points rather than a surface and new axes that best summarize the variance across the vertices are determined. Thus, the PCA is able to work with different facial poses producing a descriptive model of the facial shape.

This approach has been extended to the 3D face recognition by Hesher et al. (2002). The method apply the PCA directly to the range images, while they use the Euclidean distance to measure similarities among the resulting feature vectors. The authors state this method reached a recognition rate of 100% on a dataset of 222 range images of 37 subjects with different facial expressions. Further investigations on PCA in the 3D framework have been carried out by Heseltine et al. They presented two different works based on PCA theory, showing experimental results with several facial surface representations given by different convolution kernels and several distance metrics such as the Euclidean and cosine. The first method (Heseltine et al., 2004a) is based on a PCA-based eigen surface approach and is gathered on a data set of 330 three dimensional mesh models available by The University of York (The 3D Face Database, 2003). It reaches a recognition rate of 87.3%. The second approach (Heseltine et al., 2004b) is an adaptation of traditional 2D Belhumeur’s fisherface approach (Belhumeur et al., 1997) to 3D facial surface data (see Fig. 10). The results are gathered on a data set of 1770 three-dimensional mesh models with 280 subjects with several poses and facial expressions. The highest recognition rate it reaches is 88.7% when the surface gradients representation and the cosine distance metrics are used.

3D SHAPE-BASED FACE

3D shape-based face recognition algorithms can be broadly classified into the following categories: (1) point cloudbased, (2) depth map-based, (3) profile-based, (4) point signature-based, and (5) curvature-based algorithms. The most prominent method in current 3D face recognition systems is to use 3D point clouds to represent faces. In point cloud-based approaches, raw 3D point sets are used to register faces, and then features are extracted from registered faces. Often, the similarity between two facial point sets is determined by the quality of the alignment produced by the iterative closest point (ICP) [P. Besl et.al.(1992)] algorithm, [G. Medioni et.al.(2003), X. Lu et.al.(2004), S. Malassiotis et.al.(2004), T. Papatheodorou et.al.(2004), C. Xu et.al.(2004)].However, the ICP algorithm can only handle rigid transformations. In Lu and Jain(2005) extend their ICP-based algorithm such that thin plate spline (TPS) warping algorithm is used to establish registration in non-rigidly deformed locations. A similar approach was previously used in [M.O. IÃâ¹Ãâ¢rfanoglu et.al.(2004)] where TPS warping is used to establish dense correspondence. A completely different idea was proposed in [A.M. Bronstein et.al. (2003)] where the distances between 3D facial points are approximated by geodesic distances. In their work, authors apply multidimensional scaling algorithm to the geodesic distance matrix to obtain a canonical face representation. In their later work [A.M. Bronstein et.al.(2004), C. Hesher et.al.(2003)], they have extended their approach using surface gradients field. Their experimental result confirms that canonical form matching is robust to expression variations and outperforms 2D image-based eigenfaces [C. Hesher et.al. (2003)].

depth map-based ,Another popular approach to represent 3D faces is to project the 3D depth data to a 2D image according to the z-depth of the 3D points. In [C. Hesher et.al.(2003)], principal component analysis (PCA) and independent component analysis (ICA) were applied to the 2D depth images. Lee et al. [2003] use the means and the variances of the depth values of the local windows around the central nose region to represent faces from 2D depth images. In Srivastava et al. find the optimal linear subspace via the simulated annealing approach, and extract features in that space. Their results show that optimal linear subspace method outperforms PCA, LDA, and ICA-based feature extraction methods. Both 2D texture and depth images are used and combined in various works [K.I. Chang et.al.(2005), F. Tsalakanidou et.al.(2003)].

Silhouette-based representations are also applied to the 3D face recognition problem. In [C. Beumier et.al.(2000)], central and lateral profiles derived from 3D facial surfaces are used for recognition. Matching of the profiles of is carried out using iterative conditional mode (ICM) optimization. Curvature values computed along the profile curves are used as features. In [C. Beumier et.al.(2001)], authors extend their system where gray level color information is fused with shape features.

Point signatures are popular 3D descriptors for face recognition. In [C.-S. Chua et.al.(2000)], point signatures are used for both coarse registration and for rigid facial region detection which provide expression invariance. In their later work [Y. Wang et.al.(2002)], authors include texture into their systems by using 2D Gabor wavelets. Another 3D shape descriptor similar to the point signatures was used for face recognition in [Z. Wu et.al.(2004)] where authors proposed local shape maps to extract 2D histograms from 3D feature points. Their approach does not require registration, and the similarity between two faces is calculated by a voting algorithm as in [C.-S. Chua et.al.(2000)].

Surface curvatures play an important role in representing 3D faces [G. Gordon (1992)]. In [H.T. Tanaka et.al.(1998)], maximum and minimum principal directions are represented by two enhanced Gaussian images (EGIs) and similarity between faces is computed by Fisher’s spherical correlation method. Moreno et al. [2003] segment a facial surface into seven regions using curvature and extract several features such as region areas, area relations, and curvature means. Similar to curvature descriptors, surface normals are also used to represent 3D shape of faces in [S. Tsutsumi et.al.(1998)]. Combination of different shape features is also found to be beneficial in 3D face recognition systems. Go¨kberk et al. [2005] showed that when point cloud-based, surface-normal-based and depth image based shape representations are fused at the decision level in a rankbased manner, significant performance improvement is possible. Similarly, a surface-based recognizer and a profilebased recognizer are combined at the decision level in [G. Pan et.al.(2003), G. Pan et.al.(2005)].

Our aim in this paper is to study the state-of-the-art techniques frequently used in the purely 3D shape-based face recognition systems. Comparative analysis starts by dividing the 3D face recognition task into two consecutive subtasks, and for each subtask we implement various approaches. The first subtask, namely the registration of facial surfaces is carried out using two most commonly used approaches. The first registration method is based on a nonlinear warping of facial surfaces using TPS, and the second approach is based on a rigid transformation using the ICP technique. For warping-based registration, we propose a novel 3D facial feature localization algorithm since TPS warping needs several correspondent landmarks. The realization of the ICP-based registration algorithm is also novel and faster than the ones already present in the literature. Our implementation uses an average face model to define dense correspondence. This significantly reduces the computational effort. The second subtask in face recognition is the extraction of 3D facial features. We provide a comparative analysis of the most commonly used features such as point clouds, facial profiles, surface curvature-based features, 2D depth image-based approaches, and surface normals. We note that surface normal features are not so popular in the 3D face recognition community. However, we show that the use of surface normal features can significantly outperform other approaches. The second contribution of the paper is the systematic analysis of several fusion algorithms, which operate solely on the 3D shape features. Up to now, simple decision-level fusion algorithms were employed to fuse shape and texture information. Only a small number of studies emphasized the decision-level fusion of 3D shape-based face classifiers [B. Go¨kberk et.al.(2005), G. Pan et.al.(2003), G. Pan et.al.(2005)].

Distinctive outward facial appearance is a combination of colour and shape. Both these aspects of the face can be used together in so-called multi-modal approaches [A.S. Mian et.al.(2007), K.I. Chang et.al.(2003), M. Husken et.al.(2005), F. Tsalakanidou et.al.(2003)].In this review, however, we focus on the shape component only, which is defined as all the geometrical information that remains when colour, pose (rotation and location) and size (scale) effects are filtered out.

Morphometrics is the study of variation and change in the form (size and shape) of organisms or objects [M. Webster(2006)]. Morphometrics has a quantitative element that enables numerical comparisons between different shapes to be made. Quantitative and numerical data are extracted from the shape, reducing it to a series of numbers (typically concatenated into a so-called vector description), and facilitating objective (vector) comparisons between different objects.

Applied to faces these objective comparisons generate a numerical score of similarity, which is needed for recognition. In an authentication setup, the similarity scores between two given faces must be high enough to claim identity. For identification scenarios, the similarity scores between a probe face of unknown identity and a set of faces in a gallery with known identity are generated. The gallery face with the highest similarity to the probe face is used to establish identity. In a classification context the similarity score of a face with faces from a certain population must be high enough to belong to that population.

There are several different approaches in the literature for extracting and comparing data from facial shapes, each with their own strengths and weaknesses. However, whatever approach is used, three issues always exist that have to be taken into account. (1) The type of facial representation used from which the data is extracted. (2) The way pose or facial orientation differences between different faces are dealt with, which is easier in 3D than in 2D but still an important challenge. (3) Is whether or not the extracted data is embedded into any form of statistical shape analysis.

3D SHAPE ALONE

Cartoux et al. [1989] approach 3D face recognition by segmenting a range image based on principal curvature and finding a plane of bilateral symmetry through the face. This plane is used to normalize for pose. They consider methods of matching the profile from the plane of symmetry and of matching the face surface, and report 100% recognition for either in a small dataset.

Lee and Milios [1990] segment convex regions in a range image based on the sign of the mean and Gaussian curvatures, and create an extended Gaussian image (EGI) for each convex region.Amatch between a region in a probe image and in a gallery image is done by correlating EGIs. The EGI describes the shape of an object by the distribution of surface normal over the object surface. A graph matching algorithm incorporating relational constraints is used to establish an overall match of probe image to gallery image. Convex regions are asserted to change shape less than other regions in response to changes in facial expression. This gives some ability to cope with changes in facial expression. However, EGIs are not sensitive to change in object size, and so two similar shape but different size faces will not be distinguishable in this representation.

Gordon [1992] begins with a curvature-based segmentation of the face. Then a set of features are extracted that describe both curvature and metric size properties of the face. Thus each face becomes a point in feature space, and nearestneighbor matching is done. Experiments are reported with a test set of three views of each of eight faces and recognition rates as high as 100% are reported. It is noted that the values of the features used are generally similar for different images of the same face, ‘‘except for the cases with large feature detection error, or variation due to expression’’ [Gordon [1992].

Nagamine et al. [1992] approach 3D face recognition by finding five feature points, using those feature points to standardize face pose, and then matching various curves or profiles through the face data. Experiments are performed for 16 subjects, with 10 images per subject. The best recognition rates are found using vertical profile curves that pass through the central portion of the face. Computational requirements were apparently regarded as severe at the time this work was performed, as the authors note that ‘‘using the whole facial data may not be feasible considering the large computation and hardware capacity needed’’ [Gordon (1992)].

Achermann et al. extend eigenface and hidden Markov model (HMM) approaches used for 2D face recognition to work with range images. They present results for a dataset of 24 persons, with 10 images per person, and report 100% recognition using an adaptation of the 2D face recognition algorithms.

Tanaka et al. [1998] also perform curvature-based segmentation and represent the face using an extended Gaussian image (EGI).Recognition is performedusingaspherical correlation of the EGIs. Experiments are reported with a set of 37 images from a National Research Council of Canada range image dataset [1988], and 100% recognition is reported.

Chua et al. [2000] use ‘‘point signatures’’ in 3D face recognition. To deal with facial expression change, only the approximately rigid portion of the face from just below the nose up through the forehead is used in matching. Point signatures are used to locate reference points that are used to standardize the pose. Experiments are done with multiple images with different expressions from six subjects, and 100% recognition is reported.

Achermann and Bunke [2000] report on a method of 3D face recognition that uses an extension of Hausdorff distance matching. They report on experiments using 240 range images, 10 images of each of 24 persons, and achieve 100% recognition for some instances of the algorithm. Hesher et al. [2003] explore principal component analysis (PCA) style approaches using different numbers of eigenvectors and image sizes. The image data set used has six different facial expressions for each of 37 subjects. The performance figures reported result from using multiple images per subject in the gallery. This effectively gives the probe image more chances to make a correct match, and is known to raise the recognition rate relative to having a single sample per subject in the gallery [2003].

Medioni and Waupotitsch [34] perform 3D face recognition using an iterative closest point (ICP) approach to match face surfaces. Whereas most of the works covered here use 3D shapes acquired through a structured-light sensor, this work uses 3D shapes acquired by a passive stereo sensor. Experiments with seven images each from a set of 100 subjects are reported, with the seven images sampling different poses. An EER of ‘‘better than 2%’’ is reported.

Moreno and co-workers [2003] approach 3D face recognition by first performing a segmentation based on Gaussian curvature and then creating a feature vector based on the segmented regions. They report results on a dataset of 420 face meshes representing 60 different persons, with some sampling of different expressions and poses for each person. Rank-one recognition of 78% is achieved on the subset of frontal views.

Lee et al. [2003] perform 3D face recognition by locating the nose tip, and then forming a feature vector based on contours along the face at a sequence of depth values. They report 94% correct recognition at rank five, but do not report rank-one recognition. The recognition rate can change dramatically between ranks one and five, and so it is not possible to project how this approach would perform at rank one.

Pan et al. [2003] experiments with 3D face recognition using both a Hausdorff distance approach and a PCA-based approach. In experiments with images from the M2VTS database [1999] they report an equal-error rate (EER) in the range of 3–5% for the Hausdorff distance approach and an EER in the range of 5–7% for the PCA-based approach.

Lee and Shim [2004] consider approaches to using a ‘‘depth-weighted Hausdorff distance’’ and surface curvature information (the minimum, maximum, and Gaussian curvature) for 3D face recognition. They present results of experiments with a data set representing 42 persons, with two images for each person. A rank-one recognition rate as high as 98% is reported for the best combination method investigated, whereas the plain Hausdorff distance achieved less than 90%.

Lu et al. [2004] report on results of an ICP-based approach to 3D face recognition. This approach assumes that the gallery 3D image is a more complete face model and the probe 3D image is a frontal view that is likely a subset of the gallery image. In experiments with images from 18 persons, with multiple probe images per person, incorporating some variation in pose and expression, a recognition rate of 97% was achieved.

Russ et al. [2004] present results of Hausdorff matching on range images. They use portions of the dataset used in [2003] in their experiments. In a verification experiment, 200 persons were enrolled in the gallery, and the same 200 persons plus another 68 imposters were represented in the probe set. A probability of correct verification as high as 98% (of the 200) was achieved at a false alarm rate of 0 (of the 68). In a recognition experiment, 30 persons were enrolled in the gallery and the same 30 persons imaged at a later time were represented in the probe set. A 50% probability of recognition was achieved at a false alarm rate of 0.

The recognition experiment uses a subset of the available data ‘‘because of the computational cost of the current algorithm’’ [2004]. Xu et al. [2004] developed a method for 3D face recognition and evaluated it using the database from Beumier and Acheroy [2001]. The original 3D point cloud is converted to a regular mesh. The nose region is found and used as an anchor to find other local regions. A feature vector is computed from the data in the local regions of mouth, nose, left eye, and right eye. Feature space dimensionality is reduced using principal components analysis, and matching is based on minimum distance using both global and local shape components. Experimental results are reported for the full 120 persons in the dataset and for a subset of 30 persons, with performance of 72 and 96%, respectively. This illustrates the general point that reported experimental performance can be highly dependent on the dataset size. Most other works have not considered performance variation with dataset size. It should be mentioned that the reported performance was obtained with five images of a person used for enrollment in the gallery. Performance would generally be expected to be lower with only one image used to enroll a person.

Bronstein et al. [2005] present an approach to 3D face recognition intended to allow for deformation related to facial expression. The idea is to convert the 3D face data to an ‘‘eigenform’’ that is invariant to the type of shape deformation that is modeled. In effect, there is an assumption that ‘‘the change of the geodesic distances due to facial expressions is insignificant.’’ Experimental evaluation is done using a dataset containing 220 images of 30 persons (27 real persons and 3 mannequins), and 100% recognition is reported. A total of 65 enrollment images were used for the 30 subjects, so that a subject is represented by more than one image. As already mentioned, use of more than one enrollment image per person will generally increase recognition rates. The method is compared to a 2D eigenface approach on the same subjects, but the face space is trained using just 35 images and has just 23 dimensions.

The method is also compared to a rigid surface matching approach. Perhaps the most unusual aspect of this work is the claim that the approach ‘‘can distinguish between identical twins.’’ Go¨kberk et al. [2005] compare five approaches to 3D face recognition using a subset of the data used by Beumier and Acheroy [2001]. They compare methods based on extended Gaussian images, ICP matching, range profile, PCA, and linear discriminant analysis (LDA). Their experimental dataset has 571 images from 106 people. They find that the ICP and LDA approaches offer the best performance, although performance is relatively similar among all approaches but PCA. They also explore methods of fusing the results of the five approaches and are able to achieve 99% rank-one recognition with a combination of recognizers.

This work is relatively novel in comparing the performance of different 3D face recognition algorithms, and in documenting a performance increase by combining results of multiple algorithms. Additional work exploring these sorts of issues would seem to be valuable. Lee et al. [2005] propose an approach to 3D face recognition based on the curvature values at eight feature points on the face. Using a support vector machine for classification, they report a rank-one recognition rate of 96% for a data set representing 100 persons. They use a Cyberware sensor to acquire the enrollment images and a Genex sensor to acquire the probe images. The recognition results are called ‘‘simulation’’ results, apparently because the feature points are manually located.

Lu and Jain [2005] extend previous work using an ICPbased recognition approach [X.Lu et.al.(2004)] to deal explicitly with variation variation in facial expression. The problem is approached as a rigid transformation of probe to gallery, done with ICP,along with a non-rigid deformation, done using thin-plate spline (TPS) techniques. The approach is evaluated using a 100-person dataset, with neutralexpression and smiling probes, matched to neutralexpression gallery images.

The gallery entries are whole-head data structures, whereas the probes are frontal views. Most errors after the rigid transformation result from smiling probes, and these errors are reduced substantially after the non-rigid deformation stage. For the total 196 probes (98 neutral and 98 smiling), performance reaches 89% for shape-based matching and 91% for multi-modal 3D + 2D matching [2005].

Russ et al. [2005] developed an approach to using Hausdorff distance matching on the range image representation of the 3D face data. An iterative registration procedure similar to that in ICP is used to adjust the alignment of probe data to gallery data. Various means of reducing space and time complexity of the matching process are explored.

Experimental results are presented on a part of the FRGC version 1 data set, using one probe per person rather than all available probes. Performance as high as 98.5% rank-one recognition, or 93.5% verification at a false accept rate of 0.1%, is achieved. In related work, Koudelka et al. [2005] have developed a Hausdorff-based approach to prescreening a large dataset to select the most likely matches for more careful consideration [2005].

Pan et al. [2005] apply PCA, or eigenface, matching to a novel mapping of the 3D data to a range, or depth, image. Finding the nose tip to use as a center point, and an axis of symmetry to use for alignment, the face data are mapped to a circular range image. Experimental results are reported using the FRGC version 1 data set. The facial region used in the mapping contains approximately 12,500–110,000 points. Performance is reported as 95% rank-one recognition or 2.8% EER in a verification scenario. It is not clear whether the reported performance includes the approximately 1% of the images for which the mapping process fails.

Chang et al. [2005] describe an ‘‘multi-region’’ approach to 3D face recognition. It is a type of classifier ensemble approach in which multiple overlapping subregions around the nose are independently matched using ICP, and the results of the multiple 3D matches fused. The experimental evaluation in this work uses essentially the FRGC version 2 data set, representing over 4000 images from over 400 persons. In an experiment in which one neutral-expression image is enrolled as the gallery for each person, and all subsequent images (of varied facial expressions) are used as probes, performance of 92% rank-one recognition is reported.

Passalis et al. [2005] describe an approach to 3D face recognition that uses annotated deformable models. An average 3D face is computed on a statistical basis from a training set. Landmark points on the 3D face are selected based on descriptions by Farkas [1994]. Experimental results are presented using the FRGC version 2 data set. For an identification experiment in which one image per person is enrolled in the gallery (466 total) and all later images (3541) are used as probes, performance reaches nearly 90% rankone recognition.

3D FACE MODELING -STATISTICAL DEFORMATION MODEL

We present an automatic and efficient method to fit a statistical deformation model of the human face to 3D scan data. In a global to local fitting scheme, the shape parameters of this model are optimized such that the produced instance of the model accurately fits the 3D scan data of the input face. To increase the expressiveness of the model and to produce a tighter fit of the model, our method fits a set of predefined face components and blends these components afterwards. Quantitative evaluation shows an improvement of the fitting results when multiple components are used instead of one. Compared to existing methods, our fully automatic method achieves a higher accuracy of the fitting results. The accurately generated face instances are manifold meshes without noise and holes, and can be effectively used for 3D face recognition: We achieve 100% correct identification for 876 queries in the UND face set, 98% for 244 queries in the GAVAB face set, and 98% for 700 queries in the BU-3DFE face set. Our results show that model coefficient based face matching outperforms contour curve and landmark based face matching, and is more time efficient than contour curve matching.

The use of 3D scan data for face recognition purposes has become a popular research area. With high recognition rates reported for several large sets of 3D face scans, the 3D shape information of the face proved to be a useful contribution to person identification. The major advantage of 3D scan data over 2D color data, is that variations in scaling and illumination have less influence on the appearance of the acquired face data. However, scan data suffers from noise and missing data due to self-occlusion. To deal with these problems, 3D face recognition methods should be invariant to noise and missing data, or the noise has to be removed and the holes interpolated.

Alternatively, data could be captured from multiple sides, but this requires complex data acquisition. In this chapter we present a method that produces an accurate fit of a statistical 3D shape model of the face to the scan data. We show that the 3D geometry of the generated face instances, which are without noise and holes, can be effectively used for 3D face recognition.

Previous techniques are based on 3D geodesic surface information, such as the methods of Bronstein et al. [2005] and Berretti et al. [2007]. The geodesic distance between two points on a surface is the length of the shortest path between two points. To compute accurate 3D geodesic distances for face recognition purposes, a 3D face without noise and without holes is desired. Since this is typically not the case with laser range scans, the noise has to be removed and the holes in the 3D surface interpolated. However, the success of basic noise removal techniques, such as Laplacian smoothing is very much dependent on the resolution and density of the scan data. Straightforward techniques to interpolate holes using curvature information or flat triangles often fail in case of complex holes, as pointed out in , J. Davis,et.al.(2002). The use of a deformation model to approximate new scan data and interpolate missing data is a gentle way to regulate flaws in scan data.

A well known statistical deformation model specifically designed for surface meshes of 3D faces, is the 3D morphable face model of Blanz and Vetter [1999]. This statistical model was built from 3D face scans with dense correspondences to which Principal Component Analysis (PCA) was applied. In their early work, Blanz and Vetter [1999] fit this 3D morphable face model to 2D color images and cylindrical depth images from the CyberwareTM scanner. In each iteration of their fitting procedure, the model parameters are adjusted to obtain a new 3D face instance, which is projected to 2D cylindrical image space allowing the comparison of its color values (or depth values) to the input image. The parameters are optimized using a stochastic Newton algorithm. More recently, Blanz et al. [2007] proposed a method to fit their 3D morphable face model to more common textured depth images. In the fitting process, a cost function is minimized using both color and depth values after the projection of the 3D model to 2D image space. To initialize their fitting method, they manually select seven corresponding face features on their model and in the depth scan. A morphable model of expressions was proposed by Lu et al. [2008].

Amberg et al. [2008] built a PCA model from 270 identity vectors and a PCA model from 135 expression vectors and combined the two into a single morphable face model. They fitted this model to 3D scans of both the UND and GAVAB face sets, and use the acquired model coefficients for expression invariant face matching with considerable success.

Non-statistical deformation models were proposed as well. Huang et al. [2006] proposed a global to local deformation framework to deform a shape with an arbitrary dimension (2D, 3D or higher) to a new shape of the same class. They show their framework’s applicability to 3D faces, for which they deform an incomplete source face to a target face. Kakadiaris et al. [2006] deform an annotated face model to scan data. Their deformation is driven by triangles of the scan data attracting the vertices of the model. The deformation is restrained by a stiffness, mass and damping matrix, which control the resistance, velocity and acceleration of the model’s vertices. The advantage of such deformable faces is that they are not limited to the statistical changes of the example shapes, so the deformation has less restriction. However, this is also their disadvantage, because these models cannot rely on statistics in case of noise and missing data.

The scans that we fit the morphable face model to, are the 3D face scans of the UND, a subset of the GAVAB [A. Moreno et.al.(2004), F. B. ter Haar,et.al.(2008)] and a subset of the BU-3DFE L. ,Yin, [2006] databases. The UND set contains 953 frontal range scans of 277 different subjects with mostly neutral expression. The GAVAB set consists of nine low quality scans for each of its 61 subject, including scans for different poses and expressions. From this set we selected, per subject, four neutral scans, namely the two frontal scans and the scans in which subjects look up and down. Acquired scan data from these poses differ in point cloud density, completeness and relatively small facial changes. The BU-3DFE set was developed for facial expression classification. This set contains one neutral scan and 24 expression scans having different intensity levels, for each of its 100 subjects. From this set we selected the neutral scans and the low level expression scans. Although the currently used morphable model is based on faces with neutral expressions only, it makes sense to investigate the performance of our face model fitting in case of changes in pose and expressions. These variations in 3D scan data, which are typical for a non-cooperative scan environment, allows us to evaluate our 3D face recognition methods.

Before pose normalization, we applied a few basic preprocessing steps to the scan data: the 2D depth images were converted to triangle meshes by connecting the adjacent depth samples with triangles, slender triangles and singularities were removed, and only considerably large connected components were retained. Afterwards, the face is segmented by removing the scan data with a Euclidean distance larger than 100 mm from the nose tip.

In general, 3D range scans suffer from noise, outliers, and missing data and their resolution may vary. The problem with single face scans, the GAVAB scans in particular, is that large areas of the face are missing, which are hard to fill using simple hole filling techniques. When the morphable face model is fitted to a 3D face scan, a model is obtained that has no holes, has a proper topology, and has an assured resolution.

ITERATIVE FACE FITTING

With the defined distance measure for an instance of our compressed morphable face model, the m-dimensional space can be searched for the optimal instance. The fitting is done by choosing a set of m weights , measuring the RMSdistance of the new instance to the scan data, selecting new weights and continue until the optimal instance is found. Knowing that each instance is evaluated using a large number of vertices, an exhaustive search for the optimal set of m weights is too computationally expensive. A common method to solve large combinatorial optimization problems is simulated annealing (SA) [S. Kirkpatrick,et.al.(1983)]. In our case, random m-dimensional vectors could be generated which represent different morphs for a current face instance. A morph that brings the current instance closer to the scan data is accepted (downhill), and otherwise it is either accepted (uphill to avoid local minima) or rejected with a certain probability. The fitting process starts with the mean face and morphs in place towards the scan data, which means that the scan data should be well aligned to the mean face. To do so, the segmented and pose normalized face is placed with its center of mass on the center of mass of the mean face, and finely aligned using the Iterative Closest Point (ICP) algorithm [P. J. Besl et.al.(1992)]. The ICP algorithm iteratively minimizes the RMS distance between vertices. To further improve the effectiveness of the fitting process, our approach is applied in a coarse fitting and a fine fitting step.

COARSE FITTING

The mean face is coarsely fitted to the scan data by adjusting the weights of the first ten principal eigenvectors .Fitting the model by optimizing the first ten eigenvectors results in the face instance , with global face properties similar to those of the scan data. After that, the alignment of the scan to be further improved with the ICP algorithm.

FINE FITTING

Starting with the improved alignment, we again fit the model to the scan data.

MULTIPLE COMPONENTS

Knowing that the morphable model was generated from 100 3D face scans, an increase of its expressiveness is most likely necessary to cover a large population. To increase the expressiveness, also Blanz and Vetter [1999] proposed to independently fit different components of the face, namely the eyes, nose, mouth, and the surrounding region. Because each component is defined by its own linear combination of shape parameters, a larger variety of faces can be generated with the same model. The fine fitting scheme from the previous section was developed to be applicable to either the morphable face model as a whole, but also to individual components of this model.

FACE MATCHING

Our model fitting algorithm determines a set of model coefficients that morphs the mean face to a clean model instance that resembles the 3D face scan. Based on this model instance, we use three different methods to perform face matching. Two methods use the newly created 3D geometry as input, namely the landmarks based and contour based methods. The third method uses the model coefficients as a feature vector to describe the generated face instance.

LANDMARKS

All vertices of two different instances of the morphable model are assumed to have a one-to-one correspondence. Assuming that facial landmarks such as the tip of the nose, corners of the eyes, etc. are morphed towards the correct position in the scan data, we can use them to match two 3D faces.

CONTOUR CURVES

Another approach is to fit the model to scans A and B and use the new clean geometry as input for a more complex 3D face recognition method. To perform 3D face recognition, we extract from each fitted face instance three 3D facial contour curves, and match only these curves to find similar faces.

MODEL COEFFICIENTS

The iterative model fitting process determines an optimal weight for each of the m eigenvectors. These weights, or model coefficients, multiplied by describe a path along the linearly independent eigenvectors through the m dimensional face space. For two similar scans one can assume these paths are alike, which means that the set of m model coefficients can be used as a feature vector for face matching. In case of multiple components, each component has its own set of m model coefficients. In [V. Blanz, et.al.(2007)], sets of model coefficients where simply concatenated to a single coefficient vector. Here, we also concatenate the coefficient vectors of multiple components. To determine the similarity of faces with these coefficient vectors, we use four distance measures. In [B. Amberg et.al.(2008)], the authors assume that caricatures of an identity lie on a vector from the origin to any identity and use the angle between two coefficient vectors as a distance measure.

ANALYSIS

In this section we evaluate our fitting results for the UND, GAVAB, and BU-3DFE datasets. We perform a qualitative and quantitative evaluation of the acquired model fits and compare the results with other model fitting methods. To prove that the use of multiple components improves the fitting accuracy over a single component, we compare the quantitative measures and relate the fitting accuracy to face recognition by applying different face matching algorithms to the produced fits. By applying and comparing different face matching methods, we end up with a complete 3D face recognition system with high recognition rates for all three datasets.

Starting with the 3D face scans from a dataset, we apply our face segmentation method. Our face segmentation method correctly normalized the pose of all face scans and adequately extracted the tip of the nose in each of them. For the 953 scans of the UND face set, we evaluated the tip of the nose extraction by computing the average distance and standard deviation of the 953 automatically selected nose tips to our manually selected nose tips, which was 2.4 ± 1.3 mm. Since our model fitting method aligns the face scan to the mean face and at a later stage to the coarsely fitted face instance, these results are good enough.

COMPARISON

Blanz et al. [18] reported a mean depth error over 300 UND scans of 1.02 mm when they neglected outliers. For our fitted single component to UND scans the error davr.depth is 0.65 mm, which is already more accurate. For the fitted multiple components these errors are 0.47 and 0.43, for four and seven components respectively. Our time to process a raw scan requires ca. 3 seconds for the face segmentation, ca. 1 second for the coarse fitting, and ca. 30 seconds for the fine fitting on a Pentium IV 2.8 GHz. Blanz method reported ca. 4 minutes on a 3.4 GHz Xeon processor, but includes texture fitting as well. Huang et al. [49] report for their deformation model a matching error of 1.2 mm after a processing time of 4.6 minutes. Recently, Amberg et al. [2008] proposed a competitive fitting time of 40 to 90 seconds for their face model with 310 model coefficients and 11.000 vertices.

FACE MATCHING

we can use the morphed face instances to perform 3D face recognition. For this experiment, we computed the 953 × 953, 244 × 244, and 700 × 700 dissimilarity matrices and sorted the ranked lists of face models on decreasing similarity. From these ranked lists, we computed the recognition rate (RR), the mean average precision (MAP) and the verification rate at 0.1% false acceptance rate (VR@0.1%FAR). A person is recognized (or identified) when the face retrieved on top of the ranked list (excluding the query) belongs to the same subject as the query. For 77 subjects in the UND set only a single face instance is available which cannot be identified, so for this set the RR is based on the remaining 876 queries. The mean average precision (MAP) of the ranked lists are also reported, to elaborate on the retrieval of all relevant faces, i.e. all faces from the same subject. Instead of focusing on 3D face retrieval application, one could use 3D face matching for imposter detection as well. For an imposter/client detection system, all face matches with a dissimilarity above a carefully selected threshold are rejected. Lowering this threshold means that more imposters are successfully rejected, but also that less clients are accepted. We use the dissimilarity threshold at which the false acceptance rate is 0.1%, which is also used in the face recognition vendor test. Because the VR@0.1%FAR depends on similarity values, it is not only important to have relevant faces on top of the ranked lists, but also that their similarity values are alike and differ from irrelevant faces.

Since, the VR@0.1%FAR evaluation measure depends on the acquired similarity values there are several ways to influence this measure. Rank aggregation with the use of Consensus Voting or Borda Count [T. Faltemier et.al.(2008)], for instance, reassigns similarity values based on the ranking. This way one can abstract from the actual similarity values, which allows for the selection of a different imposter threshold and change the VR@0.1%FAR. Of course, a rank based threshold can not be used in case of a one-to-one face matching, that is, a scenario in which someone’s identity must be confirmed or rejected. The application domain for rank-based measures is the one-tomany face matching, that is, a scenario in which we search for the most similar face in a large database.

In case of the face matching based on model coefficients, we assume that caricatures of an identity lie on a vector from the origin to any identity. If we normalize the lengths of these vectors, we neglect the caricatures and focus on the identity. This normalization step also regulates the similarity values and thus influences the VR@0.1%FAR. In Table 5.3, we report the face matching results based on the L1 and L2 distances between coefficient vectors, before and after length normalization. Remarkable is the significant increase of the VR@0.1%FAR for the normalized coefficient vectors, whereas the rankings are similar as shown by the MAPs. Although we show in Table 5.3 only the results for the face model fitting using seven components, it also holds for the one and four component case. Because the L1 distance between normalized coefficient vectors slightly outperforms the L2 distance measure, we use this measure whenever we evaluate the performance of model coefficients.

Face retrieval and verification results based on anthropometric landmarks, contour curves, and model coefficients. To each set of face scans we fitted the morphable face model using one, four, and seven components. Each fitted component produces a 99 dimensional model coefficient vector with a different face instance as a result. The performance of our face matching depends on both the number of components as well as the applied feature set. The two main observations are that (1) the coefficient based method outperforms the landmark based and contour based methods, and (2) that the use of multiple components can increase the performance of 3D face matching. In the next paragraphs we elaborate on these observations. The automatically selected anthropometric landmarks have a reasonable performance on the UND face scans, but are not reliable enough for effective 3D face matching in the two other sets. The contours perform well for retrieval and verification purposes in the UND face set. However, their performance drops significantly for the other two sets, because the contour curves cannot be effectively used in case of facial deformations. The use of the model coefficients consistently outperforms both the landmarks based and contour based face matching. Besides the difference in performance, the three methods differ in running time as well. The landmark based method matches two faces using only 15 coordinates, whereas the contour based method matches two faces using 135 coordinates. The coefficient based method matches faces using 99 weights times the number of fitted components. So, the coefficient based method using four components has the approximately the same running time as the contour based method. The observation that multiple (four or seven) components increases the performance of our face matching holds for all results except the landmark based and contour based methods in the GAVAB set. The problem with this set is that a low quality scans of a person looking up or down causes artifacts on and around the nose. In such cases a more accurate fit of the face model’s nose harms, because the performance of landmark and contour based methods are heavily dependent on an accurate selection of the nose tip. Although the face matching improves from the single to multiple component case, there is no consensus for the four or seven component case. The use of either four or seven components causes either a marginal increase or decrease of the evaluation scores. Although, face matching with the use of 1000 model coefficients is usually referred to as time efficient, one could argue to use four components instead of seven, because the number of coefficients is smaller.

Comparison. Blanz et al. [2007] achieved a 96% RR for 150 queries in a set of 150 faces (from the FRGC v.1). To determine the similarity of two face instances, they computed the scalar product of the 1000 obtained model coefficients. In the previous chapter, we achieved 95% RR on the UND set using the three selected contour curves, and 98% RR with an ICP-based method.

Where other methods need manual initialization, we presented a fully automatic 3D face morphing method that produces a fast and accurate fit for the morphable face model to 3D scan data. Based on a global to local fitting scheme the face model is coarsely fitted to the automatically segmented 3D face scan. After the coarse fitting, the face model is either finely fitted as a single component or as a set of individual components.

BOOTSTRAPPING ALGORITHM FOR 3D FACE MODEL

We present a new bootstrapping algorithm to automatically enhance a 3D morphable face model with new face data. Our algorithm is based on a Morphable model fitting method that uses a set of predefined face components. This fitting method produces accurate model fits to 3D face data with noise and holes. In the fitting process, the dense pointto- point correspondences between the scan data and the face model can become less reliable at the border of components. In this chapter, we solve this by introducing a blending technique that improves on the distorted correspondences close to the borders. Afterwards, a new face instance is acquired similar to the 3D scan data and in full correspondence with the face model. These newly generated face instances can then be added to the morphable face model to build a more descriptive one. To avoid our bootstrapping algorithm from needlessly adding redundant face data, we incorporate a redundancy estimation algorithm. We tested our bootstrapping algorithm on a set of scans acquired with different scanning devices, and on the UND data set. Quantitative and qualitative evaluation shows that our algorithm successfully enhances an initial morphable face model with new face data, in a fully automatic manner.The process of using a statistical model to enhance itself automatically, is referred to as bootstrapping the synthesis of the model [T. Vetter et.al.(1997)]. The difficulty of bootstrapping is that: (1) If the model (as is) fits a new example well, there is no use of adding the new example to the model. This must be automatically verified. (2) If the model doesn’t fit the new example, the correspondences are incorrect and the example cannot be added to the model. (3) It should be fully automatic. Nowadays, several statistical models are available, ready to be used and reused. In this chapter we present a bootstrapping algorithm based on an initial statistical model, which automatically fits to new scan data with noise and holes, and which is capable of measuring the redundancy of new example faces.

The importance for bootstrapping statistical models was posed by Vetter et al. [1997]. They introduced a bootstrapping algorithm for statistical models, and showed that the use of merely an optic flow algorithm was not enough to establish full correspondence between example faces and a reference face. Instead, they attain an effective bootstrapping algorithm by iteratively fitting the face model, applying the optic flow algorithm, and updating the face model. Blanz and Vetter also used this bootstrapping algorithm in [V. Blanz et.al.(1999)] to build a 3D morphable face model.

Their bootstrapping algorithm works well in case of input data with constant properties, but fails when input data is incomplete and when the optic flow algorithm fails. To bootstrap the 3D morphable face model with more general face data, Basso et al. [2006] added a smoothness term to regularize the positions of the vertices where the optic flow correspondence is unreliable. In case a 3D morphable face model is not yet available, a reference face can be used as an approximation instead, which is a major advantage. Amberg et al. [2007] proposed a non-rigid Iterative Closest Point (ICP) algorithm to establish dense correspondences between a reference face and face scans, but they need an initial rigid transformation for the reference face based on 14 manually selected landmarks. Afterwards, the reference face and the fitted face instances can be used to construct a new morphable face model.

Basso et al. [2007] fit the morphable face model to scan data using implicit representations. They also use multiple components and blend the implicit functions at the borders of components, but they loose the full point-to-point correspondence in the process. So the fitted examples cannot be added to the morphable model.

Huang et al. [2006] proposed a global to local deformation framework to deform a shape with an arbitrary dimension (2D, 3D or higher) to a new shape of the same class. Their method also operates in the space of implicit surfaces, but uses a non-statistical deformation model. They show their framework’s applicability to 3D faces, for which they deform an incomplete source face to a target face.

The use of multiple components has been used by Blanz et al. to improve the face model fitting [V. Blanz et.al.(2007)] and for face recognition purposes [V. Blanz et.al.(2003)], but so far the resulting face instances were not accurate enough to be incorporated in the statistic model. The explicit point-to-point correspondences of the fitted face instance and the statistical model had to be established by techniques based on optic flow or non-rigid ICP. In the previous chapter, a set of predefined face components was used to increase the descriptiveness of a 3D morphable face model. With the use of multiple components, a tighter fit of the face model was obtained and higher recognition rates were achieved. However, by fitting each component individually, components started to intersect, move apart, or move across. So, afterwards the full point-to-point correspondences between the morphable model and the fitted instance were distorted. The postprocessing method to blend the borders of the components introduces a new set of surface samples without correspondence to the model either.

Without the use of unreliable optic flow [C. Basso et.al.(2006)] or semi-automatic non-rigid ICP [B. Amberg et.al.(2007)], we are able to bootstrap the 3D morphable face model with highly accurate face instances. As a proof of concept, we (1) fit the initial morphable face model to several 3D face scans using multiple components, (2) blend the components at the borders such that accurate point-topoint correspondences with the model are established, (3) add the fitted face instances to the morphable model, and (4) fit the enhanced morphable model to the scan data as one single component. In the end, we compare each single component fit obtained with the enhanced morphable model to the single component fit obtained with the initial morphable model. Qualitative and quantitative evaluation shows that the new face instances have accurate point-topoint correspondences that can be added to the initial morphable face model. By comparing the multiple and single component fit, our bootstrapping algorithm automatically distinguishes between new face data to add and redundant data to reject. This is important to keep both the model fitting and the face recognition with model coefficients time-efficient.

MORPHABLE FACE MODEL

We fit the morphable face model to 3D scan data to acquire full correspondence between the scan and the model. we crop the morphable face model and lower its resolution so that n=12,964 vertices remain for the fitting. We fit the morphable face model to 3D scan data from the UND [K. I. Chang et.al.(2005)], GAVAB [A. Moreno et.al.(2004)], BU- 3DFE [L. Yin et.al.(2006)], Dutch CAESAR [CAESARsurvey( 2008)], and our local dataset. From all except the UND set, we randomly select four scans yielding a first test set of 18 scans. These scans vary in pose, facial expression, resolution, accuracy, and coverage. This set of 18 face scans is used to test our bootstrapping algorithm. To test the automatic redundancy check, we use a subset of 277 face scans from the UND dataset, namely the first scan of each new subject.

BOOTSTRAPPING ALGORITHM

The main problem in bootstrapping the 3D morphable face model, is that (1) we only want to add example faces that are not covered by the current model, (2) new example faces suffer from noise and missing data, which makes it hard to establish the point-topoint correspondences, and (3) it should be fully automatic. To establish point-to-point correspondences between the 3D morphable face model and new face data with noise and missing data. This method either fits the model as a single component or as a set of predefined components. In case the model is fitted as a single component, the final model fit is in full point-to-point correspondence with the face model, but adds no additional information to the current face model. In case the model is fitted as a set of predefined components, this method produces model fits that go beyond the current statistics of the face model, but the point-to-point correspondence are inaccurate or lost. In this section, we briefly describe the used model fitting method, then we explain our algorithm to establish dense point-to-point correspondences between the multiple component fits and the morphable face model, and finally, we explain how the bootstrapping algorithm can distinguish between new face data to add to the model and redundant data to reject.

MODEL FITTING

This model fitting algorithm iteratively adjusts them coefficients for each component, such that the vertices move closer to the vertices of the scan data. After all components are fitted to the scan data individually, an accurate representation of the new face.

ANALYSIS

To elaborate on the performance of our bootstrapping algorithm, we applied it to the dataset of 17 different face scans and to the subset of 276 UND scans. The small set is used to evaluate the model fitting and correspondence estimation algorithm. The UND set is used to test the redundancy estimation.

FACIAL EXPRESSION BASED MODELING

This statistical model consists of face data with neutral expressions only, and with the bootstrapping algorithm we presented in the previous chapter, we could extend the model with small expression deformations only. Adding these expression deformations to the same morphable face model causes the statistical spaces of identities and expressions to interfere, which means that the model coefficients are no longer reliable for face identification. As a result, we should be able to fit the model to a neutral and an expression scan of the same person, but we cannot use the coefficients to identify one with theother.

This chapter presents a new automatic and efficient method to fit a statistical expression model of the human face to 3D scan data. To achieve expression invariant face matching we incorporated expression-specific deformation models in the fitting method. In a global to local fitting scheme, the identity and expression coefficients ofthis model are adjusted such that the produced instance of the model accurately fits the 3D scan data of the input face. Quantitative evaluation shows that the expression deformations as well as a set of predefined face components improve on the fitting results. 3D face matching experiments on the publicly available UND, GAVAB, BU-3DFE, FRGC v.2 datasets show high recognition rates of respectively 99%, 98%, 100%, and 97% with the use of the identity coefficients. Results show that not only the coefficients that belong to the globally optimized model fit perform well, but that the coefficients of four locally optimized model fits can produce similar recognition rates. Finding the optimal model fit is hard and loosening this requirement could make a system more robust.

Statistical models of the human face have proven to be an effective tool for person identification of 3D face scans. To build a statistical model, a set of example faces is required with face features in full correspondence. With such a model, a new face instance can be constructed as a linear combination of the example faces. For 3D face identification, the idea is to use the statistical model to construct a face instance that resembles an input image. The way these example faces are combined linearly to represent an input face provides both global and local information about the input face that can be used to classify and identify different input faces. Expressions are a problem, because they change the resemblance of the input faces.

Most of the early 3D face recognition methods focused on variants of the Iterative Closest Point (ICP) [P. J. Besl et.al. (1992)] algorithm to find similarity between two 3D face scans. As 3D face recognition became more challenging with larger datasets and expression scans, the ICP-based methods showed two main disadvantages. The non-rigid expression deformations forced the ICP-based methods to rely on smaller face regions such as the nose and forehead, and the computational expensive face matching lowered its practical use. Methods of Faltemier et al. [2008] and Mian et al. [2007] reported high recognition rates based on nose regions in combinations with ICP. For efficient face matching, the extraction of person specific features became the new area of interest.

With high recognition rates, low computational costs during face matching, and high robustness to noise and missing data, 3D morphable face model based methods prove to perform well. To build a 3D morphable face model, dense correspondences are required among a set of 3D example faces. The mean face and the statistical variation of these faces can be computed using Principal Component Analysis (PCA). Using the statistical face variations, the mean face can be deformed to fit the noisy scan data. The way such a model is deformed (larger, wider, longer nose, etc.), provides information on the geometric shape properties of the input face. The coefficients that induce these deformations form a relatively small feature vector for efficient face matching. For reliable model coefficients, the model deformation must be independent of changes in the face pose. Therefore, the model fitting is often combined with an ICP algorithm to compensate for the rigid transformation between closest point features. Because both the model fitting and the ICP algorithm are local optimization methods, a coarse alignment between the scan data and the model should be automatically established first.

Blanz and Vetter use a 3D morphable face model to model 3D faces out of 2D images [V. Blanz Et.al.(1999)]. In [2007], Blanz et al. fit a morphable model to 3D scan data and use the deformation weights (or model coefficients) to recognize faces with neutral expressions. In each iteration of their stochastic Newton algorithm, the current model instance is projected to 2D image space and the model coefficients are adjusted according to the difference in texture and depth values. For the coarse alignment and to initiate the morphable model, they manually select seven corresponding face features on their model and in the depth scan.

Amberg et al. [2008] build a PCA model from 270 identity vectors and a PCA model from 135 expression vectors and combined the two into a single morphable face model. Their method fits this model to 3D scan data by iteratively finding closest point pairs to improve on the alignment, the identity deformation, and the expression deformation at the same time. Their local optimization method, which does not guarantee convergence to the global minimum, returns a set of identity coefficients that perform well in terms of face recognition. For the initial alignment of the scan to the model, they use our automatic face pose normalization method.

Lu and Jain [2008] train a morphable expression model for each expression in their test set. Starting from an existing neutral scan, they fit each of their expression models separately to adjust the vertices in a small region around the nose to lower the ICP error between that particular neutral scan and an expression scan. The expression model that produces the most accurate fit is used to deform the neutral scan. For the initial alignment they use three automatically detected feature points. For the fitting, they combine the accurate ICP alignment for the rigid transformation with the fast eigenspace projection [M. Turk et.al.(1991)] for the expression deformation. This process is iterated until convergence and the lowest residual error is used as the dissimilarity score between the neutral scan and the new scan. Although the authors use PCA models, their method can be classified as an ICP based method, because the fitting procedure has to be repeated for every pair of face scans in the dataset. The expression models are merely used to improve on the ICP fitting procedure.

Mpiperis et al. [2008] build a bilinear PCA model for the BU-3DFE dataset suitable for both expression and identity recognition after a face scan is brought into full correspondence with the model. To establish full correspondence, they detect the boundary of the mouth, elastically deform a low resolution face mesh to the scan data (considering the mouth), and subdivide the mesh for denser correspondences. The bilinear PCA model is solely used to map the full correspondence to expression and identity coefficients that are either used for expression classification or person identification.

Kakadiaris et al. [2006] deform an annotated subdivision face model to scan data. Their non-statistical deformation is driven by triangles of the scan data attracting the vertices of the model. The deformation is restrained by a stiffness, mass and damping matrix, which control the resistance, velocity and acceleration of the model’s vertices. They use the newly created geometry for wavelet analysis and achieve state of the art recognition results on the Face Recognition Grand Challenge (FRGC) [P. J. Phillips et.al.(2006)].

Firstly, we introduce seven morphable expression models, for the “expressions” anger, disgust, fear, happiness, sadness, surprise, and inflated cheeks. Secondly, we use a new morphable identity model to perform expression invariant 3D face identification, in combination with the expression model. Starting with a dataset of neutral scans, expression scans, and a small set of annotated landmarks, we describe how to build a strong multi-resolution morphable model for both identity and expression variations of the human face. A new feature in this modeling method is the decoupling of the pose normalization and deformation modeling, so that the model fitting becomes highly timeefficient.

Thirdly, we introduce a model fitting method that combines eigenspace sampling, eigenspace projection, and predefined face components. This method is able to produce accurate fits for the morphable identity model in combination with the best expression model to new expression scans. The statistics captured in the Morphable face model allows for the robust handling of noise and holes. Afterwards, the final expression instance and its model coefficients can be used as the complete and noiseless representation of the expression scan, to automatically extract the facial landmarks, to bootstrap the face model, to remove the expression, and for expression invariant face recognition. Fourthly, for 3D face recognition with model coefficients, we propose a new multiple minima approach and compare the results with the global minimum approach. Local minima in facespace are easier to find and their locations provide valuable information for face identification.

Results show that (1) our method can be applied with considerable success to a large range of datasets with high recognition rates of 99%, 98%, 100%, and 97%, for the UND, GAVAB, BU-3DFE, FRGC v.2 datasets, (2) the use of expression models is essential for a high performance, (3) the use of multiple components (MC) improves on the single component (SC) results, (4) in case of scan data with lower quality, as in the GAVABdataset, the multiple minima (MM) approach can improve the system’s performance. (5) the time-efficiency of our complete 3D face recognition system allows for its application in face authentication and face retrieval scenarios.

MORPHABLE FACE MODEL

We use a morphable face model built from both 3D neutral and expression scans of the human face. We fit this model to 3D scan data in such a way that expressions can be removed and subjects identified in an expression invariant manner. To build a morphable face model with expressions, an example set of subjects showing various expressions is required. For that, we use the BU-3DFE [databases] dataset, from which we select the 100 neutral scans and 600 expression scans at their highest intensity level. The BU- 3DFE set was developed for facial expression classification. This set contains one neutral scan and 24 expression scans having different intensity levels, for each of its 100 subjects. From this set we selected the neutral scans and the highest intensity level expression scans .The goal is to model a neutral face model from a dense set of correspondences, and a neutral to expression model for each of the expressions anger, disgust, fear, happiness, sadness and surprise. The neutral face model, which is built from the 100 neutral scans, captures the identity of different subjects, whereas a neutral-to-expression model captures the facial changes caused by a certain expression.

A morphable face model is a type of statistical point distribution model (PDM) [T. Cootes et.al.(2001)], where the points are facial features that have a different distribution among different faces. Building a morphable face model, requires n dense correspondences S = (x1, y1, z1. . . xn, yn, zn)T 3n among a set of input face scans. Principal Component Analysis (PCA) is used to capture the statistical distribution of these correspondences among the input faces. Because the automatic estimation of reliable dense correspondences among noisy face scans with expressions is still unsolved, we propose a semiautomatic correspondence estimation that requires 26 facial landmarks. With the use of these 26 landmarks, we construct a low resolution mesh that is projected to the cylindrical depth image of a 3D face scan. By subdividing the triangles of the low resolution mesh, a multi-resolution representation of the face is constructed. At each level, we assume that the vertices between different subjects or expressions correspond. The correspondences at the highest level are used to build a neutral 3D morphable face model as well as a morphable expression model for each of the expressions. Because the manual annotation of facial landmarks in 3D face scans is often a major disadvantage in statistical modeling.This automatic bootstrapping is a useful tool to limit the user input. The flow chart of our semi-automatic modeling approach. Our semi-automatic model building consists of the following steps:

A. Manual annotation of facial landmarks, including nose, eyes, eyebrows, and mouth.

B. Cylindrical depth image construction.

C. Multi-resolution face mesh construction.

D. Building the morphable identity model.

E. Building the morphable expression models.

F. Automatic bootstrapping the morphable model.

G. Data reduction.

H. Component selection.

LANDMARK ANNOTATION

In each of the 700 pose normalized (raw) BU-3DFE scans, we manually selected the same sequence of 26 facial landmarks as an initial set of correspondences. These landmarks include locations on the nose, mouth, eyes, and eyebrows, and provide a coarse notion of facial changes among different identities and expressions. This is the only user input throughout this chapter. In fact, most of these landmarks were already annotated in the BU-3DFE set and the nose tip was detected automatically.

CYLINDRICAL DEPTH IMAGE

Knowing that almost all face scans (even with facial hair and expressions) can be correctly pose normalized after the final alignment to an average nose template, it makes sense to build the morphable face model based on face scans in the coordinate system of this nose template. Each BU-3DFE scan was brought into alignment with the reference nose template, which has the desired pose and its nose tip in the origin. Although the nose template was accurately fitted to the face scans, this doesn’t mean that the nose tip of the face scan is aligned to the nose tip in the template. A smaller nose, for instance, has its tip behind the template and a larger nose in front of the template (higher z-value). To produce a cylindrical depth image for each of the face scans, we simulate a cylindrical laser range scanner. To cover most of the face, the nose template and the aligned face scans are moved 80 mm along the positive z-axis. A surface sample is acquired for each angle _ at each height y with radius distance d to the y-axis of the coordinate system. Basically, we cast a horizontal ray at height y with an angle _ in the xz-plane from the y-axis to the face scan, and store the distance to the The pose normalized faces are annotated with landmarks (first row) that correspond among different expressions and different subjects to construct an initial face mesh as a layer over the cylindrical depth image (second row). Asubdivision scheme is applied to acquire dense 3D face meshes (third row).

MORPHABLE IDENTITY MODEL

Building an identity based face model requires a training set of neutral faces of different subjects in full correspondence. Turk and Pentland [1991] also described how to compute the “eigenfaces” that define this “face space”, for 2D intensity images.

MORPHABLE EXPRESSION MODEL

Building an expression model requires full correspondence between all the neutral faces and the sets of expression faces. Matrix A is now initiated with the difference between the expression face and neutral face of subject. In the work of Lu and Jain [2008], experiments with an expression-generic and expression-specific models show that the latter outperforms the former.

AUTOMATIC BOOTSTRAPPING

For face recognition purposes it is important to have an identity model Sid that describes a large human population. The face space allows for the interpolation between example faces and the extrapolation outside its statistical boundary, but only to some extend.

MORPHABLE MODEL FITTING

The task of the model fitting algorithm is to find the face instance Sexpr.inst in the high dimensional face space that produces the best point-to-point correspondence with a new face scan. Additionally, the model fitting algorithm should be robust to noise and perform well even when large areas of the face are missing.

EXPRESSION FITTING

The main difficulty in model fitting is that neither the expression coefficients nor the identity can be optimized without optimizing the other. When the model is fitted to a new scan with an unknown expression, it makes sense to coarsely estimate the identity based on expression invariant regions and then to select the best expression model and search for its optimal expression parameters.

IDENTITY FITTING

After the expression fitting, we have obtained a coarse identity vector that, in combination with the final expression vector, produces a relatively good fit to the scan data. For the purpose of face recognition, each subject needs a unique expression invariant identity vector _. Amberg et al. [5] proposed to produce the best possible fit and to use the decoupled identity vector for face recognition. In the previous chapter, a more accurate fit was produced by fitting predefined face components individually. Here, we use both methods and propose a new descriptor. This results in a higher fitting accuracy [I. Mpiperis et.al.(2008)]. With the use of a kD-tree the closest point correspondences can be found efficiently. For high efficiency, we compute in algorithm ESSamp only the correspondences from model to scan, because the model and its kD-tree change in each iteration. For the eigenspace sampling we consider two closest points pairs to correspond if the distance between them is smaller than 50 mm, for the eigenspace projection we use correspondences closer than 10 mm. We stop traversing a kD-tree, when this criterion can no longer be met.

FACE MATCHING

After the morphable model is fitted to each of the face scans, we have obtained three feature vectors of model coefficients, namely, the single-component vector, the multicomponent vector, and the multi-minima vector. For the face matching we use each of these vectors individually to do 3D face recognition. To determine the similarity of faces with these coefficient vectors, we use the L1 distance between the normalized coefficient vectors.

ANALYSIS

We can evaluate the fits qualitatively by looking at the more frequent surface interpenetration of the fitted model and face scan (Fig. 7.9), which means a tighter fit. Note that fitting method is robust to missing data and even creates accurate face instances when half of the face is missing. Comparison UND. Several authors report recognition rates for theUNDdataset. Blanz et al. [2007] achieved a 96% RR for 150 queries in a set of 150 faces. Samir et al. [2006] reported 90.4% RR for 270 queries in a set of 470 faces. Mian et al. [2005] reported 86.4% RR for 277 queries in a set of 277 scans. Amberg et al. [2008] used all 953 scans and achieved 100% RR. Comparison GAVAB. The GAVAB dataset has been used in the Shape Retrieval Contest 2008 [R. C. Veltkamp et.al.(2008)] to compare 3D face retrieval methods. Results of different approaches vary between 60% and 100% RR. Recently, Amberg et al. [2008] achieved a recognition rate of 99.7% on this dataset. They use a morphable head model that covers the neck and ears as well, features that may aid the person identification.

Comparison BU-3DFE. Mpiperis et al. [2008] performed experiments on the BU-3DFE dataset. They used two methods for the expression invariant face matching, a symmetric bilinear model and geodesic polar coordinates, with respectively 86% and 84% RR. The authors report that their recognition results outperform Bronstein et al.’s [2005] canonical image representation.

Comparison FRGC v.2. Lu et al. [2008] applied their expression-specific deformation models to only 100 subjects of the FRGC v.2 and report 92% recognition rate and 0.7 VR@0.1%FAR, which is considerably lower than the results with our expression-specific deformation models. Moreover, we do not need a neutral face scan for the deformation nor the computational expensive ICP algorithm for the matching. Other 3D shape based methods that report the VR@0.1%FAR for the all-to-all face matching experiment are, Maurer et al. [2005] with 0.78 VR, Mian et al. [2007] with 0.87 VR, Cook et al. [2006] with 0.92 VR, and Faltemier et al. [2008] with 0.93 VR. Most of them use the computational expensive ICP algorithm during face matching and simply neglect data in regions with expressions. Kakadiaris et al. [2007] reported a 97% RR and 0.97 VR@0.1%FAR for slightly different experiments.

The model fitting method that we presented, coarsely fits the identity model in combination with each of the expression models and keeps the overall best fit. Because separate models are used for the identity and expression deformations, the expression can be easily neutralized and the separate identity coefficients can be used for expression invariant face matching. Three identity coefficient vectors were acquired for the face matching, one based on the face as a single component, one for multiple face components, and one for multiple local minima. Compared to the literature, all our coefficient vectors proved to perform very well on the publicly available datasets. The use of multiple face components is an easy way to improve on the face matching performance, and in case of low quality scans the multiple minima vector can be a good alternative. Therefore, our method can be very well applied to authentication scenarios as well as face retrieval scenarios.

2D + 3D-BASED CLASS

Multimodal approaches combine information coming from 2D image as well as 3D model of faces. Recently Chang et al. (2003) investigated on possible improvements that 2D face biometric can receive integrating the 3D also. The method, they proposed, performs separately the PCA on the intensity and range images and then combines results obtained from both strategies to get a global response from the system. The authors assert four important conclusion: ‘‘(1) 2D and 3D have similar recognition performance when considered individually, (2) Combining 2D and 3D results using a simple weighting scheme outperforms either 2D or 3D alone, (3) Combining results from two or more 2D images using a similar weighting scheme also outperforms a single 2D image, and (4) Combined 2D + 3D outperforms the multi-image 2D result’’ (Chang et al., 2004). Experiments have been conducted on a dataset of 275 subjects by using a single and a multiprobe set. The recognition rate is 89.5% for the intensity images and 92.8% for the range images, while the combined solution provides a global rate of 98.8% (see Fig. 11).

Bronstein et al. (2003) presented a new method based on a bending invariant canonical representation (Fig. 12), they called canonical image that models deformations resulting from facial expression and pose variations. They observe that facial expressions are not arbitrary, but they can be modelled by using isometric transformations. The canonical image stores these geometric invariants and it is built by calculating the geodesic distances between points on facial surface. The 2D face image is mapped onto the canonical image shape flattening the texture coordinates onto the canonical surface. The experimental results are performed on a database of 157 subjects but nothing has been said about recognition rates.

On the contrary, Tsalakanidou et al. (2003) proposed an HMM approach to integrate depth data and intensity image. The method start localizing the face with a depth and brightness based procedure, while the recognition task exploits the embedded hidden Markov model techniq that is applied to 3D range images as well as 2D images. The experimental results are gathered on a very large database of 3000 range and greyscale images of 50 subjects, with various facial expressions, poses, illuminations and with/ without glasses, reporting a recognition rate of 90.7% on 2D intensity images and 80% on 3D range images, while the system reaches a rate of 91.67%, when both information are combined. Papatheodorou and Rueckert (2004) proposed a 4D registration method based on Iterative Closest Point (ICP), but adding textural information too. The data acquisition is done with a stereo camera system composed by three cameras and a pattern projector, while the measurement of facial similarity involves a 4D Euclidean distance (represented by colors1 as shown in Fig. 13) between four-dimensional points: the three spatial coordinates more the texel intensity information. They report various results on a dataset collected from 62 subjects with 13 distinct 3D meshes and 2D texture maps considering several facial expression and poses. The percentage of correct matches and correct rejection are used as performance measures. In case of frontal pose, results show that the use of both texture and shape improves performances, while a percentage of correct recognition ranging from 66.5% to 100%, depending on several poses and expressions. All 3D based methods introduced so far are summarized in Table 4 in addition to a small set of parameters, that can be considered meaningful for a more complete and accurate evaluation of discussed approaches. In general, the recognition rate is a widely used measure for the evaluation of face recognition methods, but it strongly depends on the number of people in the database and the number of images per subject gathered for the experimental results. In addition the key features (illumination (i), expression (e), pose (p), occlusions (o)) considered on the models in the probe and gallery set are reported, in order to take into account for the testingframework, in which 3D methods have been tested.

CONCLUSION

Here we give the complete survey of 3D based face modeling methods. We conclude that three-dimensional face modeling needs better algorithms. Here, better means more tolerant of real-world variety factors. At the same time, better Also means less computationally demanding. Three dimensional face modeling in general seems to require much more computational effort per match than does 2D face modeling. Here 3D morphable face modeling is better than the other methods.

References

AIM@SHAPE. Shape Repository athttp://www.aimatshape.net/resources, Nov 2006.
L. Akarun, B. Gökberk, and A. Salah. 3D FaceRecognition for Biometric Applications. In EUSIPCO,2005.
F. R. Al-Osaimi, M. Bennamoun, and A. Mian.Integration of Local and Global Geometrical Cues for3D Face Recognition. Pattern Recognition, 41(3):1030–1040,2008.
P. Alliez, D. Cohen-Steiner, Y. Tong, and M. Desbrun.Voronoi-Based Variational Reconstruction ofUnoriented Point Sets. In Symposium on Geometryprocessing, pages 39–48, 2007.
B. Amberg, R. Knothe, and T. Vetter. ExpressionInvariant Face Recognition with a Morphable Model. InAutomatic Face and Gesture Recognition, 2008.
B. Amberg, S. Romdhani, and T. Vetter. Optimal StepNonrigid ICP Algorithms for Surface Registration. InCVPR, pages 1–8, 2007.
N. Amenta and Y.-J. Kil. Defining Point-set Surfaces.In Trans. on Graphics, volume 23, pages 264–270, 2004.
B. B. Amor, K. Ouji, M. Ardabilian, and L. Chen. 3DFace Recognition by ICPbased Shape Matching. In
ICMI, 2005.
G. Barequet and M. Sharir. Filling Gaps in theBoundary of a Polyhedron. Computer Aided GeometricDesign, 12(2):207–229, 1995.
C. Basso, P. Paysan, and T. Vetter. Registration ofExpressions Data using a 3DMorphable Model. InAutomatic Face and Gesture Recognition, pages 205–210, 2006.
C. Basso and A. Verri. Fitting 3D Morphable ModelsUsing Implicit Representations. Journal of VirtualReality and Broadcasting, 4(18), 2007.
R. Bergevin, M. Soucy, H. Gagnon, and D. Laurendeau.Towards a General Multiview Registration Technique.18(5):540–547, 1996.
F. Bernardini, J. Mittleman, H. Rushmeier, C. Silva, andG. Taubin. The Ball- Pivoting Algorithm for Surface
Reconstruction. Trans. on Visualization and ComputerGraphics, 5(4):349–359, 1999.
F. Bernardini and H. Rushmeier. The 3D ModelAcquisition Pipeline. Computer Graphics Forum,21(2):149–172, 2002.
S. Berretti, A. Del Bimbo, P. Pala, and F. Silva Mata.Face Recognition by Matching 2D and 3D GeodesicDistances. In MCAM07, pages 444–453, 2007.
P. J. Besl and N. D. McKay. A Method for Registrationof 3D Shapes. PAMI, 14(2):239–256, 1992.
Binghamton University 3D Facial Expression Database.BU-3DFE.
V. Blanz, K. Scherbaum, and H.-P. Seidel. Fitting aMorphable Model to 3D Scans of Faces. In ICCV,pages 1–8, 2007.
V. Blanz and T. Vetter. A Morphable Model for theSynthesis of 3D Faces. In SIGGRAPH, pages 187–194,1999.
V. Blanz and T. Vetter. Face Recognition Based onFitting a 3DMorphable Model. PAMI, 25(9):1063–1074, 2003.
K. W. Bowyer, K. Chang, and P. Flynn. A Survey ofApproaches and Challenges in 3D and Multi-modal 3D+ 2D Face Recognition. CVIU, 101(1):1–15, 2006.
M. Bronstein, M. M. Bronstein, and R. Kimmel.Three-Dimensional Face Recognition. IJCV, 64(1):5–30, 2005.
CAESAR-survey. The Civilian American and EuropeanSurface Anthropometry Resource athttp://store.sae.org/caesar, Dec 2008.
R. J. Campbell and P. J. Flynn. A Survey of Free-formObject Representation and Recognition Techniques.CVIU, 81(2):166–210, 2001.
J. C. Carr, R. K. Beatson, J. B. Cherrie, T. J. Mitchell,W. R. Fright, B. C. McCallum, and T. R. Evans.Reconstruction and Representation of 3D Objects withRadial Basis Functions. In SIGGRAPH, pages 67–76,2001.
K. I. Chang, K. W. Bowyer, and P. J. Flynn. AnEvaluation of Multimodal 2D+3D Face Biometrics.PAMI, 27(4):619–624, 2005.
K. I. Chang, K.W. Bowyer, and P. J. Flynn. MultipleNose Region Matching for 3D Face Recognition underVarying Facial Expression. PAMI, 28(10):1695–1700,2006.
C.-S. Chen, Y.-P. Hung, and J.-B. Cheng. RANSACbasedDARCES: a New Approach to Fast AutomaticRegistration of Partially Overlapping Range Images.PAMI, 21(11):1229–1234, 1999.
Y. Chen and G. Medioni. Object Modelling byRegistration of Multiple Range Images. Image andVision Computing, 10(3):145–155, 1992.
J. Cheng and H. Don. A Graph Matching Approach to3-D Point Correspondences. Int. Journal of PatternRecognition and Artificial Intelligence, 5(3):399–412,1991.
C. S. Chua and R. Jarvis. Point Signatures: A NewRepresentation for 3D Object Recognition. IJCV,25(1):63–85, 1997.
P. Cignoni, C. Rocchini, and R. Scopigno. Metro:Measuring Error on Simplified Surfaces. ComputerGraphics Forum, 17:167–174, 1998.
J. Cook, V. Chandran, and C. Fookes. 3D FaceRecognition using Log-Gabor Templates.In BMVC,pages 83–93, 2006.
J. Cook, V. Chandran, S. Sridharan, and C. Fookes.Face Recognition from 3D Data using Iterative ClosestPoint Algorithm and Gaussian Mixture Models. In3DPVT, pages 502–509, 2004.
T. Cootes, G. Edwards, and C. Taylor. ActiveAppearance Models. PAMI, 23(6):681–685, 2001.
S. Cunnington and A. J. Stoddart. N-view Point SetRegistration: a Comparison.In BMVC, pages 234–244,1999.
B. Curless and M. Levoy. A Volumetric Method forBuilding Complex Models from Range Images. InSIGGRAPH, volume 30, pages 303–312, 1996.
J. Davis and X. Chen. A Laser Range Scanner Designedfor Minimum Calibration Complexity. 3DIM, 00:91–99, 2001.
J. Davis, S. R. Marschner, M. Garr, and M. Levoy.Filling Holes in Complex Surfaces using VolumetricDiffusion. In 3DPVT, pages 428–861, 2002.
T. K. Dey and S. Goswami. Provable SurfaceReconstruction from Noisy Samples. In Symposium onComputational Geometry, pages 330–339, 2004.
D. W. Eggert, A. W. Fitzgibbon, and R. B. Fisher.Simultaneous Registration of Multiple Range Views foruse in Reverse Engineering of CAD Models. CVIU,69(3):253–272, 1998.
T. Faltemier, K. Bowyer, and P. Flynn. A RegionEnsemble for 3-D Face Recognition. Trans. OnInformation Forensics and Security, 1(3):62–73, 2008.
V. Fiorin, P. Cignoni, and R. Scopigno. Out-of-coreMLS Reconstruction. In Computer Graphics andImaging, pages 27–34, 2007.
P. Fitzpatrick. Euclid’s Elements of Geometry. RichardFitzpatrick, 2008.
M. Garland and P. S. Heckbert. Surface Simplificationusing Quadric Error Metrics. In SIGGRAPH, pages209–216, 1997.
B. Gökberk, M. O. Irfanoglu, and L. Akarun. 3D ShapebasedFace Representation and Feature Extraction forFace Recognition. Image and VisionComputing,24(8):857–869, 2006.
J. Gwilt. The Architecture of Marcus Vitruvius Pollio:In Ten Books. Lockwood, 1874. [48] K. Higuchi, M.Hebert, and K. Ikeuchi. Building 3-D Models fromUnregistered Range Images. CVGIP-GMIP, 57(4):315–333, 1995.
X. Huang, N. Paragios, and D. N. Metaxas. ShapeRegistration in Implicit Spaces Using InformationTheory and Free Form Deformations. PAMI,28(8):1303–1318,2006.
D. Huber and M. Hebert. 3-D Modeling Using aStatistical Sensor Model and Stochastic Search. InCVPR, pages 858–865, 2003.
Imhausen. Calculating the Daily Bread: Rations inTheory and Practice. HistoriaMathematica, 30(1):3–16,2003.
INUS Technology. RapidForm 2004 PP2 athttp://www.rapidform.com, Nov 2006.
ISTI-CNR Visual Computing Laboratory. MeshAlign,MeshLab, MeshMerge, OM and Metro at
http://vcg.isti.cnr.it, June 2009.
E. Johnson. Spin Images: A Representation for 3-DSurface Matching. PhD thesis, Carnegie MellonUniversity, Pittsburgh, 1997.
T. Ju. Robust Repair of Polygonal Models. Trans. OnGraphics, 23(3):888–895, 2004.
Kakadiaris, G. Passalis, G. Toderici, N. Murtuza, andT. Theoharis. 3D Face Recognition. In BMVC, pages869–878, 2006.
A. Kakadiaris, G. Passalis, G. Toderici, M. N.Murtuza, Y. Lu, N. Karampatziakis, and T. Theoharis.Three-Dimensional Face Recognition in the Presence ofFacial Expressions: An Annotated Deformable ModelApproach. PAMI, 29(4):640–649, 2007.
Y. J. Kil, N. Amenta, and B. Mederos. Laser ScannerSuper-resolution. In Point Based Graphics, 2006.
R. Kimmel and J. Sethian. Computing Geodesic Pathson Manifolds. In Proc. Of National Academy of Sciences, volume 95(15), pages 8431–8435, 1998.
S. Kirkpatrick, C. D. Gelatt, and M. P. Vecchi.Optimization by Simulated Annealing. Science,Number 4598, 13 May 1983, 220, 4598:671–680, 1983.
D. C. Lay. Linear Algebra and Its Applications.Addison-Wesley, 1997.
M. Levoy, K. Pulli, B. Curless, S. Rusinkiewicz, D.Koller, L. Pereira, M. Ginzton, S. Anderson, J. Davis, J.Ginsberg, J. Shade, and D. Fulk. The DigitalMichelangelo Project: 3D Scanning of Large Statues. InSIGGRAPH, pages 131–144, 2000.
C. Li, A. Barreto, J. Zhai, and C. Chin. Exploring FaceRecognition by Combining 3D Profiles and Contours.In SoutheastCon, pages 576–579, 2005.
W. E. Lorensen and H. Cline. Marching Cubes: a HighResolution 3D Surface Construction Algorithm. InSIGGRAPH, volume 21, pages 303–312, 1987.
X. Lu and A. Jain. Deformation Modeling for Robust3D Face Matching. PAMI, 30(8):1346–1356, 2008.
T. Maurer, D. Guigonis, I. Maslov, B. Pesenti, A.Tsaregorodtsev, D. West, and G. Medioni. Performanceof GeometrixActiveIDTM 3D Face RecognitionEngine on the FRGC Data. CVPR Workshop, 0:154–160, 2005.
S. Mian, M. Bennamoun, and R. Owens. MatchingTensors for Pose Invariant Automatic 3D FaceRecognition. In A3DISS, 2005.
S. Mian, M. Bennamoun, and R. Owens. Automatic3D Face Detection Normalization and Recognition. In3DPVT, pages 735–742, 2006.
S. Mian, M. Bennamoun, and R. Owens. AnEfficient Multimodal 2D-3D Hybrid Approach toAutomatic Face Recognition. PAMI, 29(11):1927–1943, 2007.
S. Mian, M. Bennamoun, and R. A. Owens. FromUnordered Range Images to 3D Models: A FullyAutomatic Multiview Correspondence Algorithm. InTheory and Practice of Computer Graphics, pages 162–166. Computer Society Press, 2004.
S. Mian, M. Bennamoun, and R. A. Owens. 3DModel-based Object Recognition and Segmentation inScenes. PAMI, 2006.
S. Mian, M. Bennamoun, and R. A. Owens. A NovelRepresentation and Feature Matching Algorithm for
Automatic Pairwise Registration of Range Images.IJCV, 66(1):19–40, 2006.
Moreno and A. Sanchez. GavabDB: a 3D FaceDatabase. In Workshop on Biometrics on the InternetCOST275, Vigo, pages 77–85, 2004.
Mpiperis, S. Malassiotis, and M. G. Strintzis. BilinearMod els for 3-D Face and Facial ExpressionRecognition. Trans. on Information Forensics andSecurity, 3(3):498–511, 2008.
P. Nair and A. Cavallaro. SHREC’08 Entry:Registration and Retrieval of 3D Faces using a PointDistribution Model. In Shape Modeling andApplications, pages 257– 258, 2008.
P. J. Neugebauer. Reconstruction of Real-WorldObjects via Simultaneous Registration and RobustCombination of Multiple Range Images. Int. Journal ofShape Modeling, 3(1-2):71–90, 1997.
J. Novatnack and K. Nishino. Scale-Dependent/Invariant Local 3D Shape Descriptors forFully Automatic Registration of Multiple Sets of RangeImages. In ECCV, pages 440–453, 2008.
U. Park, Y. Tong, and A. K. Jain. Face Recognitionwith Temporal Invariance: A 3D Aging Model. InAutomatic Face and Gesture Recognition, 2008.
P. J. Phillips, P. J. Flynn, T. Scruggs, K.W. Bowyer,andW.Worek. Preliminary Face Recognition GrandChallenge Results. In Automatic Face and GestureRecognition, pages 15–24, 2006.
F. Pighin, J. Hecker, D. Lischinski, R. Szeliski, and D.Salesin. Synthesizing Realistic Facial Expressions fromPhotographs. In SIGGRAPH, pages 75–84, 1998.
P. Pingi, A. Fasano, P. Cignoni, C. Montani, and R.Scopigno. Exploiting the Scanning Sequence forAutomatic Registration of Large Sets of Range Maps.Computer Graphics Forum, 24(3):517–526, 2005.
K. Pulli. Multiview Registration for Large Data Sets. In3DI M, pages 160–168, 1999.
H. T. F. Rhodes. Alphonse Bertillon: Father ofScientific Detection. Adelard-Schuman, 1956.
Rocchini, P. Cignoni, F. Ganovelli, C. Montani, P.Ping i, and R. Scopigno. The Marching
Intersections Algorithm for Merging Range Images.The Visual Computer, 20:149–164, 2004.
Rocchini, P. Cignoni, C. Montani, P. Pingi, and R.Scopigno. A Low Cost 3D Scanner Based on StructuredLight. In Proc. EG, volume 20(3), pages 299–308,2001.
S. Rusinkiewicz. Trimesh2 athttp://www.cs.princeton.edu/gfx/proj/ trimesh2/ , Nov2006.
S. Rusinkiewicz, O. Hall-Holt, and M. Levoy. Real-Time 3D Model Acquisition. Trans. on Graphics,21(3):438–446, 2002.
S. Rusinkiewicz and M. Levoy. Efficient Variants ofthe ICP Algorithm. In 3DIM, pages 145–152, 2001.
W. Saleem, O. Schall, G. Patane, A. Belyaev, and S. H.On Stochastic Methods for Surface Reconstruction. TheVisual Computer, 23(6):381–395, 2007.
Samir, A. Srivastava, and M. Daoudi. Three-Dimensional Face Recognition using Shapes of FacialCurves. PAMI, 28(11):1858–1863, 2006.
Samir, A. Srivastava, M. Daoudi, and E. Klassen. AnIntrinsic Framework for Analysis of Facial Surfaces.IJCV, 82(1):80–95, 2009.
Scheenstra, A. Ruifrok, and R. C. Veltkamp. ASurvey of 3D Face Recognition Methods. In AVBPA,pages 891–899, 2005.
Sharf, M. Alexa, and D. Cohen-Or. Context-BasedSurface Completion. Trans. on Graphics, 23(3):878–887, 2004.
M. Slater, A. Steed, and Y. Chrysanthou. ComputerGraphics and Virtual Environments: From Realism toReal-Time. Addison-Wesley Publishers, 2001.
Stanford Computer Graphics Laboratory. 3D ScanningRepository, Scanalyze, VripPack, and VolFill athttp://graphics.stanford.edu/software/, Nov 2006.
J. Stoddart and A. Hilton. Registration of MultiplePoint Sets. In Pattern Recognition, volume A, pages 40–44, 1996.
Stuttgart. Range Image Database athttp://range.informatik.unistuttgart. de/htdocs/html/ ,Nov 2006.
F. B. terHaar. Quadruple Alignment at http://givelab.cs.uu.nl/quadruple-alignment/ , Nov 2006.
F. B. terHaar, P. Cignoni, P. Min, and R. C. Veltkamp.A Comparison of Systems and Tools for 3D Scanning.In 3D Digital Imaging and Modeling: Applications ofHeritage, Industry, Medicine and Land, 2005.
F. B. terHaar, M. Daoudi, and R. C. Veltkamp. ShapeREtrieval Contest 2008: 3D Face Scans. In ShapeModeling and Applications (SMI), pages 225–226,2008.
F. B. terHaar and R. C. Veltkamp. Alignment,Merging and Hole Filling Experiments with 3D Range
Scans. Technical Report UU-CS-2005-047, UtrechtUniversity, 2005.
F. B. terHaar and R. C. Veltkamp. A 3D FaceMatching Framework. Technical Report UU-CS-2007-047, Utrecht University, 2007.
F. B. terHaar and R. C. Veltkamp. AutomaticMultiview Quadruple Alignment of Unordered Range
Scans. In Shape Modeling and Applications (SMI),pages 137–146, 2007.
F. B. terHaar and R. C. Veltkamp. 3D Face ModelFitting for Recognition. In European Conference onComputer Vision (ECCV), pages 652–664, 2008.
F. B. terHaar and R. C. Veltkamp. A 3D FaceMatching Framework. In Shape Modeling andApplications (SMI), pages 103–110, 2008.
F. B. terHaar and R. C. Veltkamp. A 3D FaceMatching Framework for Facial Curves. GraphicalModels, 71(2):77–91, 2009.
G. Turk and M. Levoy. Zippered Polygon Meshesfrom Range Images. In SIGGRAPH, pages 311–318,1994.
M. Turk and A. Pentland. Face Recognition usingEigenfaces. In CVPR, pages 586–591, 1991.University of South Florida, Sudeep Sarkar. USFHumanID 3D Face Database.
R. C. Veltkamp, R. Ruijsenaars, M. Spagnuolo, R. vanZwol, and F. B. terHaar. SHREC2006: 3D ShapeRetrieval Contest. Technical Report UU-CS-2006-030,Utrecht University, 2006.
R. C. Veltkamp and F. B. terHaar. SHREC2007: 3DShape Retrieval Contest. Technical Report UU-CS-2007-015, Utrecht University, 2007.
R. C. Veltkamp and F. B. terHaar. SHREC2008: 3DShape Retrieval Contest. In Shape Modeling andApplications, pages 215–263, 2008.
T. Vetter, M. J. Jones, and T. Poggio. A BootstrappingAlgorithm for Learning Linear Models of ObjectClasses. In CVPR, pages 40–46, 1997.
S.-F.Wang and S.-H. Lai. Estimating 3D Face Modeland Facial Deformation from a Single Image Based onExpression Manifold Optimization. In ECCV, pages589– 602, 2008.
A.Wehra and U. Lohrb. Airborne Laser Scanning – anIntroduction and Overview. Journal of Photogrammetryand Remote Sensing, 54(2-3):68–82, 1999.
T. Weise, B. Leibe, and L. V. Gool. Fast 3D Scanningwith Automatic Motion Compensation. In CVPR, pages1–8, 2007.
T. Whitmarsh, R. C. Veltkamp, M. Spagnuolo, S.Marini, and F. B. terHaar. Landmark Detection on 3DFace Scans by Facial Model Registration. In 1st Int.Workshop on Shape and Semantics, pages 71–76, 2006.
J. V. Wyngaerd, L. V. Gool, and M. Proesmans.Invariant-based Registration of Surface Patches. InICCV, pages 301–306, 1999.
C. Xu, T. Tan, Y. Wang, and L. Quan. CombiningLocal Features for Robust Nose Location in 3D FacialData. Pattern Recognition Letters, 27(13):1487–1494,2006.
L. Yin, X.Wei, Y. Sun, J.Wang, and M. J. Rosato. A3D Facial Expression Database For Facial BehaviorResearch. In Automatic Face and Gesture Recognition,pages 211–216, 2006.
L. Zhang, B. Curless, and S. M. Seitz. Rapid ShapeAcquisition Using Color Structured Light and MultipassDynamic Programming. In 3DPVT, pages 24–36,2002.
H. Zhao and G. Xu. Triangular Surface Mesh Fairingvia Gaussian Curvature Flow. Journal of Computationaland Applied Mathematics, 195(1):300–311, 2006.
Z. Zonghua, P. Xiang, S. Weiqiang, and H. Xiaotang.A Survey of Surface Reconstruction from MultipleRange Images. In Asia-Pacific Conf. on SystemsIntegrity and Maintenance, volume 8, pages 519–523,2000.
Bartlett, M. S., Movellan, J. R., &Sejnowski, T. J.(2002). Face recognition by independent componentanalysis. IEEE Transactions on Neural Networks, 13(6),1450-1464.
Baudat, G., &Anouar, F. (2000). Generalizeddiscriminant analysis using a kernel approach. NeuralComputation, 12(10), 2385-2404.
Belhumeur, P. N., Hespanha, J. P., &Kriengman, D. J.(1997). Eigenfaces vs. Fish[126]erfaces: Recognition using class specific linearprojection. IEEE Transactions on Pattern Analysis andMachine Intelligence, 19(7), 711-720.
Belkin, M., &Niyogi, P. (2002). Laplacianeigenmapsand spectral techniques for embedding and clustering.Advances in Neural Information Processing Systems,14, 585-591.
Billings, S. A., & Lee, K. L. (2002). Nonlinear Fisherdiscriminant analysis using a minimum squared errorcost function and the orthogonal least squaresalgorithm. Neural Networks, 15(2), 263-270.
Bolle, R. M., Connell, J. H., Pankanti, S., Ratha, N. K.,& Senior, A. W. (2004). Guide to biometrics. NewYork: Springer-Verlag.
Brunelli, R., &Falavigna, D. (1995). Personidentification using multiple cues. IEEE Transactionson Pattern Analysis and Machine Intelligence, 12(10),955–966.
Cawley, G. C., & Talbot, N. L. C. (2003). Efficientleave-one-out cross-validation of kernel fisherdiscriminant classifiers. Pattern Recognition, 36(11),2585-2592.
Chen, L. F., Liao, H. Y., Lin, J. C., Kao, M. D., & Yu,G. J. (2000). A new LDA-based face recognition system which can solve the small sample size problem.Pattern Recognition, 33(10), 1713-1726.
Draper, B. A., Baek, K., Bartlett, M. S., &Beveridge,J. R. (2003). Recognizing faces with PCA and ICA.Computer Vision and Image Understandin,, 91(1/2),115-137.
uin, R. P. W., &Loog, M. (2004). Lineardimensionality reduction via a heteroscedasticextension of LDA: the Chernoff criterion. IEEETransactions on Pattern Analysis and MachineIntelligence, 26(6), 732-739.
Fierrez-Aguilar, J., Ortega-Garcia, J., Gonzalez-Rodriguez, J., &Bigun, J. (2005). Discriminativemultimodal biometric authentication based on qualitymeasure. Pattern Recognition, 38(5), 777-779.
Gestel, T. V., Suykens, J. A. K., Lanckriet, G.,Lambrechts, A., De Moor, B., &Vanderwalle, J.(2002). Bayesian framework for least squares supportvector machineclassifiers, gaussianprocesss and kernel fisherdiscriminant analysis. Neural Computation, 15(5),1115-1148.
Gokberk, B., Salah, A. A., &Akarun, L. (2003). Rankbaseddecision fusion for 3D shape-based facerecognition. Lecture Notes in Computer Science, 3546,10191028.
Grother, P., &Tabassi, E. (2004). NIST BiometricScores Set - Release 1 (BSSR1). Available athttp://www.itl.nist.gov/iad/894.03/biometricscores.
Howland, P., & Park, H. (2004). Generalizingdiscriminant analysis using the generalized singular va
Lu e decomposition. IEEE Transactions on PatternAnalysis and Machine Intelligence, 26(8), 995-1006.
Jain, A., Bolle, R., &Pankanti, S. (1999a). Biometrics:Personal Identification inNetworked Society. Boston Hardbound: KluwerAcademic.
Jain, A. K., Prabhakar, S., & Chen, S. (1999b).Combining multiple matchers for a high securityfingerprint verification system. Pattern RecognitionLetters, 20, 1371-1379.
Jain, A. K., Ross, A., &Prabhakar, S. (2004). Anintroduction to biometric recognition. IEEE Trans.Circuits and Systems for Video Technology, 14(1), 4-20.
Jain, A. K., Nandakumar, K., & Ross, A. (2005). Scorenormalization in multimodel biometric systems. PatternRecognition, 38(5), 2270-2285.
Jin, Z., Yang, J. Y., Hu, Z. S., & Lou, Z. (2001). FaceRecognition based on uncorrelated discriminanttransformation. Pattern Recognition, 34(7), 1405-1416.
Jing, X.Y., Zhang, D., & Yang, J. Y. (2003). Facerecognition based on a group deci-sion-makingcombination approach. Pattern Recognition, 36(7),1675-1678.
Kim, H. C., Kim, D., Bang, S. Y., & Lee, S. Y. (2004).Face recognition using the second-order mixture-ofeigenfacesmethod. Pattern Recognition, 37(2), 337-349.
Kirby, M., &Sirovich, L. (1990). Application of theKL procedure for the characterization of human faces,IEEE Transactions on Pattern Analysis Machine Intelligence,12(1), 103-108.
Kong, A., Zhang, D., &Kamel, M. (2006). Palmprint identification using feature level fusion, Pattern Recognition, 39, 478-487.
Lathauwer, L., Moor, B., &Vndewalle, J.(2000). Amultilinear singular value decomposition. SIAMJournal on Matrix Analysis and Application, 21(4),1233-1278.
Lawrence, N. D., &Schölkopf, B. (2001). Estimatinga kernel Fisher discriminant in the presence of labelnoise. Proceedings of 18th International Conference onMachine Learning. San Francisco, CA, 306-313.
Liang, Z., & Shi, P. (2005). An analytical method forgeneralized low rank approximation matrices. PatternRecognition, 38(11), 2213-2216.
Liang, Z., Zhang, D., & Shi, P. (2007). The theoreticalanalysis of GRLDA and its applications. PatternRecognition, 40(3), 1032-1041.
Liu, K., Cheng, Y. Q., Yang, J. Y., & Liu, X. (1992).An efficient algorithm for Foley-Sammon optimal set of discriminant vectors byalgebraic method. International Journal of PatternRecognition and Artificial Intelligence, 6(5), 817-829.
Liu, K., Cheng, Y. Q., & Yang, J. Y. (1993). Algebraicfeature extraction for image recognition based on anoptimal discriminant criterion. Pattern Recognition,26(6), 903-911.
Liu, C., & Wechsler, H.(2000). Robust codingschemes for in dexing and retrieval from largeface databases. IEEE Transactions on ImageProcessing, 9(1), 132-137.
Liu, C., & Wechsler, H.(2001). A shape-and texturebasedenhanced Fisher classifierfor face recognition. IEEE Transactions on Image Processing, 10(4), 598-608.
Liu, C., & Wechsler, H. (2003). Independentcomponent analysis of Gabor features for facerecognition. IEEE Transactions on Neural Networks,14(4), 919-928.
Liu, C. (2004). Gabor-based kernel PCA withfractional power polynomial models for facerecognition. IEEE Transactions on Pattern Analysisand Machine Intelligence, 26(5), 572-581.
Loog, M., Duin, R. P. W., &Haeb-Umbach, R. (2001).Multiclass linear dimension reduction by weightedpairwise Fisher criteria. IEEE Transactions on PatternAnalysis and Machine Intelligence, 23(7), 762-766.
Lu, J. W., Plataniotis, K. N., &Venetsanopoulos, A.N. (2003a). Face recognition using LDA-basedalgorithms. IEEE Transactions on Neural Networks,14(1), 195200.
Lu, J., Plataniotis, K. N., &Venetsanopoulos, A. N.(2003b). Face recognition using kernel directdiscriminant analysis algorithms. IEEE Transactionson Neural Networks, 14(1), 117-126.
Lu, G., Zhang, D., & Wang, K. (2003). PalmprintRecognition using Eigenpalms Features, PatternRecognition Letters, 24(9-10), 1463-1467.
Miller, B. (1994). Vital signs of identity. IEEESpectrum, 31(2), 22-30.
Mika, S., Rätsch, G., Weston, J., Schölkopf, B., &Müller, K. R. (1999a). Fisher discriminant analysis withkernels. IEEE International Workshop on NeuralNetworks for Signal Processing, Madison, Wisconsin, August 23-25 (pp. 41-48).
Mika, S., Rätsch, G., Schölkopf, B., Smola, A.,Weston, J., & Müller, K. R. (1999b). Invariant featureextraction and classification in kernel spaces. Advancesin Neural Information Processing Systems, 12, 526-532.
Mika, S., Rätsch, G., & Müller, K. R. (2001a). Amathematical programming approach to the KernelFisher algorithm. Advances in Neural InformationProcessing Systems, 13, 591-597.
Mika, S., Smola, A. J., &Schölkopf, B. (2001b). Animproved training algorithmfor kernel fisherdiscriminants. International Workshopon Artificial Intelligence and Statistics, San Francisco,CA, January 4-7, 98-104.
Mika, S., Rätsch, G., Weston, J., Schölkopf, B., Smola,A., & Müller, K. R. (2003). Constructing descriptiveand discriminative non-linear features: Rayleigh coefficientsin kernel feature spaces. IEEE Transactions onPattern Analysis and Machine Intelligence, 25(5), 623-628.
Moghaddam, B. (2002). Principal manifolds andprobabilistic subspaces for visual recognition. IEEETransactions on Pattern Analysis Machine Intelligence,24(6), 780-788.
Nandakumar, K., Chen, Y., Jain, A. K., &Dass, S.(2006). Quality-based score level fusion inmultibiometric systems. Proceedings of InternationalConference on Pattern Recognition, Hong Kong,August 20-24 (pp. 473-476).
Pentland, A., Moghaddam, B., &Starner, T. (1994).View-based and mododulareigenspaces for facerecognition. IEEE Conference on Computer Vision andPattern Recognition, Seattle, June 21-23 (pp. 84-91).
Pentland, A. (2000). Looking at people: sensing forubiquitous and wearable computing. IEEE Transactionson Pattern Analysis Machine Intelligence, 22(1), 107-119.
Petridis, S., &Perantonis, S. T. (2004). On the relationbetween discriminant analysis and mutual informationfor supervised linear feature extraction. PatternRecognition, 37(5), 857-874.
Poh, N., &Bengio, S. (2006). Database, protocol andtools for evaluating score-level fusion algorithms inbiometric authentication. Pattern Recognition, 39(2),223-233.
Ross, A., &Govindarajan, R. (2005). Feature levelfusion using hand and face biometrics. Proceedings ofSPIE Conference on Biometric Technology for HumanIdentification, Orlando, USA, March 28 (pp. 196-204).
Roth, V., &Steinhage, V. (2000). Nonlineardiscriminant analysis using kernel functions. Advancesin Neural Information Processing Systems, 12, 568-574.
Roweis, S. T., & Saul, L. K. (2000). Nonlineardimensionality reduction by locally linear embedding.Science, 290, 2323-2326.
Schölkopf, B., Smola, A., & Muller, K. R. (1998).Nonlinear component analysis as a kernel eigenvalueproblem. Neural Computation, 10(5), 1299-1319.
Sirovich, L., & Kirby, M. (1987). Low-dimensionalprocedure for characterization of human faces. Journalof Optical Society of America A, 4, 519-524.
Swets, D. L., &Weng, J.(1996). Using discriminanteigenfeatures for image retrieval. IEEE Transactions on Pattern Analysis and Machine Intelligence, 18(8), 831-836.
Tao, D., Li, X., Hu, W., Maybank, S. J., & Wu, X.(2005). Supervised tensor learning. IEEE InternationalConference on Data Mining, Houston, Texas,November 27-30 (pp. 450-457).
Tenenbaum, J. B., de Silva, V., & Langford, J. C.(2000). A global geometric framework for nonlineardimensionality reduction. Science, 290, 2319-2323.
Tian, Q., Barbero, M., Gu, Z. H., & Lee, S. H. (1986).Image classification by the Foley-Sammon transform.Optical Engineering, 25(7) 834-839.
Turk, M., &Pentland, A. (1991a). Eigenfaces forrecognition. Journal of Cognitive Neuroscience, 3(1),71-86.
Turk, M., &Pentland, A. (1991b). Face recognitionusing Eigenfaces. IEEE Conference on ComputerVision and Pattern Recognition, Maui, Hawaii, June 3-6(pp. 586-591).
Ulery, B., Hicklin, A. R., Watson, C., Fellner, W., &Hallinan, P. (2006). Studies of biometric fusion. NISTTechnical Report IR7346.
Vasilescu, M. A. O., &Terzopoulos, D. (2003).Multilinear subspace analysis of image ensembles.IEEE Conference on Computer Vision and PatternRecognition, Madison, Wisconsin, June 16-22 (pp. 93-99).
Wayman, J. (2005). Biometric systems: Technology,design and performance evaluation. London: Springer.
Wu, X., Zhang, D., & Wang, K. (2003). Fisherpalmsbased Palmprint recognition. Pattern RecognitionLetters, 24(15), 2829-2838.
Xu, J., Zhang, X., & Li, Y. (2001). Kernel MSEalgorithm: a unified framework for KFD, LS-SVM, andKRR. Proceedings of the International Joint Conferenceon Neural Networks, Washington, D.C., July 15-19 (pp.1486-1491).
Xu, Y., Yang, J. Y., & Yang, J. (2004). A reformativekernel Fisher discriminant analysis. PatternRecognition, 37(6), 1299-1302.
Yan, S., Xu, D., Yang, Q., Zhang, L., Tang, X., &Zhang, H. J. (2007). Multilinear discriminant analysisfor face recognition. IEEE Transactions on ImageProcessing, 16(1), 212-220.
Yang, M. H. (2002). Kernel Eigenfaces vs. kernelFisherfaces: Face recognition using kernel methods.Proceedings of the Fifth IEEE International Conferenceon Automatic Face and Gesture Recognition (RGR’02),Washington D. C., May 21 (pp. 215-220).
Yang, J., & Yang, J. Y. (2002). Generalized K-Ltransform based combined feature extraction. PatternRecognition, 35(1), 295-297.
Yang, J., & Yang, J. Y. (2003). Why can LDA beperformed in PCA transformed space? PatternRecognition, 36(2), 563-566.
Yang, J., Yang, J. Y., Zhang, D., & Lu, J. F. (2003a).Feature fusion: parallel strategy vs. serial strategy.Pattern Recognition, 36(6), 1369-1381.
Yang, J., Yang, J. Y., Frangi, A. F. ,& Zhang, D.(2003b). Uncorrelated projection discriminant analysisand its application to face image feature extraction.International Journal of Pattern Recognition andArtificial Intelligence, 17(8), 1325-1347.
Yang, J., Jin, Z., Yang, J. Y., Zhang, D., &Frangi, A.F. (2004a). Essence of Kernel Fisher Discriminant:KPCA plus LDA. Pattern Recognition, 37(10), 2097-2100.
Yang, J., Frangi, A. F., & Yang, J. Y. (2004b). A newkernel Fisher discriminant algorithm with application tonface recognition, Neurocomputing, 56, 415-421.
Yang, J., Zhang, D., Frangi, A. F., & Yang, J. Y.(2004c). Two-dimensional PCA: A new approach toface representation and recognition. IEEE Transactionson Pattern Analysis and Machine Intelligence, 26(1),131-137.
Yang, J., Zhang, D., Yang, J. Y., Zhong, J., &Frangi,A. F. (2005). KPCA plus LDA: A complete kernelFisher discriminant framework for feature extractionand recognition. IEEE Transactions on Pattern Analysisand Machine Intelligence, 27(2), 230-244.
Ye, J. (2004). Generalized low rank approximations ofmatrices. The Twenty-First International Conference onMachine Learning, Bonn, Germany, August 7-11.
Ye, J., Janardan, R., Park, C. H., & Park, H. (2004).An optimization criterion for generalized discriminantanalysis on undersampled problems. IEEE Transactionson Pattern Analysis and Machine Intelligence, 26(8),982-994.
Yu, H., & Yang, J. (2001). A direct LDA algorithm forhigh-dimensional data—with application to facerecognition. Pattern Recognition, 34(10), 2067-2070.
Yuen, P. C., & Lai, J. H. (2002). Face representationusing independent component analysis. PatternRecognition, 35(6), 1247-1257.
Zhang, D. (2000). Automated biometrics: technologies and systems. Boston Hardbound: Kluwer Academic.
Zhang, D. (2002). Biometrics solutions forauthentication in an e-world. Boston Hardbound:Kluwer Academic.
Zhang, D. (2004). Palmprint authentication. BostonHardbound: Kluwer Academic Publishers.
Zhang, D., Peng, H., Zhou, J., & Pal, S. K. (2002). Anovel face recognition system using hybrid neural anddual eigenfaces methods. IEEE Transactions on System,Man, and Cybernetics, Part A, 32(6), 787-793.
Zhang, D., Kong, W. K., You, J., & Wong, M.(2003).On-line palmprint identification.IEEE Transactions on Pattern Analysis MachineIntelligence, 25(9), 1041-1050.
[218]Zhao, L., & Yang, Y. (1999). Theoretical analysis ofillumination in PCA-based vision systems. PatternRecognition, 32(4), 547-564.
Zhao, W., Krishnaswamy, A., Chellappa, R., Swets,D., &Weng, J. (1998). Discriminant analysis of
principal components for face recognition. In H.Wechsler,P.J. Phillips, V. Bruce, F.F. Soulie , & T.S. Huang(Eds.), Face Recognition: From Theory to Applications73-85, Berlin: Springer-Verlag.
[221]Zhao, W., Chellappa, R., & Rosenfeld, A. (2003). Facerecognition: a literature survey. ACM ComputingSurvey, 36(4), 399-458.
Zuo, W., Wang, K., & Zhang, D. (2005). Bi-directionalPCA with assembled matrix distance metric.International Conference on Image Processing Genova,Italy, September 11-14 (pp. 958-961).
Zuo, W., Wang, K., Zhang, D., & Zhang, H. (2007).Combination of two novel LDA-based methods for facerecognition. Neurocomputing, 70(4-6), 735-742.
Takaaki Akimoto, YasuhitoSuenaga, and RichardS. Wallace, Automatic Creation of 3D Facial Models,IEEE Computer Graphics & Applications, Sep., 1993.
Coleman EN, Jain R, Obtaining 3-DimensionalShape of Textured and Specular Surfaces Using Four-Source Photometry, IEEE Computer Graphics andImage Processing, 18:309-328, 1982.
P. Fua and Y.G. Leclerc, Taking Advantage ofImage-Based and Geometry- Based Constraints toRecover 3-D Surfaces, Computer Vision and ImageUnderstanding, 64(1):111-127, Jul., 1996.
Horace H.S. Ip, Lijin Yin, Constructing a 3Dindividual head model from two orthogonal views. TheVisual Computer, Springer-Verlag, 12:254-266, 1996.
TsuneyaKurihara and Kiyoshi Arai, ATransformation Method for Modeling and Animation ofthe Human Face from Photographs , ComputerAnimation, Springer-Verlag Tokyo, pp. 45-58, 1991.
Yuencheng Lee, DemetriTerzopoulos, and KeithWaters, Realistic Modeling for Facial Animation, InComputer Graphics (Proc. SIGGRAPH), pp. 55-62,1996.
N. Magnenat-Thalmann, D. Thalmann, Thedirection of Synthetic Actors in the film Rendez-vous àMontrèal, IEEE Computer Graphics and Applications,7(12):9-19, 1987.
N. Magnenat-Thalmann, I. S. Pandzic, Jean-Claude Moussaly, Zhyong Huang, JianhuaShen, The
Making of Xian Terra-cotta Soldiers,ComputerGraphics: Developments in Virtual Environments,(Academic Press) (ISBN 0-12-227741-4), 1995.
Marc Proesmans, Luc Van Gool. Readingbetween the lines - a method for extracting dynamic 3Dwith texture. In Proceedings of VRST, pp. 95-102,1997.
HrroshiSaji, Hirohisa Hioki, YoshihisaShinagawa, Kensyu Yoshida and Tosiyasu L. Kunii,Extracting of 3D Shapes from the moving human faceusing lighting Switch Photometry.Creating andAnimating the virtual world, Springer-Verlag, Tokyo,pp. 69-86, 1992.
Lance Williams,Performance-Driven FacialAnimation, InComputer Graphics (Proc. SIGGRAPH) ,pp. 235-242, 1990.
Liu et al., 2001b] Liu, Z., Zhang, Z., Jacobs, C., andCohen, M. (2001b). Rapid modeling of animated facesfrom video. Journal of Visualization and ComputerAnimation, 12(4):227–240.
Kang and Jones, 1999] Kang, S. B. and Jones, M.(1999). Appearance-based structure from motion usinglinear classes of 3-d models. Manuscript.
[Vetter and Poggio, 1997] Vetter, T. and Poggio, T.(1997). Linear object classes and image synthesis froma single example image. IEEE Transations on PatternAnalysis and Machine Intelligence, 19(7):733–742.
Blanz and Vetter, 1999] Blanz, V. and Vetter, T.(1999). A morphable model for the synthesis of 3dfaces. In Computer Graphics, Annual ConferenceSeries, pages 187–194. Siggraph.
[Fua and Miccio, 1998] Fua, P. and Miccio, C. (1998).From regular images to animated heads: A least squaresapproach. In European Conference on ComputerVision, pages 188–202.
[Pighin et al., 1998] Pighin, F., Hecker, J., Lischinski,D., Szeliski, R., and Salesin, D. H. (1998).Synthesizingrealistic facial expressions from photographs. In Proc.SIGGRAPH 98, pages 75–84.
[Zheng, 1994] Zheng, J. Y. (1994). Acquiring 3-dmodels from sequences of contours. IEEE Transactionson Pattern Analysis and Machine Intelligence, 16(2):163–178.
[Akimoto et al., 1993] Akimoto,T.,Suenaga,Y.,andWallace,R.S.(1993). Automatic 3d facial models. IEEEComputer Graphics and Applications, 13(5):16–22.
Dariush et al., 1998] Dariush, B., Kang, S. B., andWaters, K. (1998). Spatiotemporal analysis of faceprofiles: Detection, segmentation, and registration. InProc. of the 3rd International Conference on AutomaticFace and Gesture Recognition, pages 248–253. IEEE.
[H.S.Ip and Yin, 1996] H.S.Ip, H. and Yin, L. (1996).Constructing a 3d individualized head model from twoorthogonal views. The Visual Computer, (12):254–266.
Lee, Y. C., Terzopoulos, D., andWaters, K. (1995). Realistic modeling for facialanimation. In Computer Graphics, Annual ConferenceSeries, pages 55–62. SIGGRAPH.
Lee, Y. C., Terzopoulos, D., andWaters, K. (1993). Constructing physicsbased facialmodels of individuals. In Proceedings of GraphicsInterface, pages 1–8.
Marschner, S. R., Guenter, B.,and Raghupathy, S. (2000). Modeling and rendering forrealistic facial animation. In Rendering Techniques,pages 231–242. Springer Wien New York.
Ostermann et al., 1998] Ostermann, J., Chen, L.,, andHuang, T. S. (1998). Animated talking head with
personalized 3d head model. The Journal of VLSISignal Processing–Systems for Signal, Image, andVideo Technology, 20(1):97–105.
Cyberware(2003).Cyberware.Description retrived from http://www.cyberware.com/.
Eyetronics(2003).Eyetronics.Description retrived from http://www.eyetronics.com/.
C. Beumier, M. Acheroy, Face verification from 3Dand grey level cues, Pattern Recognition Letters 22(2001) 1321–1329.
A.M. Bronstein, M.M. Bronstein, R. Kimmel,Expression-invariant 3D face recognition, in:International Conference on Audio- and Video-BasedPerson Authentication (AVBPA 2003), LNCS, vol.2688, 2003, pp. 62–70.
K. Chang, K. Bowyer, P. Flynn, Face recognitionusing 2D and 3D facial data, in: Multimodal UserAuthentication Workshop, December 2003, pp. 25–32.
Godil, S. Ressler, P. Grother, Face recognition using3D facial shape and color map information: comparisonand combination, i Biometric Technology for HumanIdentification, SPIE, vol. 5404, April 2005, pp. 351–361.
Gokberk, A.A. Salah, L. Akarun, Rank-based decisionfusion for 3D shape-based face recognition, in:International Conference on Audio- and Video-based Biometric Person Authentication (AVBPA 2005),LNCS, vol. 3546, July 2005, pp. 1019–1028.
M. Husken, M. Brauckmann, S. Gehlen, C. von derMalsburg, Strategies and benefits of fusion of 2D and3D face recognition, in: IEEE Workshop on FaceRecognition Grand Challenge Experiments, June 2005.
S. Lao, Y. Sumi, M. Kawade, F. Tomita, 3D templatematching for pose invariant face recognition using 3Dfacial model built with iso-luminance line based stereovision, in: International Conference on PatternRecognition (ICPR 2000), 2000, pp. II:911–916.
X. Lu, D. Colbry, A.K. Jain, Matching 2.5D scans forface recognition, in: International Conference onPattern Recognition (ICPR 2004), 2004, pp. 362–366.
X. Lu, A.K. Jain, Integrating range and textureinformation for 3D face recognition, in: 7th IEEEWorkshop on Applications of Computer Vision(WACV _05), 2005, pp. 155–163.
T. Maurer, D. Guigonis, I. Maslov, B. Pesenti, A.Tsaregorodtsev, D. West, G. Medioni, Performance ofgeometrixactiveidtm 3D face recognition engine on thefrgc data, in: IEEE Workshop on Face Recognition Grand Challenge Experiments, June 2005.
G. Medioni, R. Waupotitsch, Face recognition andmodeling in 3D, in: IEEE International Workshop onAnalysis and Modeling of Faces and Gestures (AMFG2003), October 2003, pp. 232–233.
K. Messer, J. Matas, J. Kittler, J. Luettin, G. Maitre,XM2VTSDB: the extended M2VTS database, in:
Second International Conference on Audio- and VideobasedBiometric Person Authentication, 1999, pp. 72–77.
Neven Vision, Inc. Nevenvision machine visiontechnology. <http:// www.nevenvision.com/>.
T. Papatheodorou, D. Reuckert, Evaluation ofautomatic 4D face recognition using surface and textureregistration, in: Sixth International Conference onAutomated Face and Gesture Recognition, May 2004,pp. 321–326.
F. Tsalakanidou, S. Malassiotis, M. Strintzis, Faceauthentication and authentication using color and depthimages, IEEE Transactions on Image Processing 14 (2)(2005) 152–168.
F. Tsalakanidou, S. Malassiotis, M. Strintzis,Integration of 2D and 3D images for enhanced faceauthentication, in: Sixth International Conference onAutomated Face and Gesture Recognition, May2004,pp. 266–271.
F. Tsalakanidou, D. Tzocaras, M. Strintzis, Use ofdepth and colour eigenfaces for face recognition,Pattern Recognition Letters 24 (2003) 1427–1435.
Y. Wang, C. Chua, Y. Ho, Facial feature detection andface recognition from 2D and 3D images, PatternRecognition Letters 23 (2002) 1191–1202.
C. Xu, Y. Wang, T. Tan, L. Quan, Automatic 3D facerecognition combining global geometric features withlocal shape variation information, in: Sixth InternationalConference on Automated Face and GestureRecognition, May 2004, pp. 308–313.
Blanz, V., Vetter, T., 2003. Face recognition based onfitting a 3D morphable model. IEEE Trans. PatternAnal. Machine Intell. 25 (9), 1063–1074.
Lu, X., Hsu, R., Jain, A., Kamgar-Parsi, B., Kamgar-Parsi, B., 2004. Face recognition with 3D model-basedsynthesis. In: Proc. Internat. Conf. on BiometricAuthentication (ICBA), Hong Kong, July, pp. 139–146.
Hu, Y.X., Jiang, D.L., Yan, S.C., Zhang, H.J., 2004.Automatic 3D Reconstruction for Face Recognition. In:Proc. Proceeding of IEEE International Conf. onAutomatic Face and Gesture Recognition (FGR 2004),Seoul, Korea, May, pp. 843–850.
Ansari, A., Abdel-Mottaleb, M., 2003. 3-D facemodeling using two views and a generic face modelwith application to 3-D face recognition. In: Proc. IEEEConf. on Advanced Video and Signal BasedSurveillance (AVSS 2003) Miami, FL, USA, July, pp.37–44.
Cook, J., Chandran, V., Sridharan, S., Fookes, C.,2004. Face recognition from 3D data using iterativeclosest point algorithm and Gaussian mixture models.In: Proc. 2nd Internat. Symposium on 3D DataProcessing, Visualization and Transmission (3DPVT2004), Thessaloniki, Greece, 6–9 September.
Irfanoglu, M.O., Gokberk, B., Akarun, L., 2004. 3Dshape-based face recognition using automaticallyregistered facial surfaces. In: Proc. 17th Internat. Conf.on Pattern Recognition (ICPR2004), Cambridge, pp.183–186.
Medioni, G., Waupotitsch, R., 2003. Face recognitionand modeling in 3D. In: Proc. IEEE Internat. Workshopon Analysis and Modeling of Faces and GestureRecognition (AMFG’03). Nice, France, October, pp.232–233.
Lu, X., Colbry, D., Jain, A.K., 2004. Threedimensionalmodel based face recognition. In: Proc.17th Internat. Conf. on Pattern Recognition(ICPR2004), Cambridge, pp. 362–366.
Gordon, G., 1991. Face recognition based on depthmaps and surface curvature. In: Proc. GeometricMethods in Computer Vision, SPIE, pp. 1–12.
Moreno, A.B., Sanchez, A., Velez, J.F ., Diaz, F.J.2003. Face recognition using 3D surface-extracteddescriptors. In: Proc. Irish Machine Vision and Image,(IMVIP’03), September.
Chua, C.S., Han, F., Ho, Y.K., 2000. 3D human facerecognition using point signature. In: Proc. 4th IEEEInternat. Conf. on Automatic Face and GestureRecognition (FG 2000) Grenoble, France, March,pp.233–238.
Tanaka, H.T., Ikeda, M., Chiaki, H., 1998. Curvaturebasedface surface recognition using sphericalcorrelation – principal directions for curved objectrecognition. In: Proc. 3rd Internat. Conf. on Face &Gesture Recognition (FG’98), Nara, Japan, April, pp.372–377.
Wang, Y., Pan, G., Wu, Z., 2004. Sphere-spin-image:A viewpointinvariant surface representation for 3D facerecognition. In: Proc. Internat. Conf. on ComputationalScience (ICCS’04), Lecture Notes in ComputerScience, Vol. 3037 (June), pp. 427–434.
Gao, Y., Leung, M.K.H., 2002. Face recognition usingline edge map. IEEE Trans. Pattern Anal. MachineIntell. 24 (6), 764–779.
Heseltine, T., Pears, N., Austin, J., 2004a. Threedimensionalface recognition: An eigensurface approach. In: Proc. Internat. Conf. on Image Processing(ICIP 2004), October, Singapore.
Heseltine, T., Pears, N., Austin, J., 2004b. Threedimensionalface recognition: A fischersurfaceapproach. In: Proc. Image Analysis and Recognition:Internat. Conf. (ICIAR 2004), Porto, Portugal(September– October), pp. 684–691.
Chang, K., Bowyer, K., Flynn, P., 2004. An evaluationof multi-modal 2D + 3D face biometrics. IEEE Trans.Pattern Anal. Machine Intell. (PAMI 2004).
Bronstein, A., Bronstein, M., Kimmel, R., 2003.Expression-invariant 3D face recognition. Proc. Audio&Video-based Biometric Person Authentication(AVBPA). In: Lecture Notes in Computer Science, Vol.2688. Springer, pp. 62–69.
Tsalakanidou, F., Tzovaras, D., Strintzis, M.G., 2003.Use of depth and colour Eigenfaces for facerecognition. Pattern Recognition Lett. 24 (9–10), 1427-1435.
Papatheodorou, T., Rueckert, D., 2004. Evaluation ofautomatic 4D face recognition using surface and textureregistration. In: Proc. Sixth IEEE Internat. Conf. onAutomatic Face and Gesture Recognition Seoul, Korea,May, pp. 321–326.
P. Besl, N. McKay, A method for registration of 3Dshapes, IEEE Trans. on PAMI 14 (1992) 239–256.
G. Medioni, R. Waupotitsch, Face recognition andmodeling in 3D, in: Proceedings of the IEEEInternational Workshop on Analysis and Modeling ofFaces and Gestures, 2003, pp. 232–233.
X. Lu, D. Colbry, A. Jain, Three-dimensional modelbased face recognition, in: Proceedings of theInternational Conference on Pattern Recognition, pp.362–366, 2004.
S. Malassiotis, M. Strintzis, Pose and illuminationcompensation for 3D face recognition, in: Proceedingsof the International Conference on Image Processing,vol. 1, pp. 91–94, 2004.
T. Papatheodorou, D. Reuckert, Evaluation ofautomatic 4D face recognition using surface and textureregistration, in: Proceedings of the Sixth InternationalConference on Automated Face and GestureRecognition, pp. 321–326, 2004.
C. Xu, Y. Wang, T. Tan, L. Quan, Automatic 3D facerecognition combining global geometric features withlocal shape variation information, in: Proceedings of theSixth International Conference on Automated Face andGesture Recognition, pp. 308–313, 2004.
X. Lu, A.K. Jain, Deformation analysis for 3D facematching, in: Proceedings of the Seventh IEEEWorkshop on Applications of Computer Vision, pp. 99–104, 2005.
M.O. I?rfanoglu, B. Go¨kberk, L. Akarun, 3D shapebasedface recognition using automatically registeredfacial surfaces, in: Proceedings of the InternationalConference on Pattern Recognition, pp. 183–186, 2004.
A.M. Bronstein, M.M. Bronstein, R. Kimmel,Expression-invariant 3D face recognition, Proceedingsof Audio- and Video-based Biometric PersonAuthentication, LNCS 2688 (2003) 62–69.
A.M. Bronstein, M.M. Bronstein, A. Spira, R. Kimmel,Face recognition from facial surface metric in: T.Pajdla, J. Matas (Eds.), Proceedings of the Eighth European Conference on Computer Vision, LNCS, vol.3022 (2004), pp. 225–237.
Bronstein, M. Bronstein, R. Kimmel, Threedimensionalface recognition, International Journal ofComputer Vision 64 (1) (2005) 5–30.
Hesher, A. Srivastava, G. Erlebacher, A noveltechnique for face recognition using range imaging, in:Proceedings of the Seventh International Symposium onSignal Processing and Its Applications, vol. 2, pp. 201–204, 2003.
Y. Lee, K. Park, J. Shim, T. Yi, 3D face recognitionusing statistical multiple features for the local depthinformation, in: Proceedings of the 16th InternationalConference on Vision Interface, 2003.
Srivastava, X. Liu, C. Hesher, Face recognition usingoptimal linear components of range images, submittedto Journal of Image and Vision Computing.
K.I. Chang, K.W. Bowyer, P.J. Flynn, An evaluationof multi-modal 2DC3D face biometrics, IEEETransactions on PAMI 27 (4) (2005) 619– 624.
F. Tsalakanidou, D. Tzocaras, M. Strintzis, Use ofdepth and colour eigenfaces for face recognition,Pattern Recognition Letters 24 (2003) 1427–1435.
Beumier, M. Acheroy, Automatic 3D faceauthentication, Image and Vision Computing 18 (4)(2000) 315–321.
Beumier, M. Acheroy, Face verification from 3D andgrey level cues, Pattern Recognition Letters 22 (2001)1321–1329.
C.-S. Chua, F. Han, Y.-K. Ho, 3D human facerecognition using point signature, in: Proceedings ofInternational Conference on Automatic Face andGesture Recognition, pp. 233–237, 2000.
Y. Wang, C. Chua, Y. Ho, Facial feature detection andface recognition from 2D and 3D images, PatternRecognition Letters 23 (10) (2002) 1191–1202.
Z. Wu, Y. Wang, G. Pan, 3D face recognition usinglocal shape map, in: Processings of the InternationalConference on Image Processing, vol. 3, 2004, pp.2003–2006.
G. Gordon, Face recognition based on depth andcurvature features, in: Proceedings of the IEEEComputer Society Conference on Computer Vision andPattern Recognition, pp. 108–110, 1992.
H.T. Tanaka, M. Ikeda, H. Chiaki, Curvature-basedface surface recognition using spherical correlationprincipal directions for curved object recognition, in:Proceedings of the Third International Conference onAutomated Face and Gesture Recognition, pp. 372–377,1998.
A.B. Moreno, A. Sanchez, J.F. Velez, F.J. Diaz, Facerecognition using 3D surface-extracted descriptors, in:Proceedings of the Irish Machine Vision and ImageProcessing Conference, 2003.
S. Tsutsumi, S. Kikuchi, M. Nakajima, Faceidentification using a 3D gray-scale image-a method forlessening restrictions on facial directions, in:Proceedings of the third IEEE International Conferenceon Automatic Face and Gesture Recognition, pp. 306–311, 1998.
Go¨kberk, A.A. Salah, L. Akarun, Rank-based decisionfusion for 3D shape-based face recognition in: T.Kanade, A. Jain, N.K. Ratha (Eds.), Proceedings of Audio- and Video-based Biometric PersonAuthentication, LNCS, vol. 3456 (2005), pp. 1019–1029.
G. Pan, Z. Wu, Y. Pan, Automatic 3D face verificationfrom range data, in: Proceedings of the InternationalConference on Acoustics, Speech, and SignalProcessing, vol. 3, pp. 193–196, 2003.
G. Pan, Z. Wu, 3D face recognition from range data,International Journal of Image and Graphics 5 (3)(2005) 573-583.
K.I. Chang, K.W. Bowyer, P.J. Flynn, Facerecognition using 2d and 3d facial data, in: ACMWorkshop on Multimodal User Authentication, 2003,25–32.
A.S. Mian, M. Bennamoun, R. Owens, Keypointdetection and local feature matching for textured 3dface recognition, Int. J. Comput. Vis. 79 (1) (2007) 1–12.
M. Husken, M. Brauckmann, S. Gehlen, C.V. derMalsburg, Strategies and benefits of fusion of 2d and 3dface recognition, in: CVPR’05: Proceedings of the 2005IEEE Computer Society Conference on ComputerVision and Pattern Recognition (CVPR’05) –Workshops, Washington, DC, USA, IEEE ComputerSociety, 2005.
K.I. Chang, K. Bowyer, P.J. Flynn, An evaluation ofmulti-modal 2d+3d face biometrics, IEEE Trans.Pattern Anal. 27 (2005) 619–624.
F. Tsalakanidou, D. Tzovaras, M. Strintzis, Use ofdepth and colour eigenfaces for face recognition,Pattern Recogn. Lett. (2003) 1427–1435.
M. Webster, Introduction to GeometricMorphometrics, 2006.
3DMD Systems. 3q qlonerator.<http://www.3q.com/offerings_prod. htm/>.
B. Achermann, H. Bunke, Classifying range images ofhuman faces with Hausdorff distance, in: 15-thInternational Conference on Pattern Recognition,September 2000, pp. 809–813.
B. Achermann, X. Jiang, H. Bunke, Face recognitionusing range images, International Conference on VirtualSystems and MultiMedia (1997) 129–136.
C. Beumier, M. Acheroy, Face verification from 3Dand grey level cues, Pattern Recognition Letters 22(2001) 1321–1329.
V. Blanz, T. Vetter, Face recognition based on fitting a3D morphable model, IEEE Transactions on PatternAnalysis and Machine Intelligence 25 (2003) 1063–1074.
C. Boehnen, P.J. Flynn, Accuracy of 3D scanningtechnologies in a face scanning context, in: FifthInternational Conference on 3D Imaging and Modeling(3DIM 2005), June 2005, pp. 310–317.
K.W. Bowyer, Face recognition technology and thesecurity versus privacy tradeoff, IEEE Technology andSociety (2004) 9–20.
K.W. Bowyer, K. Chang, P.J. Flynn, A survey of 3Dand multi-modal 3D + 2D face recognition, in: 17-thInternational Conference on Pattern Recognition,August 2004, pp. 358–361.
K.W. Bowyer, K. Chang, P.J. Flynn, A survey of 3Dand multimodal 3D + 2D face recognition, Face Processing: Advanced Modeling and Methods, toappear.
A.M. Bronstein, M.M. Bronstein, R. Kimmel,Expression-invariant 3D face recognition, in:International Conference on Audio- and Video-BasedPerson Authentication (AVBPA 2003), LNCS, vol.2688, 2003, pp. 62–70.
A.M. Bronstein, M.M. Bronstein, R. Kimmel, Threedimensionalface recognition, International Journal ofComputer Vision (2005) 5–30.
J.Y. Cartoux, J.T. LaPreste, M. Richetin, Faceauthentication or recognition by profile extraction fromrange images, in: Proceedings of the Workshop onInterpretation of 3D Scenes, 1989, pp. 194– 199.
K. Chang, K. Bowyer, P. Flynn, An evaluation ofmulti-modal 2D + 3D face biometrics, IEEETransactions on Pattern Analysis and MachineIntelligence 27 (4) (2005) 619–624.
K. Chang, K. Bowyer, P. Flynn, Face recognitionusing 2D and 3D facial data, in: Multimodal UserAuthentication Workshop, December 2003, pp. 25–32.
K. Chang, K.W. Bowyer, S. Sarkar, B. Victor,Comparison and combination of ear and face images forappearance-based biometrics, IEEE Transactions onPattern Analysis and Machine Intelligence 25 (9)(2003) 1160–1165.
K.I. Chang, K.W. Bowyer, P.J. Flynn, Adaptive rigidmulti-region selection for handling expression variationin 3D face recognition, in: IEEE Workshop on FaceRecognition Grand Challenge Experiments, June 2005.
C. Chua, F. Han, Y.K. Ho, 3D human face recognitionusing point signature, IEEE International Conference onAutomatic Face and Gesture Recognition (2000) 233–238.
L. Farkas, Anthropometry of the Head and Face,Raven Press, New York, 1994.
Godil, S. Ressler, P. Grother, Face recognition using3D facial shape and color map information: comparisonand combination, i Biometric Technology for HumanIdentification, SPIE, vol. 5404, April 2005, pp. 351–361.
Gokberk, A.A. Salah, L. Akarun, Rank-based decisionfusion for 3D shape-based face recognition, in:International Conference on Audio- and Video-basedBiometric Person Authentication (AVBPA 2005),LNCS, vol. 3546, July 2005, pp. 1019–1028.
G. Gordon, Face recognition based on depth andcurvature features, Computer Vision and PatternRecognition (CVPR) (June) (1992) 108–110.
Hesher, A. Srivastava, G. Erlebacher, A noveltechnique for face recognition using range imaging, in:Seventh International Symposium on Signal Processingand Its Applications, 2003, pp. 201–204.
M. Husken, M. Brauckmann, S. Gehlen, C. von derMalsburg, Strategies and benefits of fusion of 2D and3D face recognition, in: IEEE Workshop on FaceRecognition Grand Challenge Experiments, June 2005.
M.L. Koudelka, M.W. Koch, T.D. Russ, A prescreenerfor 3D face recognition using radial symmetry and theHausdorff fraction, in:IEEE Workshop on FaceRecognition Grand Challenge Experiments, June 2005.
S. Lao, Y. Sumi, M. Kawade, F. Tomita, 3D templatematching for pose invariant face recognition using 3D facial model built with iso-luminance line based stereovision, in: International Conference on PatternRecognition (ICPR 2000), 2000, pp. II:911–916.
J.C. Lee, E. Milios, Matching range images of humanfaces, in: International Conference on Computer Vision,1990, pp. 722–726.
Y. Lee, K. Park, J. Shim, T. Yi, 3D face recognitionusing statistical multiple features for the local depthinformation, in: 16th International Conference onVision Interface, June 2003. Available at<www.visioninterface.org/vi2003/>.
Y. Lee, J. Shim, Curvature-based human facerecognition using depth-weighted Hausdorff distance,in: International Conference on Image Processing(ICIP), 2004, pp. 1429–1432.
Y. Lee, H. Song, U. Yang, H. Shin, K. Sohn. Localfeature based 3D face recognition, in: InternationalConference on Audio- and Videobased BiometricPerson Authentication (AVBPA 2005), LNCS,vol.3546, July 2005, pp. 909–918.
X. Lu, D. Colbry, A.K. Jain, Matching 2.5D scans forface recognition, in: International Conference onPattern Recognition (ICPR 2004), 2004, pp. 362–366.
X. Lu, A.K. Jain, Deformation analysis for 3D facematching, in: 7th IEEE Workshop on Applications ofComputer Vision (WACV _05), 2005, pp. 99–104.
X. Lu, A.K. Jain, Integrating range and textureinformation for 3D face recognition, in: 7th IEEEWorkshop on Applications of Computer Vision(WACV _05), 2005, pp. 155–163.
T. Maurer, D. Guigonis, I. Maslov, B. Pesenti, A.Tsaregorodtsev, D. West, G. Medioni, Performance ofgeometrixactiveidtm 3D face recognition engine on thefrgc data, in: IEEE Workshop on Face RecognitionGrand Challenge Experiments, June 2005.
G. Medioni, R. Waupotitsch, Face recognition andmodeling in 3D, in: IEEE International Workshop onAnalysis and Modeling of Faces and Gestures (AMFG2003), October 2003, pp. 232–233.
K. Messer, J. Matas, J. Kittler, J. Luettin, G. Maitre,XM2VTSDB: the extended M2VTS database, in:Second International Conference on Audio- and VideobasedBiometric Person Authentication, 1999, pp. 72–77.
J. Min, K.W. Bowyer, P. Flynn, Using multiple galleryand probe images per person to improve performance offace recognition, Notre Dame Computer Science andEngineering Technical Report (2003).Minolta Inc. Konica Minolta 3D digitizer.<http://www.minoltausa. com/vivid/>.
A.B. Moreno, A ´ ngelSa´nchez, J.F. Ve´lez, F.J.Di´az, Face recognition using 3D surface-extracteddescriptors, in: Irish Machine Vision and ImageProcessing Conference (IMVIP 2003), September 2003.
T. Nagamine, T. Uemura, I. Masuda, 3D facial imageanalysis for human identification, in: InternationalConference on Pattern Recognition (ICPR 1992), 1992,pp. 324–327.
Neven Vision, Inc. Nevenvision machine visiontechnology. <http:// www.nevenvision.com/>.
G. Pan, S. Han, Z. Wu, Y. Wang, 3D face recognitionusing mapped depth images, in: IEEE Workshop on Face Recognition Grand Challenge Experiments, June2005.
G. Pan, Z. Wu, Y. Pan, Automatic 3D face verificationfrom range data, in: International Conference onAcoustics, Speech, and Signal Processing (ICASSP),2003, pp. III:193–196.
T. Papatheodorou, D. Reuckert, Evaluation ofautomatic 4D face recognition using surface and textureregistration, in: Sixth International Conference onAutomated Face and Gesture Recognition, May 2004,pp. 321–326.
G. Passalis, I. Kakadiaris, T. Theoharis, G. Toderici,N. Murtuza, Evaluation of 3D face recognition in thepresence of facial expressions: an annotated deformablemodel approach, in: IEEE Workshop on FaceRecognition Grand Challenge Experiments, June 2005.
[370]P.J. Phillips, P.J. Flynn, T. Scruggs, K.W. Bowyer, J.Chang, K. Hoffman, J. Marques, J. Min, W. Worek,
Overview of the face recognition grand challenge,Computer Vision and Pattern Recognition (CVPR)(2005), pp. I:947–954.
P.J. Phillips, P. Grother, R.J. Michaels, D.M.Blackburn, E. Tabassi, J. Bone, FRVT 2002: overviewand summary. Available at <www.frvt.org/>.
P.J. Phillips, H. Moon, P.J. Rauss, S. Rizvi, TheFERET evaluation methodology for face recognitionalgorithms, IEEE Transactions on Pattern Analysis andMachine Intelligence 22 (10) (2000).
M. Rioux, L. Cournoyer, Nrcc three-dimensionalimage data files, National Research Council of Canada,NRC 29077, June 1988.
T.D. Russ, K.W. Koch, C.Q. Little, 3D facialrecognition: a quantitative analysis, in: 45-th AnnualMeeting of the Institute of Nuclear MaterialsManagement (INMM), July 2004.
T.D. Russ, M.W. Koch, C.Q. Little, A 2D rangeHausdorff approach for 3D face recognition, in: IEEEWorkshop on Face Recognition Grand ChallengeExperiments, June 2005.
Scheenstra, A. Ruifrok, R.C. Veltkamp, A survey of3D face recognition methods, in: InternationalConference on Audio- and Video-based BiometricPerson Authentication (AVBPA 2005), LNCS, vol.3546, July 2005, pp. 891–899.
H.T. Tanaka, M. Ikeda, H. Chiaki, Curvature-basedface surface recognition using spherical correlationprincipal directions for curved object recognition, in:Third International Conference on Automated Face andGesture Recognition, 1998, pp. 372–377.
F. Tsalakanidou, S. Malassiotis, M. Strintzis, Faceauthentication and authentication using color and depthimages, IEEE Transactions on Image Processing 14 (2)(2005) 152–168.
F. Tsalakanidou, S. Malassiotis, M. Strintzis,Integration of 2D and 3D images for enhanced faceauthentication, in: Sixth International Conference onAutomated Face and Gesture Recognition, May 2004,pp. 266–271.
F. Tsalakanidou, D. Tzocaras, M. Strintzis, Use ofdepth and colour eigenfaces for face recognition,Pattern Recognition Letters 24 (2003) 1427–1435.
Y. Wang, C. Chua, Y. Ho, Facial feature detection andface recognition from 2D and 3D images, PatternRecognition Letters 23 (2002) 1191–1202.
Xu, Y. Wang, T. Tan, L. Quan, Automatic 3D facerecognition combining global geometric features with
local shape variation information, in: Sixth InternationalConference on Automated Face and GestureRecognition, May 2004, pp. 308–313.
P. Yan, K.W. Bowyer, Empirical evaluation ofadvanced ear biometrics, in: IEEE Workshop onEmpiricalEvaluation Methods in Computer Vision(EEMCV 2005), June 2005.
P. Yan, K.W. Bowyer, A fast algorithm for ICP-based3D shape biometrics, in: Fourth IEEE Workshop onAutomatic Identification Advanced Technologies(AutoID 2005), October 2005 (to appear).
W. Zhao, R. Chellappa, P. J. Phillips and A. Rosenfeld“Face Recognition: A Literature Survey” ACM
Computing Surveys, Vol. 35, No. 4, December 2003,pp. 399–458.
SushmaJaiswal, Sarita Singh Bhadauria, Rakesh SinghJadon and Tarun Kumar Divakar,Brief description ofimage based 3D face recognition methods ,Volume 1,Number 4, 1-15, DOI: 10.1007/3DRes.04(2010)02.
SushmaJaiswal ,Sarita Singh Bhadauria , RakeshSingh Jadon “Creation 3D Animatable FaceMethodology Using Conic Section-Algorithm”,Information TechnologyJournal,ISSN:18125638,pages no.292-298,2007.
Sushma Jaiswal , Sarita Singh Bhadauria , RakeshSingh Jadon,”Automatic 3D Face Model from 2DImage-Through Projection” Journal: InformationTechnology Journal Year: 2007 Vol: 6 Issue: 7Pages/record No.: 1075-1079.