A Hierarchical Approach to Facial Aging
Amrutha Sethuram Karl Ricanek
Eric Patterson
Face Aging Group, Computer Science Department, UNCW, USA
{sethurama, ricanekk, pattersone}@uncw.edu
http://www.faceaginggroup.com
Abstract
Active Appearance Models (AAMs) have been used as
a promising tool in the field of synthetic age progression.
However, they are yet to be demonstrated on a large hu-
man population with wide variation. This paper presents
a novel AAM-based hierarchical approach to facial aging.
This work is motivated from studies in medical and anthro-
pological literature on classification of human faces based
on gender, ethnic and age groups. The proposed hierar-
chical model approach is a ethnicity and gender specific
aging paradigm. Specifically, the Caucasian (European de-
scent) and African American ethnic groups are considered.
This work will further show that using individual hierarchi-
cal models generate better age-progressed synthetic images
when compared to a general model approach. The results
are evaluated by visual perception of the intended age group
and preservation of identity. Also, a quantitative evaluation
was performed using FaceVACS, a commercial face recog-
nition system, as a surrogate measure. Higher match scores
for synthetic images generated by hierarchical models when
compared to those generated by a general model suggests
the efficiency of the proposed hierarchical model approach.
1. Introduction
The only thing that is constant is change. The changes
that a human face undergoes with age is inevitable. The
effects of aging lead to significant variations in both shape
and texture in a human face. The degree of variations is
influenced by factors such as gender, ethnicity, exposure to
sunlight and effects due to smoking, drinking and weight
loss or gain. Recently, sufficient emphasis has been laid on
this “real-world” problem of aging which can considerably
degrade the performance of facial recognition algorithms.
Robustness of these algorithms to any such temporal vari-
ations is highly desirable for applications in law enforce-
ment, forensics, medicine and other fields.
978-1-4244-7030-3/10/$26.00 ©2010 IEEE
In this work, we define a hierarchical model for id-
iosyncratic facial aging and develop a synthetic facial ag-
ing system based upon hierarchical models. Adult aging
is a distinct process from childhood growth and develop-
ment. While in the formative years, pre-adult process leads
to shape-based changes in the mid and lower face, adult ag-
ing consists of shape deformation from weight change, mi-
nor bone remodeling, and tissue degeneration. Significant
textural changes due to a variety of complex phenomenon
occur due to adult aging [1]. Further, facial aging can be
divided into intrinsic aging and extrinsic aging.Intrinsic ag-
ing, also known as the natural aging process is character-
ized by skin lines, furrows,dryness and laxity of the skin.
Extrinsic aging caused due to external factors such as sun
exposure, repetitive facial expressions, gravity, sleeping po-
sitions, and smoking is marked by fine wrinkles and dyspig-
mentation. There is some evidence in the anthropological
literature that patterns of facial skin aging varies in differ-
ent ethnic groups. The most obvious ethnic skin difference
is related to skin colour which is dominant by the presence
of melanin. The different characteristics of facial skin ag-
ing in Caucasian, African American, Indian, Hispanic and
East Asian women is discussed in [2]. Findings from [25]
suggests that skin wrinkling and sagging skin appears a
decade sooner on Caucasian skin. Further from [25], indi-
cations are that skin wrinkling and sagging caused by pho-
toaging is a predominant problem of lighter skin type than
darker skin. Age related changes in facial wrinkles and sag-
ging in the skin of Japanese, Chinese and Thai women are
compared in [3]. Comparison of the occurrence of age-
associated changes in the facial skin between French and
Japanese women is discussed in [24]. Recently Tsuka-
hara et. al [26] compared the photoaging in Caucasian and
Japanese females living in Cincinnati and Tokyo and con-
cluded that wrinkle and facial sagging scores were higher
in the Caucasian women. However, the more comprehen-
sive data is that of Hillebrand et al. [27] comparing dif-
ferent ethnic groups living in Los Angeles. Findings from
[27] concluded that increased skin wrinkling was observed
100
in the order: Caucasian > Hispanic > African American
> East Asian. Similarly, although the aging processes of
the male and female face share many common features, dis-
crete anatomic differences exist between the male and fe-
male face. In the literature found in plastic surgery, some
suggest the existence of differences in aging due to gender
differences. Aging changes in the male face is discussed in
[4].
This suggests that in general, the aging pattern of an in-
dividual belonging to a specific ethnic group and gender
can be formulated as an intra-class problem. Developing
separate models grouped by ethnicity and gender to fetch
better aging patterns is thus justified. In this work, we
consider a novel hierarchical approach to synthetic facial
aging. Hierarchical models for male and female belong-
ing to Caucasian American and African American ethnic
groups are developed based on Active Appearance Models
(AAM). Our goal is to create synthetically age progressed
images that portray actual aging effects documented in med-
ical and anthropological literature, considering the intrin-
sic aging factors based on the individual ethnic and gender
groups. Also, a general AAM model is developed using all
the images from the hierarchical groups. The age synthe-
sized images from the hierarchical models and the general
model are then visually compared. In addition, FaceVACS
SDK,
c a state of the art face recognition SDK from Cog-
nitec, is used to generate match scores for the two models
to demonstrate the effectiveness of the proposed hierarchi-
cal model approach when compared to the general model.
The remainder of the paper is organized as follows. In
section 2, a brief review of literature concerning aging of the
human face is provided. The hierarchical model approach is
discussed in section 3 with the results presented in section
4. Lastly, we present some concluding remarks with a brief
discussion of ongoing research.
2. Related work in face aging
The computational approaches that have been considered
for simulation of aging can be broadly divided into geomet-
ric or physical based [5] [6]and analysis-synthesis model-
based methods [5] [7] [8] In general, computational meth-
ods take a two-step approach towards facial aging, involv-
ing estimating the current age of the face in an image and
then generating parameters to modify the image to appear
age-progressed by a certain amount of years. Tiddeman et.
al [9] used wavelets to model wrinkles on age-based face
composites. Ramanathan et. al [5] characterized age-based
appearance variations using subspace methods and in [6],
they proposed a two-fold approach towards modeling facial
aging in adults. Park et. al [10] [12] propose an automatic
aging simulation technique by using a 3D morphable model.
Recently, a compositional and a dynamic model for face ag-
ing has been presented in [11]. One of the best approaches
101
demonstrated thus far for synthesizing changes in the face
is the use of AAM for a feature space that takes into account
aging variation. General aging models based on AAMs
[13], [14] have been used for growth and development (pre-
adult aging) in [15], [16] while [17] [18] [19] [23] have
focused on adult aging. A thorough survey of some of the
techniques is presented in [20], although the authors do not
go in much depth on the results generated via AAM-based
methods.
3. Hierarchical Models for synthetic age pro-
gression
Development of accurate, individualized, idiosyncratic
models require a large set of longitudinal data for the same
individual across adult lifespan. However, it has still been a
challenge to gather such data. The data used for developing
the hierarchical models were chosen mainly from Morph
[21] [29] and the PAL Database [30]. A few images from
FG-NET [22] [28] were included in the models for pur-
poses of testing
3.1. Methodology
For our hierarchical model approach, four models were
considered - Caucasian American Male (CM), Caucasian
American Female (CF), African American Male (AM) and
African American Female (AF). For each of these models,
a total of ninety-five quality greyscale images in the (18-65
)year age range, that represent the ethnic and gender group
were selected to train each model. Five images were chosen
in the (18-20) year age group. Ten images were selected
for every succeeding five year age range. Further to model
the shape and texture variation using the AAM model more
efficiently, separate AAM models were built for the younger
(18-45 years) group and the older group(46-65 years) within
each of these hierarchical groups.
The general methodology to develop the models and
generate synthetic aged images is as shown in the flowchart
in Figure 1. The representative images were first hand
annotated using a detailed 161 dense annotation scheme
similar to the one in [21]. For our hierarchical model
approach, individual AAM models were then developed,
based on gender, ethnicity and age-group to generate pa-
rameters for the faces within the group. For example, an
individual African American Female model was built for
the younger age group using only the images in that cat-
egory. Another individual aged-AAM model for African
American Female in the older age group was developed and
so on and so forth. For the general model, all the images
across gender, ethnicity and age-group were pooled together
to develop a single generalized AAM model as shown in the
flowchart. For each of these models, support vector regres-
sion (SVR) was then used to learn age-related properties of
 Figure 1. Methodology to generate synthetic aged images using Hierarchical/General AAM model approach

Figure 2. Our model (last row) compared with the work of Patterson et. al [19] (top three rows). The top two rows are at original age 30
then (40,50,60,70), the third row is a synthetic version aged from 20 to 80. The last row shows aging the same individual at 20, 30, 40, 50,
60 and 65 years using our latest hierarchical model

each of the AAM parameters within the group training set.
Feasible random faces were generated using Monte-Carlo
simulation, classifying each using age-estimation learned
by SVR for each of the groups. Finally, for each of the
groups, a table of representative age parameters, where each
year is a bin represented by the average at that year cre-
ated by the simulation process was generated. This table
was used to index and difference AAM parameters to age-
progress or regress the parameters in the feature space for
synthesizing a new image.
Since individual models were built for the younger and
older subgroups within the hierarchical models, for any
young image that had to be age-progressed to the older
age group, the synthetic image of the source image age-
progressed to the 45th year using the younger model was in-
cluded in the older model before the older model was built.

102
Figure 3. Comparison of the first principal mode of the Hierarchical and General Models. Top left shows variation in the first mode for
each of the individual hierarchical models in the younger age group. Top right shows variation in the first mode for each of the individual
hierarchical models in the older age group. The bottom row shows variation in the first mode for the general model

In addition, to aid in the visual comparison of the synthetic
images generated by the hierarchical and the general model,
few images from the FG-Net database and a few celebrity
images viz. Sean Connery, Bill Clinton, Oprah Winfrey etc
were included in the models. Some of the results are as
shown in the next section.
Figure 2 compares the synthetic images generated with
our hierarchical model with that of work done by Patterson
et. al [19]. It is evident from the figure that our hierarchi-
cal model can generate better synthetic images for the older
decades, in terms of texture and age-representation.
3.2. Results
Synthetic images generated using both the hierarchical
models and the general model are presented in this section.
Figure 3 shows the variation in the first principal mode for
both the hierarchical and the general models. It can be in-
ferred from the figure that individual AAM models devel-
oped based on gender, ethnicity and age-group can capture
the intra-class variation better when compared to the gen-
eral model. Such a specialized individual AAM model can
generate better synthetic images that are representative of
gender, ethnicity and age-group as is evident from Figures
4 - 13. The synthetic images shown in Figures 4 and
6 were generated using the Caucasian American Female
Model. The synthetic images shown in Figures 5 and 7
were generated using the Caucasian American Male Model.
Figure 8 shows the age progressed images of Bill Clinton
and Daniel Craig that were generated by the same model.
The African American Female model was used to generate
synthetic age progressed images of Michelle Obama Figure
10 and Oprah Winfrey Figure 11. Figure 12 which shows
the age progressed images of Sean Connery and figure 9
were both aged using Caucasian American Male model. Fi-
nally, age progressed images of Sidney Poitier were gener-
ated using the African American Male model as shown in
Figure 13.
It should be noted that only the youngest images were
included in the models to generate age-progressed synthetic
images. The older images are included in the figures only
to compare the images generated by the models.
It is evident from Figures 4 - 13 that the hierarchical
models can better model the shape and textural changes of
a specific group when compared to the general model. For
the synthetic aged images generated from the hierarchical
models, identity and intended age group is better preserved
than the general model. Further, in an attempt to quantify
the performance of the hierarchical model and the general

103

Figure 4. Row 1: Original images at 20, 35, 49 and 60 years Row
2: Synthetic images at the same ages generated by CF hierarchical
model Row 3 : Synthetic Images at the same ages generated by
general model
Figure 5. Row 1: Original images at 21, 37, 53 and 63 years Row
2: Synthetic images at the same ages generated by CM hierarchical
model Row 3 : Synthetic Images at the same ages generated by
general Model
model in generating synthetic images of a specific group,
match scores were obtained using the FaceVACS SDK ver-
sion 7.1.0. A higher score generated by the SDK is an in-
dication of a better match. The original young test images
that were used in the model for synthetic age progression
were used as gallery images. The corresponding images
that were reconstructed by the hierarchical models of the
104
Figure 6. Row 1: Original images at 18, 30, 40, 48 and 61 years R
ow 2: Synthetic images at the same ages generated by CF Hierar-
chical Model Row 3 : Synthetic Images at the same ages generated
by General Model
Figure 7. Row 1: Original images at 18, 30, 46 and 54 years Row
2: Synthetic images at the same ages generated by CM Hierarchi-
cal Model Row 3 : Synthetic Images at the same ages generated
by General Model
younger group and the general model were used to construct
the probe. The probe images were then matched against the
parent image in the gallery. A sample of the results are as
shown in Table 1 for various images. It is clear from the
table that synthetic images generated from the hierarchical
models consistently have a better match score than those
generated by the general model. This suggests that indi-
vidual models built specifically for a ethnic, gender and age
group - are better capable of generating synthetic images for
aging a person in accordance with the intra-class variation
of the group.
 SourceImage Hierarchical Model Synthetic Image Score General Model Synthetic Image Score
003A20.jpg 0.997 0.57
004A21.jpg 0.792 0.473
005A18.jpg 0.539 0.363
048A18.jpg 0.999 0.451
Charlie34.jpg 0.999 0.363
MichelleObama28.jpg 0.999 0.998
OprahWinfrey31.jpg 0.999 0.994

Table 1. Match Scores for the synthetic images against the original images using FaceVACS.

Figure 8. Row 1: Original images of Bill Clinton @ 46 years and
Daniel Craig @39 years Row 2: Synthetic images of Clinton at
46, 50, 55, 60 and 65 years generated by CM Hierarchical Model
Row 3 : Synthetic Images of Craig at at 46, 50, 55, 60 and 65 years
generated by Hierarchical Model
Figure 10. Row 1: Original images of Michelle Obama at 28 and
45 years Row 2: Synthetic images at 28, 35, 45, 55 and 65 years
generated by AF Hierarchical Model Row 3 : Synthetic images at
28, 35, 45, 55 and 65 years generated by General Model

4. Conclusion and Future Work

Figure 9. Row 1: Original images of an individual at 34 years Row
2: Synthetic images at 34, 40,50, 60 and 65 years generated by CM
Hierarchical Model Row 3 : Synthetic images at 34, 40,50, 60 and
65 years generated by General Model
In this paper, we have presented a hierarchical model ap-
proach to facial aging. AAM based synthetic models were
developed for the Caucasian American Male, Caucasian
American Female, African American Male and African
American Female populations. In addition, each of these
groups were further divided into younger and older sub-
groups to capture the effects of aging better within the
models. The synthetic age-progressed images generated by
the hierarchical models were then compared with a general
model which was built using all the training images. Vi-
sual comparison of these images suggest that the hierarchi-
cal models could better model the process of aging for a
specific ethnic and gender population. Such a specialized
model can then be used to age progress an individual within
the same group in a more efficient manner accounting for
the intra-class variation within the group. FaceVACS SDK
was used to generate match scores to evaluate the perfor-
mance of the two models. It was shown that match scores
for the original and reconstructed images from the models

105
Figure 11. Row 1: Original images of Oprah Winfrey at 31 and 55
years Row 2: Synthetic images at 31, 45, 55 and 65 years gener-
ated by AF Hierarchical Model Row 3 : Synthetic images at 31,
45, 55 and 65 years generated by General Model
Figure 12. Row 1: Original images of Sean Connery at 29 and 62
years Row 2: Synthetic images at 29, 40, 50 and 62 years gener-
ated by CM Hierarchical Model Row 3 : Synthetic images at 29,
40, 50 and 62 years generated by General Model
were consistently higher for the hierarchical models when
compared to the general model. Our work thus suggests that
specialized hierarchical models trained on a specific ethnic,
gender and age group could generate more accurate syn-
thetic images.
As this is a new area of study, there is much room for
106
Figure 13. Row 1: Original images of Sidney Poitier at 38 and 65
years Row 2: Synthetic images at 20, 38, 50 and 60 years gener-
ated by AM Hierarchical Model Row 3 : Synthetic images at 20,
38, 50 and 60 years generated by General Model
development. Collection of higher quality longitudinal data
for each of the groups will yield improved results. We are
currently working on sensitivity studies to come up with an
optimum annotation map for the images. We are also focus-
ing on expanding this experiment by creating larger crafted
training data for each of the idiosyncratic hierarchical mod-
els.
References
[1] A. M. Albert, K. Ricanek, and E. Patterson. A review of the
literature on the aging adult skull and face: Implications for
forensic science research and applications. Forensic Science
International, In Press.
[2] A. Perkins, A Kimball, G. Hillebrand. K. Miyamot0. The
different characteristics of facial skin aging in Caucasian,
African American, Indian, Hispanic, and East Asian Women.
In Journal of the American Academy of Dermatology, Volume
56, Issue 2, Supplement 2, February 2007, Page AB33
[3] K. Tsukahara, K. Sugata, O. Osanai,A. Ohuchi,Y. Miyauchi,
M. Takizawa, M. Hotta, T. Kitahara. Comparison of age-
related changes in facial wrinkles and sagging in the skin of
Japanese, Chinese and Thai women. In Journal of Dermato-
logical Science, Volume 47, Issue 1, July 2007, Pages 19-28
[4] Paul L. Leong. Aging changes in the male face Facial Plastic
Surgery Clinics of North America, Volume 16, Issue 3, August
2008, Pages 277-279
[5] N. Ramanathan and R. Chellapa. Face verification across age
progression. In IEEE Conference Computer Vision and Pat-
tern Recognition, 2005.
[6] N. Ramanathan and R. Chellapa. Modeling Shape and Textu- [21] K. Ricanek and T. Tesafaye. Morph: A longitudinal image
ral Variations in Aging Faces. In 8th IEEE Int. Conf on Auto- database of normal adult age-progression. In 7th Int. Conf.
matic Face and Gesture Recognition, 2008. on Automatic Face and Gesture Recognition, pages 341-345,
[7] A. Lanitis and C. J. Taylor. Towards automatic face identifica- April 2006
tion robust to ageing variation. In Proceedings of the Fourth [22] Minear M,Park DC. A lifespan database of adult facial stim-
IEEE Int Conf on Automatic Face and Gesture Recognition, uli.Behavior research methods, instruments and computers : a
pages 391-396, 2000 journal of the Psychonomic Society, Inc Vol: 36 Pages: 630-3
[8] K. Ricanek, E. Boone, and E. Patterson. Craniofacial aging [23] E. Patterson, A. Sethuram, M. Albert, K. Ricanek “Com-
impacts on the eigenface face biometric. In Proc. of the Sixth parison of synthetic face aging to age progression by foren-
IASTED Int. Conf. on Visualization, Imaging, and Image Pro- sic sketch artist” Proceedings of the seventh IASTED Interna-
cessing, pages 249-253, Palma de Mallorca, Spain, Aug 2006 tional Conference on Visualization, Imaging, and Image Pro-
[9] B. Tiddeman, D.M Burt, and D.Perret. Prototyping and trans- cessing, 2007.
forming facial texture for perception research. Computer [24] E Tschachler, F. Morizot “Ethnic Differences in Skin Aging”
Graphics and Applications, IEEE, 21(5): 42-50, July-August Pages 23-31 in the book ‘ ‘Skin aging” B. Gilchrest and J.
2001 Krutmann, Springer 2006.
[10] Unsang Park, Yiying Tong, Anil K Jain.Face Recognition [25] A.V. Rawlings. Ethnic skin types: are there differences in
with Temporal Invariance: A 3D Aging Model. In 8th IEEE skin structure and function? International Journal of Cosmetic
Int. Conf. on Automatic Face and Gesture Recognition, 2008. Science, 2006, 28, 79-93
[11] J. Suo, S Zhu, S Shan.A compositional and dynamic model [26] Tsukahara T, Fujimara T, Yoshida Y et al. Comparison of age
for face aging. In IEEE Transactions on Pattern Analysis and related changes in wrinkling and sagging of the skin in Cau-
Machine Intelligence, Vol 32, No 3, March 2010. casian females and in Japanese females. International Journal
[12] Unsang Park, Yiying Tong, Anil K Jain.Age Invariant Face of Cosmetic Science, 2004, 55, 373-388
Recognition. To appear In IEEE Transactions on Pattern Anal- [27] Hillebrand G.G, Levine M.J, and Miyamoto K. The age
ysis and Machine Intelligence, 2010. dependent changes in skin condition in African Americans,
[13] A. Lanitis, C. J. Taylor, and T. F. Cootes. Automatic iden- Asian Indians, caucasians, East Asians and Latinos. IFSCC
tification and coding of human faces using flexible models. Magazine. 2001, 4, 259-266
IEEE Transactions on Pattern Analysis and Machine Intelli- [28] FG-NET Aging Database, http://www.fgnet.rsunit.com
gence, 19(7), July 1997.
[29] Face Aging Group, http://www.faceaginggroup.com
[14] G. J. Edwards, C. J. Taylor, and T. F. Cootes. Interpreting
face images using active appearance models. In Proceedings [30] PAL Face Database, https://pal.utdallas.edu/facedb/
of the Fifth European Conference on Computer Vision, pages
581-595, 1998
[15] A. Lanitis, C. J. Taylor, and T. F. Cootes. Simulating aging on
face images. In Proceedings of the Second Int. Conf. on Audio
and Video-based Biometric Person Authentication, 1999.
[16] A. Lanitis, C. J. Taylor, and T. F. Cootes. Toward automatic
simulation of aging effects on face images. IEEE Transactions
on Pattern Analysis and Machine Intelligence, 24(4), April
2002.
[17] E. Patterson, K. Ricanek, A. M. Albert, and E. Boone. Auto-
matic representation of adult aging in facial images. In Proc.
of the Sixth IASTED Int. Con. on Visualization, Imaging, and
Image Processing, Palma de Mallorca, Spain, Aug 2006.
[18] E. Patterson, A. Sethuram, Albert. M, K. Ricanek, M. King.
Aspects of Age Variation in Facial Morphology Affecting
Biometrics.1st IEEE Int. Conf. on Biometrics: Theory, Ap-
plications and Systems, 2007. pages 1-6, 2007
[19] E. Patterson, A. Sethuram, K. Ricanek, F. Bingham. Im-
provements in Active Appearance Model Based Synthetic
Age Progression for Adult Aging.3rd IEEE Int. Conf. on Bio-
metrics: Theory, Applications and Systems, 2009.
[20] N. Ramanathan, R. Chellappa and S. Biswas. Computational
methods for modeling facial aging: A survey.Journal of Visual
Languages and Computing, 2009.

107