IJDAR (2017) 20:59–68
DOI 10.1007/s10032-016-0277-z
ORIGINAL PAPER
Chinese calligraphic style representation for recognition
Gao Pengcheng1 · Gu Gang1 · Wu Jiangqin1 · Wei Baogang1
Received: 29 July 2015 / Revised: 28 January 2016 / Accepted: 1 December 2016 / Published online: 18 January 2017
© Springer-Verlag Berlin Heidelberg 2017
Abstract Chinese calligraphy draws a lot of attention for
its beauty and elegance. The various styles of calligraphic
characters make calligraphy even more charming. But it
is not always easy to recognize the calligraphic style cor-
rectly, especially for beginners. In this paper, an automatic
character styles representation for recognition method is pro-
posed. Three kinds of features are extracted to represent
the calligraphic characters. Two of them are typical hand-
designed features: the global feature, GIST and the local
feature, scale invariant feature transform. The left one is
deep feature which is extracted by a deep convolutional neu-
ral network (CNN). The state-of-the-art classifier modified
quadratic discriminant function was employed to perform
recognition. We evaluated our method on two calligraphic
character datasets, the unconstraint real-world calligraphic
character dataset (CCD) and SCL (the standard calligraphic
character library). And we also compare MQDF with other
two classifiers, support vector machine and neural network,
to perform recognition. In our experiments, all three kinds of
feature are evaluated with all three classifiers, respectively,
finding that deep feature is the best feature for calligraphic
style recognition. We also fine-tune the deep CNN (alex-
net) in Krizhevsky et al. (Advances in Neural Information
Processing Systems, pp. 1097–1105, 2012) to perform cal-
B Wu Jiangqin
wujq@zju.edu.cn
Gao Pengcheng
gaopengcheng@zju.edu.cn
Gu Gang
gugang@zju.edu.cn
Wei Baogang
wbg@zju.edu.cn
1 College of Computer Science, Zhejiang University,
Hangzhou, China
ligraphic style recognition. It turns out our method achieves
about equal accuracy comparing with the fine-tuned alex-net
but with much less training time. Furthermore, the algorithm
style discrimination evaluation is developed to evaluate the
discriminative style quantitatively.
Keywords Calligraphy · Style recognition · Global feature ·
Local feature · Deep feature
1 Introduction
Chinese calligraphy is one of the finest Chinese art forms
and the most important and inseparable part of Chinese his-
tory. Its delicate esthetic effects are generally considered to be
unique among all calligraphic arts. In the last decade, numer-
ous collections of historical Chinese calligraphic works were
digitized and stored, great progress has been made in the
area of computer aided calligraphy research, and several
novel approaches have been proposed to address calligraphic
problems, including calligraphy processing and analysis [2],
calligraphic character retrieval [3,4] and recognition [5–7],
visualization [8] and specific style rendering [9]. Automatic
Chinese calligraphic character generating method [10,11]
and calligraphy beautification method [12] are also devel-
oped.
Lots of people are obsessed by the beauty of calligraphy.
The various styles of calligraphic characters make calligra-
phy even more charming. For the same character, there are
always several ways of writing, each with special taste, as
shown in Fig. 1. But how to describe the styles of calligraphic
character? Generally the style of Chinese calligraphy char-
acters mainly depends on the posture of the strokes and the
structure of the characters. The width of characters in differ-
ent styles is also different. As we can see in Fig. 1, the stokes
123
60
Fig. 1 Five main styles of the same character. From left to right: seal
script, clerical script, standard script, semi-cursive script, and cursive
script
of seal script character are much thinner than the strokes of
clerical script character. For those who are familiar with Chi-
nese calligraphy, they can easily tell the differences between
different styles. But for the beginner, it is not always the
same case. In CADAL,1 which is the biggest digital library
in China, plenty of digital calligraphic character images are
collected. Due to the large scale of calligraphic character
images, automatic method which is used to label all these
calligraphic images is needed to provide better service to
users. But how to represent the style of the calligraphic char-
acters using computer is still a challenging work.
In this paper, different kinds of feature will be evaluated
quantitatively for the representation of calligraphic style.
Qualitative visualization will be performed to show how
efficient it could be to represent calligraphic style using
different kinds of feature. Furthermore, an automatic recog-
nition method will be used to perform style recognition on
different kinds of features.
The remainder of this paper is organized as follows: A
brief related work is summarized in Sect. 2. Section 3 gives
the detail of feature extraction. Section 4 describes the clas-
sifiers used in this paper. Experimental results and analysis
are given in Sect. 5. And in Sect. 6, we give our conclusion.
2 Related work
For the printed characters, character style is more like the
character font. Font recognition is a fundamental problem
in document analysis and recognition and closely relates
with optical character recognition (OCR) techniques. For
the printed character, the first step is to classify the font of
the character, then recognize the character in the single font
character library. The recognition method reduces the influ-
ence of alternative shapes for each class of font. Besides,
font recognition can be used to achieve automatic typeset-
ting. In [13,14], character feature basing on texture analysis
and wavelet transform was extracted to perform font recogni-
tion. In [15] sparse discriminative information preservation
(SDIP) was proposed select printed character features for
further font recognition. Furthermore, recurrent neural net-
work was introduced to solve the font recognition with noisy
1 China Academic Digital Associate Library - http://www.cadal.zju.
edu.cn.
123
G. Pengcheng et al.
data in [16]. All these methods focused on printed char-
acters, which are usually uniform. But for the handwritten
calligraphic characters, strokes are usually changeful which
results in great challenge in style recognition.
It is very natural that we extract some features from the
calligraphic characters. There are mainly two categories of
hand-engineered feature for image representation: global fea-
ture and local feature. One popular global feature is Gabor
feature. Zhuang et al.proposed the latent style model to dis-
cover writing styles for calligraphic works [17]. In their
approach, they used 2D Gabor filter to extract texture fea-
ture and verified its discriminability for calligraphy writing
style classification, the recognition ratio is more than 56 %
for each style. They also used Gabor feature to classify
the writing style of Chinese words in [18]. It proves that
Gabor feature is feasible for discriminating different cal-
ligraphy writing styles, but the performance still needs to
be improved. Another global feature called GIST descriptor
which was originally proposed in [19] has recently shown
good results for calligraphic character image searching [6].
It does not require any form of segmentation and use low
dimension to represent the image. Besides the global fea-
ture, local feature is also adopted in many applications. One
of the most famous local feature is scale invariant feature
transform (SIFT) descriptor. SIFT descriptor was originally
developed by David Lowe [20], and it was successfully used
in recognition, stitching and many other applications because
of its robustness. Another local feature called PCA-SIFT
[21] which was proposed by Yan Ke used principal compo-
nent analysis (PCA) to normalize the gradient patch instead
of histogram. Herbert [22] presented a faster method called
SURF, which used Fast-Hessian detector. Rublee [23] also
proposed the ORB feature based on the well-known FAST
keypoint detector [24] and the BRIEF descriptor [25]. All
these local feature extraction methods are usually comprised
of two stages. In the first stage, detection of interesting salient
points is performed. In the second stage, descriptors are com-
puted to characterize the appearance of the local regions
surrounding these salient points. For the calligraphic char-
acter images, we wish to find some keypoints that contain
the information of style.
It is always a challenging work to discover effective
representations that capture salient semantics for percep-
tual learning. The conventional visual representation, based
on flat feature representations involving quantized gradi-
ent filter, has achieved impressive performance but has
likely plateaued in recent years. Besides all the above hand-
engineered representations, there are also some other features
generated by deep model. It has long been argued that deep or
layered compositional architectures should be able to capture
salient aspects of given domain [26,27]. The deep convolu-
tional neural networks (CNN) proposed in [1] have achieved
competition-winning numbers on large benchmark datasets
Chinese calligraphic style representation for recognition
consisting of more than one million images. But in some
other tasks, the applicability of deep CNN is greatly lim-
ited because of the difficulty of collecting large-scale image
datasets. To address this problem, [28] proposed to transfer
image representations learned with CNNs on large datasets
to other tasks with limited training data.
In this paper, we chose three kinds of feature to repre-
sent the style of the calligraphic character. The typical global
feature, GIST descriptor and the typical local feature, SIFT
descriptor are evaluated. The last kind of feature is extracted
by deep network, which is called deep feature in this paper.
The deep feature was generated by model used in [27]. With
all these three features extracted, there are at least two impor-
tant questions remain:
• Which feature could capture the calligraphic styles infor-
mation better? The hand-designed features or the deep
feature?
• How the calligraphic styles are related to each other? Is
there a clear distinction between different styles?
We address these questions both qualitatively and quanti-
tatively, via visualizations of different kinds of feature, and
experimental comparison to current state-of-the-art method
in the following sections.
3 Style representation
In this section, three kinds of features are extracted to cap-
ture the calligraphic style information from the character
images, in which some hidden patterns of style information
are expected. Performance based on these features will be
evaluated in the following sections.
3.1 Global Feature
As the typical global feature, GIST, which was initially
proposed in [19] is widely applied in image representa-
tion [6,29,30]. GIST descriptor gives a whole view of the
character image. Because GIST includes all levels of visual
information-ranging from low-level feature (e.g., contours)
to intermediate (e.g., shapes) and high-level information
(e.g., activation of sematic knowledge), it can be represented
at both perceptual and conceptual levels. Perceptual GIST
refers to the structural representation of the calligraphic
character built during perception. Conceptual GIST includes
the semantic information that is inferred while viewing a
calligraphic character or shortly after the character has disap-
peared from view. Conceptual GIST is enriched and modified
as the perceptual information bubbles up from early stages
of visual processing. The calligraphic style information may
be captured by GIST descriptor.
61
Fig. 2 Calligraphic character images and their corresponding GIST
descriptors
Fig. 3 Calligraphic character image and its corresponding SIFT key-
points descriptors
For the global feature, GIST descriptor extraction, the
image is decomposed by a bank of multi-scale oriented filters.
Here in this paper, we use 8 orientations and 3 scales. Next,
the output magnitude of each filter is averaged over 9 non-
overlapping windows arranged on a 3 × 3 grid. So the final
character image representation is a 3 × 8 × 9 = 216 dimen-
sional vector, which can be seen as a point p in d (d = 216).
Figure 2 shows two binary calligraphic character images and
their corresponding GIST descriptors.
3.2 Local feature
As the typical local feature, SIFT descriptor is successfully
used in recognition, stitching and many other applications
because of its robustness. Like many other local feature
extraction, SIFT descriptor extraction also includes two
stages. In the first stage, detection of interesting salient points
is performed. In the second stage, descriptors are computed to
characterize the appearance of the local regions surrounding
these salient points. For the calligraphic character images,
we wish to find some keypoints that contain the information
of style and then to extract features from the region around
the keypoints.
For the local feature, SIFT descriptor extraction, keypoints
are detected first. Then the gradient magnitude and orienta-
tion at each character image sample point in a region (16×16
window around keypoint) around the keypoint location are
computed and weighted by a Gaussian window. We use 4 × 4
descriptors computed from a 16 × 16 sample window, each
descriptor is accumulated in 8 orientations by summarizing
the contents over 4 × 4 subregions. So for each keypoint of
the character image, the dimension of the keypoint descrip-
tor is 4 × 4 × 8 = 128. Figure 3 gives an example of SIFT
keypoints descriptor. Among all these keypoints descriptors,
style information may be captured by the local features.
123
62 G. Pengcheng et al.

ability, statistical methods based on the Bayes decision rule

Fig. 4 Visualization of mid-layer feature, from left to right original
character image; first convolutional layer output (9 channels of 96);
fifth convolutional layer output (9 channels of 256); fifth layer output
after ReLU (9 channels of 256)
3.3 Deep feature
are widely used in pattern recognition problems. The mod-
ified quadratic discriminant function (MQDF) proposed by
Kimura [33] has been proved to be a state-of-the-art classi-
fier for handwriting characters recognition. Inspired by this,
we apply the MQDF on the recognition of calligraphic styles
for the first time.
According to Bayes decision theory, by assuming the
probability density function of each class to be multivari-
ate Gaussian and equal a priori probabilities, the quadratic
discriminant function (QDF) can be obtained as:

As the typical perceptual learning, it has long been argued FQDF (x, i) = (x − μi ) Σi−1 (x − μi ) + log |Σi | (1)
that deep or layered compositional architectures should be
able to capture salient aspects of given domain [26,27]. where μi ∈ d and Σi ∈ d×d denote the mean vector
Until recent year, the use of deep convolutional neural net- and the covariance matrix of class i. The QDF can be used
work (CNN) for image classification [1] and object detection as a distance metric in the sense that the class of minimum
[31,32] has led to significant gains in accuracy. These deep distance is assigned to the input pattern. The MQDF makes a
CNN models usually contain several layers including con- modification to the QDF by K-L transform and smoothing the
volutional layers and fully connected layers. Layer by layer, minor eigenvalues to improve the computation efficiency and
feature transfers nonlinearly from the bottom level to the classification performance. By K-L transform, the covariance
upper sematic level. We wish to use these deep CNNs to matrix Σi can be diagonalized as
capture salient semantics which may discover effective cal-
ligraphic style representations. Σi = Φi Λi Φi (2)
For the deep feature extraction, we adopted the DeCAF
[27] framework and employed the deep convolutional neu- where Λi = diag[λi1 , . . . , λid ] with λi j , j = 1, . . . , d
ral network architecture proposed in [1]. This network is being the eigenvalues (ordered in non-ascending order) of
composed of five successive convolutional layers C1…C5 Σi , and Φi = [φi1 , . . . , φid ] with φi j , j = 1, . . . , d being
followed by three fully connected layers FC6 …FC8. The the corresponding eigenvectors. Φi is orthonormal such that
three fully connected layers are computed by Y6 = σ (W6 Y5 + Φi Φi = I .
B6 ), Y7 = σ (W7 Y6 + B7 ) and Y8 = ψ(W8 Y7 + B8 ) where Yk According to Eq. 1, the QDF can be rewritten in the form
denotes the output of the k-th layer, Wk , Bk are the trainable of eigenvectors and eigenvalues:
parameters of the k-th layer, and σ (X )[i] = max(0, X [i])
 
d 
d
and ψ(X )[i] = e X [i] / j e X [ j] are the “ReLU” and “Soft- FQDF (x, i) =
1 
[φi j (x − μi )]2 + log λi j (3)
Max” nonlinear activation functions. There are more details λi j
j=1 j=1
of the architecture of CNN in [1]. We reuse layers trained
on the ImageNet dataset as the pre-trained parameters to By replacing the minor eigenvalues λi j ( j > k) with a
compute mid-level image representation. As the experiments constant δi to stabilize the generalization performance, the
shown in [27], the deep feature generated at layer FC6 per- MQDF can be obtained as
forms better than other layers’ feature, and here we choose
the layer FC6 feature as our deep feature, which has 4096 
k
1 1
dimensions. The dimensions here is much higher than the FMQDF (x, i) = ( − )[φij (x − μi )]2
λi j δi
j=1
GIST and SIFT descriptor. In our experiments, we reduce
the deep feature to 216 dimensions same as GIST descrip- 1
+ x − μi 22
tor using principal component analysis (PCA). Figure 4 gives δi
some visualization of mid-layer feature.  k
+ log λi j + (d − k) log δi (4)
j=1

4 Style recognition and discrimination evaluation where k denotes the number of principal axes. The decision
rule of MQDF for the input pattern x is
After extracting the features of character images, the next
step of style recognition is classification. By assigning the M
input sample to the category with maximum posterior prob- x ∈ class arg min FMQDF (x, i) (5)
i=1

123
Chinese calligraphic style representation for recognition
In our implementation of MQDF, we use a unified minor
eigenvalue δi = δ0 for all classes and optimize this parameter
by cross-validation on the training data. Inspired by [34], we
β  M d
set δ0 as Md i=1 j=1 λ j with β selected from [0,1].
It is also important to find out how the different styles
related to each other. Algorithm 1 shows the method of style
discrimination evaluation (SDE), which is developed to eval-
uate the discriminative styles based on the MQDF classifier.
Here we discard the wrong test cases. The distance of differ-
ent styles is defined by penalty of the MQDF rank results.
Algorithm 1 Style Discrimination Evaluation (SDE)
Input: trained MQDF classifiers on different styles, MQDFseal for the
seal scri pt, MQDFclerical for the clerical scri pt, MQDFstandar d
for the standar d scri pt, MQDFsemi for the semi −cur sive scri pt
and MQDFcur sive for the cur sive scri pt; labeled data xi (i =
1, . . . , n) with attribute st yle;
1: initiate S = {seal, clerical, standar d, semi, cur sive};
2: initiate dist (st yle1 , st yle2 ) = 0, (st yle1 , st yle2 ∈ S);
3: initiate count (st yle1 , st yle2 ) = 0, (st yle1 , st yle2 ∈ S);
4: for i from 1 to n do
5: for j in S do
6: if xi correctly classified then
7: dist (xi .st yle, j)+ = Penalt y(FM Q D F j (xi , j));
8: count (xi .st yle, j)+ = 1;
9: end if
10: end for
11: end for
12: for st yle1 , st yle2 in S do
dist (st yle1 ,st yle2 )+dist (st yle2 ,st yle1 )
13: Distance(st yle1 , st yle2 ) = count (st yle1 ,st yle2 )+count (st yle2 ,st yle1 ) ;
14: end for
Output: Distance(st yle1 , st yle2 ), (st yle1 , st yle2 ∈ S).
5 Experiments and analysis
5.1 Data preparation
In CADAL, numerous collections of historical Chinese cal-
ligraphic works are digitized and stored. A big database
Calligraphic Character Dictionary (CCD) is built, which
contains calligraphic character images (more than 110,000
character images, 8035 seal script, 12,193 clerical script,
49,481 standard script, 30,915 semi-cursive scrip and 10,113
cursive script.) labeled with semantic meaning by calli-
graphic experts. We evaluated the feature performance both
on the unconstrained Chinese calligraphic character dataset
CCD and the Standard Character Library (SCL, contains
more than 18,770 character images, more than 3800 character
images for each style), which contains five different styles of
calligraphic characters, named as seal script, clerical script,
standard script, semi-cursive script and cursive script. Fig-
ure 5 is the example from CCD and SCL. The main difference
between SCL and CCD is that the calligraphic characters in
63
SCL are uniformed while those in CCD are unconstrained,
written by real people. Both datasets can be downloaded from
CADAL calligraphy system2
In our experiments, following questions were addressed:
• Which feature could capture the calligraphic styles infor-
mation better? The hand-designed features or the deep
feature?
• How the calligraphic styles are related to each other? Are
there any clear differences between different styles?
We first visualized all three kinds feature to find their
sematic cluster and then evaluated the performance of these
different features on SCL and CCD, respectively. We also
evaluated the performance of different classifiers in task of
style recognition using deep feature. At last, the relations of
different styles are also evaluated.
5.2 Feature visualization and performance evaluation
For all the characters in SCL and CCD, SIFT descriptor, GIST
descriptor and deep feature were extracted. We evaluated the
three kinds of feature performance both on the CCD and the
SCL character library. t-SNE [35] was employed to visual-
ize all these three kinds of feature. By projecting these high
dimensional data to 2 dimensions, we can clearly discover
some patterns. Figure 6 gives the visualization of the SIFT
descriptors. There are barely clear differences between dif-
ferent calligraphic styles, no matter on CCD or SCL. There is
a better situation for GIST descriptor, as shown in Fig. 6, but
still different styles can be easily confused. Compared with
SIFT and GIST descriptors, deep feature captures much more
significant sematic patterns in calligraphic styles, as shown
in Fig. 6. By visualizing all these three kinds of feature, we
can find that the deep feature is much more suitable for cal-
ligraphic style recognition.
We also employed a classifier to evaluate the performance
of the three kinds of feature. The MQDF classifier was
employed to perform style recognition. For the SCL char-
acter dataset, we chose 3400 examples of each calligraphic
style to train the MQDF classifier, and another 450 exam-
ples of each style to test. For the real-world data, we chose
4000 examples of each calligraphic style from CCD for train-
ing and another 1000 examples of each style to test. All the
test and training examples are randomly chosen. Recognition
performance of different features on SCL and CCD is shown
in Tables 1 and 2, respectively. It is important to note that
for the style recognition using SIFT feature, we predict the
label of every point and its corresponding feature. Then for
2CADAL calligraphy system - http://www.cadal.zju.edu.cn/
NewCalligraphy.
123
64
Fig. 5 Calligraphic character examples from (a) CCD (b) SCL. From top line to bottom line cursive, clerical, seal, semi-cursive and standard
the character style prediction, we vote the style of key points
with maximum number as the final character style.
As we can see in Table 1, deep feature performs best on
SCL. For the semi-cursive and clerical script. the recogni-
tion rate is 100%. The recognition rate of the other three
style is almost 100%. The local feature, SIFT descriptor, is
not stable for the style recognition task. For the semi-cursive
and clerical scripts, the performance of SIFT descriptor is
much lower than the deep feature. The global feature, GIST
descriptor also achieved promising performance but still got
slightly lower recognition rate than deep feature. Table 2
gives the performance of different features on CCD. The
performance of deep feature is slightly dropped compar-
ing to SCL but still performs best. This is mainly because
of the complication of unconstraint real-world calligraphic
characters.
We also evaluated the performance of different classi-
fiers on the deep feature for calligraphic style recognition.
We employed the LibSVM [36] using radial basis function
(RBF) kernel to perform calligraphic style recognition. We
selected the optimal value of γ in RBF kernel and penalty
parameter C after three times of fivefold cross-validation
on training data. In our experiments, we set γ = 0.07 in
RBF kernel and penalty parameter C = 1. We also imple-
mented neural networks basing on the MATLAB toolkit
with one hidden layer to perform recognition. The net-
works were constructed by three layers: the input layer (216
units), the hidden layer (different number of hidden units
were tested including 5, 10, 15, 20, 25, 30, 35, 40 and 60
units) and the output layer (3 output units, using binary
123
G. Pengcheng et al.
codes to represent 5 different calligraphic styles). Neural
networks with different number of hidden units were imple-
mented, and the one with best performance (with 10 hidden
units) was selected to perform calligraphic style recogni-
tion. As we can see in Tables 3 and 4, the three classifiers
achieved quite close performance using the deep feature on
SCL. MQDF performs slightly better than the other two
classifiers on average recognition rate. This shows that the
deep feature captured the salient sematic style information
of SCL quite well. The recognition accuracy dropped on
CCD. This may be caused by the variety of writing style
in CCD.
In our experiments, we also fine-tuned alex-net [1] to
perform calligraphic style recognition on SCL and CCD,
respectively. Alex-net is a fully supervised trained deep CNN
and won the image classification challenge on ImageNet in
2012. In our fine-tuned alex-net, we keep the architecture
same as the original network except the last output layer.
The last layer of the original network which has 1000 output
units was replaced by 5 softmax output units corresponding
to five different calligraphic styles. Parameters of the top 7
layers were initialized with the original ones and the layer
and the last softmax layer’s parameters were randomly cho-
sen. During the fine-tuning process, we set the learning rate
of the top 7 layers smaller than the last layer to make the
old layers change very slowly with new data, but let the new
last layer learn fast. We fine-tuned the alex-net on SCL more
than 20000 iterations and on CCD more than 50000 itera-
tions. It took about 20 seconds every 20 iterations during
fine-tuning using GPU (GPU Configuration: GeForce GTX
Chinese calligraphic style representation for recognition 65

150 150
cursive cursive
semi−cursive semi−cursive
seal seal
100 clerical 100 clerical
standard standard

50 50

0 0

−50 −50

−100 −100

−150 −150
−100 −50 0 50 100 150 −100 −50 0 50 100 150
(a)

150 150
cursive cursive
semi−cursive semi−cursive
seal seal
100 clerical 100 clerical
standard standard

50 50

0 0

−50 −50

−100 −100

−150 −150
−150 −100 −50 0 50 100 150 −150 −100 −50 0 50 100 150
(b)

150 150
cursive cursive
semi−cursive semi−cursive
seal seal
100 clerical 100 clerical
standard standard

50 50

0 0

−50 −50

−100 −100

−150 −150
−150 −100 −50 0 50 100 150 −100 −50 0 50 100 150
(c)

Fig. 6 Feature visualization on CCD (left) and SCL (right) a Feature visualization of SIFT descriptor. b Feature visualization of GIST descriptor.
c Feature visualization of deep feature

123
66 G. Pengcheng et al.

Table 1 Performance of different features on SCL using MQDF Table 4 Performance of different classifiers using deep feature on CCD
Recognition rate(%)
Recognition rate(%)
SIFT GIST Deep feature
SVM Neural network MQDF
Standard script 98.3 97.0 99.6
Standard script 99.4 99.1 99.7
Semi-cursive script 69.2 96.5 100
Semi-cursive script 92.7 86.6 97.2
Seal script 97.5 97.3 99.7
Seal script 83.6 83.4 89.7
Clerical script 61.6 95.8 100
Clerical script 98.7 94.1 97.4
Cursive script 95.4 94.3 99.6
Cursive script 89.3 85.8 87.1
Average 84.40 96.18 99.78
Average 92.74 89.80 94.22
Bold numbers are shown to emphasize the best recognition accuracy
compared with other methods Bold numbers are shown to emphasize the best recognition accuracy
compared with other methods

Table 5 Recognition performance of MQDF with deep feature (DF)

Table 2 Performance of different features on CCD using MQDF
and fine-tuned alex-net on SCL
Recognition rate(%) Recognition rate(%)
SIFT GIST Deep feature DF+MQDF Fine-tuned alex-net
Standard script 91.8 94.2 99.7 Standard script 99.6 99.9
Semi-cursive script 26.7 83.1 97.2 Semi-cursive script 100 99.5
Seal script 79.1 89.3 89.7 Seal script 99.7 99.8
Clerical script 85.8 91.9 97.4 Clerical script 100 99.8
Cursive script 77.9 84.2 87.1 Cursive script 99.6 100
Average 72.26 88.54 94.22 Average 99.78 99.80
Bold numbers are shown to emphasize the best recognition accuracy Bold numbers are shown to emphasize the best recognition accuracy
compared with other methods compared with other methods

Table 6 Recognition performance of MQDF with deep feature (DF)

Table 3 Performance of different classifiers using deep feature on SCL and fine-tuned alex-net on CCD
Recognition rate(%)
Recognition rate(%) DF+MQDF Fine-tuned alex-net
SVM Neural network MQDF
Standard script 99.7 94.2
Standard script 99.3 97.3 99.6 Semi-cursive script 97.2 92.2
Semi-cursive script 96.6 97.1 100 Seal script 89.7 98.0
Seal script 100 99.3 99.7 Clerical script 97.4 97.0
Clerical script 99.3 99.2 100 Cursive script 87.1 93.6
Cursive script 95.6 99.1 99.6 Average 94.22 95.00
Average 98.16 98.40 99.78
Bold numbers are shown to emphasize the best recognition accuracy
Bold numbers are shown to emphasize the best recognition accuracy compared with other methods
compared with other methods

5.3 Styles discrimination evaluation

TITAN). Tables 5 and 6 give the performance the fine-tuned
alex-net on task of calligraphic style recognition. We can
see that our method achieved very close performance com-
paring with the fine-tuned alex-net on SCL. And on CCD,
fine-tuned alex-net achieved slightly better performance than
our method, too. It is deserved to be noticed that our method
was quite flexible and much easier to implement while the
alex-net cost dozens of hours to deploy and fine-tune on the
new data.
After addressing the question of which kind of feature and
classifier are most suitable for calligraphic character style
representation and recognition, another question is popping
out: how the different calligraphic style related to each other.
SDE (Algorithm 1) was employed to calculate the average
distance between different styles. Table 7 gives the average
distance of different styles on SCL and CCD, respectively.
All these distances are normalized by the penalty of rank
result. Two different styles with smaller distance are much

123
Chinese calligraphic style representation for recognition 67

Table 7 Distance of different

Standard Semi-cursive Seal Clerical Cursive
style on SCL (top-right triangle)
and CCD (bottom-left triangle) Standard – 2.27 2.35 3.66 1.9
respectively, smaller number
means more similar between Semi-cursive 3.3 – 3.27 2.36 1.04
two styles Seal 2.31 2.6 – 2.26 2.7
Clerical 3.42 1.4 2.5 – 3.22
Cursive 3.76 1.36 1.52 2.73 –
Bold number shows the smallest distance on different style, which means that the two styles are more similar
than other styles

150 150
cursive cursive
semi−cursive semi−cursive

100 100

50 50

0 0

−50 −50

−100 −100

−150 −150
−150 −100 −50 0 50 100 150 −100 −50 0 50 100 150

Fig. 7 Overlapping of cursive and semi-cursive on CCD (left) and SCL (right)

more similar to the pair with larger distance. As we can see
in Table 7, the top-right triangle area is the distance on SCL
and the bottom-left area is the distance on CCD. We can see
from the table that the cursive and semi-cursive script are
most similar both on SCL and CCD. This could be easily
justified in Fig. 7. In Fig. 7, we only visualized the cursive
script and semi-cursive script using deep feature. We can
see that no matter on CCD or SCL, there is a big overlap
between the two scripts.
5.4 Analysis and discussion
From the experiments above we can see that no matter on
CCD or SCL, the deep feature captures more style informa-
tion than SIFT descriptor and GIST descriptor. This can be
directly concluded from the visualization of the three kinds
of feature. But still the performance on CCD is a little lower
than the performance on SCL. This is mainly because the
characters of the same style in SCL are uniformed. But for
CCD, real-world characters are usually unconstrained. The
calligraphic character of the same style in CCD may be writ-
ten by different people. So there are still slight differences
for the characters of the same style in CCD.
6 Conclusion
In this paper, an automatic calligraphic style representa-
tion for recognition method was proposed. Three kinds of
features, the typical global feature GIST descriptor, the typ-
ical local feature SIFT descriptor and the deep feature,
were extracted. Three kinds of classifiers: MQDF, SVM
and NN are employed to perform recognition. We evalu-
ated our method on two calligraphic character datasets, CCD
(the unconstraint real-world calligraphic character dataset)
and SCL(the standard calligraphic character library). Our
experiments show that the deep feature outperforms the
other two kinds of features on calligraphic style recogni-
tion task by employing MQDF, achieving 99.78 % average
recognition accuracy on SCL and slightly lower recogni-
tion accuracy(average 94.22 %) on CCD. We also fine-tune
alex-net to perform calligraphic style recognition. It turns
out our method achieves about equal accuracy comparing
with the fine-tuned alex-net but with much less training time.
Furthermore, algorithm SDE was employed to evaluate the
relations between different styles, finding that the cursive
and semi-cursive scripts are most similar both on SCL and
CCD.

123
68
Acknowledgements This work is supported by National Natural Sci-
ence Foundation of China (No.61379073) and the CADAL Project and
Research Center, Zhejiang University.
References
1. Krizhevsky, A., Sutskever, I., Hinton, G.E.: Imagenet classification
with deep convolutional neural networks. In: Advances in Neural
Information Processing Systems, pp. 1097–1105 (2012)
2. Wang, S., Lee, H.: Dual-binarization and anisotropic diffusion of
chinese characters in calligraphy documents. In: Proceedings. Sixth
International Conference on Document Analysis and Recognition,
IEEE, pp. 271–275 (2001)
3. Zhuang, Y., Zhang, X., Wu, J., Lu, X.: Retrieval of chinese calli-
graphic character image. In: Advances in Multimedia Information
Processing-PCM 2004, pp. 17–24. Springer (2005)
4. Pengcheng, G., Jiangqin, W., Yuan, L., Yang, X., Tianjiao, M., Bao-
gang, W.: Fast image-based chinese calligraphic character retrieval
on large scale data. In: 2014 IEEE/ACM Joint Conference on Dig-
ital Libraries (JCDL), IEEE, pp. 211–220 (2014)
5. Yu, K., Wu, J., Zhuang, Y.: Skeleton-based recognition of chinese
calligraphic character image. In: Advances in Multimedia Infor-
mation Processing-PCM 2008, pp. 228–237. Springer (2008)
6. Lin, Y., Wu, J., Gao, P., Xia, Y., Mao, T.: Lsh-based large scale
chinese calligraphic character recognition. In: Proceedings of the
13th ACM/IEEE-CS Joint Conference on Digital Libraries, pp.
323–330. ACM (2013)
7. Pengcheng, G., Jiangqin, W., Yuan, L., Yang, X., Tianjiao, M.: Fast
chinese calligraphic character recognition with large-scale data. In:
Multimedia Tools and Applications, pp. 1–18 (2014)
8. Wu, Y., Zhuang, Y., Pan, Y., Wu, J.: Web based chinese calligraphy
learning with 3-d visualization method. In: 2006 IEEE International
Conference on Multimedia and Expo, IEEE, pp. 2073–2076 (2006)
9. Zhang, Z., Wu, J., Yu, K.: Chinese calligraphy specific style render-
ing system. In: Proceedings of the 10th Annual Joint Conference
on Digital Libraries, pp. 99–108. ACM (2010)
10. Xia, Y., Wu, J., Gao, P., Lin, Y., Mao, T.: Ontology-based model
for chinese calligraphy synthesis. In: Computer Graphics Forum,
vol. 32, pp. 11–20. The Eurographics Association and Blackwell
Publishing Ltd. (2013)
11. Songhua, Xu, Lau, Francis, Cheung, William K., Pan, Yunhe: Auto-
matic generation of artistic chinese calligraphy. IEEE Intell. Syst.
20(3), 32–39 (2005)
12. Li, H., Liu, P., Xu, S., Lin, S.: Calligraphy beautification method
for chinese handwritings. In: 2012 Fourth International Conference
on Digital Home (ICDH), pp. 122–127. IEEE (2012)
13. Zhu, Yong, Tan, Tieniu, Wang, Yunhong: Font recognition based
on global texture analysis. IEEE Trans. Pattern Anal. Mach. Intell.
23(10), 1192–1200 (2001)
14. Ding, Xiaoqing, Chen, Li, Tao, Wu: Character independent font
recognition on a single chinese character. IEEE Trans. Pattern Anal.
Mach. Intell. 29(2), 195–204 (2007)
15. Tao, Dapeng, Jin, Lianwen, Zhang, Shuye, Yang, Zhao, Wang,
Yongfei: Sparse discriminative information preservation for chi-
nese character font categorization. Neurocomputing 129, 159–167
(2014)
16. Tao, D., Lin, X., Jin, L., et al.: Principal component 2-D long short-
term memory for font recognition on single Chinese characters.
IEEE Trans. Cybern. 46(3), 756–765 (2016)
17. Zhuang, Yueting, Lu, Weiming, Wu, Jiangqin: Latent style model:
discovering writing styles for calligraphy works. J. Vis. Commun.
Image Represent. 20(2), 84–96 (2009)
123
G. Pengcheng et al.
18. Weiming, Lu, Zhuang, Yueting, Jiangqin, Wu: Discovering callig-
raphy style relationships by supervised learning weighted random
walk model. Multimedia Syst. 15(4), 221–242 (2009)
19. Oliva, Aude, Torralba, Antonio: Modeling the shape of the scene: a
holistic representation of the spatial envelope. Int. J. Comput. Vis.
42(3), 145–175 (2001)
20. David, G.: Distinctive image features from scale-invariant key-
points. Int. J. Comput. Vis. 60(2), 91–110 (2004)
21. Ke, Y., Sukthankar, R.: Pca-sift: a more distinctive representation
for local image descriptors. In: Proceedings of the 2004 IEEE
Computer Society Conference on Computer Vision and Pattern
Recognition, 2004. CVPR 2004, vol. 2, pp. II–506. IEEE (2004)
22. Bay, H., Tuytelaars, T., Van Gool, L.: Surf: speeded up robust fea-
tures. In: Computer Vision–ECCV 2006, pp. 404–417. Springer
(2006)
23. Rublee, E., Rabaud, V., Konolige, K., Bradski, G.: Orb: an efficient
alternative to sift or surf. In: 2011 IEEE International Conference
on Computer Vision (ICCV), pp. 2564–2571. IEEE (2011)
24. Rosten, E., Drummond, T.: Machine learning for high-speed cor-
ner detection. In: Computer Vision–ECCV 2006, pp. 430–443.
Springer (2006)
25. Calonder, M., Lepetit, V., Strecha, C., Fua, P.: Brief: binary robust
independent elementary features. In: Computer Vision–ECCV
2010, pp. 778–792. Springer (2010)
26. Hinton, Geoffrey E., Salakhutdinov, Ruslan R.: Reducing the
dimensionality of data with neural networks. Science 313(5786),
504–507 (2006)
27. Donahue, J., Jia, Y., Vinyals, O., Hoffman, J., Zhang, N., Tzeng,
E., Darrell, T.: Decaf: a deep convolutional activation feature for
generic visual recognition. In: Proceedings of The 31st Interna-
tional Conference on Machine Learning, pp. 647–655 (2014)
28. Oquab, M., Bottou, L., Laptev, I., et al.: Learning and transferring
mid-level image representations using convolutional neural net-
works. In: Proceedings of the IEEE conference on computer vision
and pattern recognition, pp. 1717–1724 (2014)
29. Weiss, Y., Torralba, A., Fergus, R.: Spectral hashing. In: Advances
in Neural Information Processing Systems, pp. 1753–1760 (2008)
30. Erin Liong, V., Lu, J., Wang, G., Moulin, P., Zhou, J.: Deep hash-
ing for compact binary codes learning. In: Proceedings of the
IEEE Conference on Computer Vision and Pattern Recognition,
pp. 2475–2483 (2015)
31. Girshick, R., Donahue, J., Darrell, T., Malik, J.: Rich feature hier-
archies for accurate object detection and semantic segmentation.
In: 2014 IEEE Conference on Computer Vision and Pattern Recog-
nition (CVPR), pp. 580–587. IEEE (2014)
32. Sermanet, P., Eigen, D., Zhang, X., Mathieu, M., Fergus, R.,
LeCun, Y.: Overfeat: integrated recognition, localization and detec-
tion using convolutional networks. arXiv preprint arXiv:1312.6229
(2013)
33. Kimura, F., Takashina, K., Tsuruoka, S., Miyake, Y.: Modified
quadratic discriminant functions and the application to chinese
character recognition. IEEE Trans. Pattern Anal. Mach. Intell. 1,
149–153 (1987)
34. Liu, C.L., Yin, F., Wang, D.H., et al.: Online and offline handwritten
Chinese character recognition: benchmarking on databases. Pattern
Recogn. 46(1), 155–162 (2013)
35. Van der Maaten, Laurens, Hinton, Geoffrey: Visualizing data using
t-sne. J. Mach. Learn. Res. 9(2579–2605), 85 (2008)
36. Chang, Chih-Chung, Lin, Chih-Jen: Libsvm: a library for support
vector machines. ACM Trans. Intell. Syst. Technol. (TIST) 2(3),
27 (2011)