This article has been accepted for publication in a future issue of this journal, but has not been

fully edited. Content may change prior to final publication. Citation information: DOI 10.1109/TMM.2020.3013376, IEEE
Transactions on Multimedia
1

Robust CAPTCHAs towards Malicious OCR

Jiaming Zhang, Jitao Sang, Kaiyuan Xu, Shangxi Wu, Xian Zhao, Yanfeng Sun, Yongli Hu, Jian Yu

Abstract—Turing test was originally proposed to examine

whether machine’s behavior is indistinguishable from a human.
The most popular and practical Turing test is CAPTCHA, which
is to discriminate algorithm from human by offering recognition-
alike questions. The recent development of deep learning has
significantly advanced the capability of algorithm in solving
CAPTCHA questions, forcing CAPTCHA designers to increase
question complexity. Instead of designing questions difficult for
both algorithm and human, this study attempts to employ the
limitations of algorithm to design robust CAPTCHA questions
easily solvable to human. Specifically, our data analysis observes
that human and algorithm demonstrates different vulnerability
to visual distortions: adversarial perturbation is significantly
annoying to algorithm yet friendly to human. We are motivated
to employ adversarially perturbed images for robust CAPTCHA
design in the context of character-based questions. Four mod-
Fig. 1. Increasing content complexity of CAPTCHAs.
ules of multi-target attack, ensemble adversarial training, im-
age preprocessing differentiable approximation, and expectation
are proposed to address the characteristics of character-based
CAPTCHA cracking. Qualitative and quantitative experimental ditional character-based scheme involving with only numbers
results demonstrate the effectiveness of the proposed solution. We and English characters. With the fast progress of machine
hope this study can lead to the discussions around adversarial learning especially deep learning algorithms, simple character-
attack/defense in CAPTCHA design and also inspire the future
attempts in employing algorithm limitation for practical usage. based CAPTCHAs fail to distinguish between algorithm and
human [4]–[6]. CAPTCHA designers were therefore forced
Index Terms—adversarial example, CAPTCHA, OCR
to increase the complexity of content to be recognized. As
shown in Fig. 1, while the extremely complex CAPTCHAs
I. I NTRODUCTION reduce the risks to be cracked by algorithms, it also heavily
LAN Turing first proposed the Turing Test question “Can increase the burden of human recognition. It is noteworthy that
A machines think like human?” [1] Turing test was initially
designed to examine machine’s exhibited intelligent behavior
the effectiveness of simply increasing content complexity is
based on the assumption that human has consistently superior
that is indistinguishable from a human, and later developed recognition capability than algorithm. The last few years have
into a form of reverse Turing test with more practical goal of witnessed human-level AI in tasks like image recognition [7],
distinguishing between computer and human. Among reverse speech processing [8] and even reading comprehension [9].
Turing tests, CAPTCHA (Completely Automated Public Tur- It is easy to imagine that with the further development of
ing test to tell Computers and Humans Apart) turns out the algorithms, continuously increasing content complexity will
most well-known one used in anti-spam systems to prevent reach such a critical point that algorithm can recognize yet
abuse use of automated programs [2]. human cannot recognize.
Most early CAPTCHAs, like the reCAPTCHA [3] which Let’s review the initial goal of CAPTCHA: to discriminate
assists in the digitization of Google books, belong to the tra- human from algorithm by designing tasks unsolvable to algo-
rithms. Therefore, the straightforward solution is to employ
J. Zhang is with the Beijing University of Technology, Beijing, China (e-
mail: zhangjm@emails.bjut.edu.cn). the limitations of algorithms to facilitate CAPTCHA design.
J. Sang is with the School of Computer and Information Technology and While algorithms have advanced their performance in many
the Beijing Key Laboratory of Traffic Data Analysis and Mining, Beijing perspectives including visual/vocal recognition accuracy, they
Jiaotong University, Beijing, China (e-mail: jtsang@bjtu.edu.cn).
K. Xu is with the Beijing Jiaotong University, Beijing, China (e- remain some notorious limitations with regards to human.
mail:15281106@bjtu.edu.cn). Researchers and practitioners already employed such limita-
S. Wu is with the Beijing Jiaotong University, Beijing, China (e- tions to design new form of CAPTCHAs, e.g., developing
mail:kirinng@gmail.com).
X. Zhao is with the Beijing Jiaotong University, Beijing, China (e- cognitive [10] and sequentially related [11] questions to chal-
mail:lavender.zxshane@gmail.com). lenge algorithm’s lack of commonsense knowledge and poor
Y. Sun is with the Beijing University of Technology, Beijing, China (e-mail: contextual reasoning ability.
yfsun@bjut.edu.cn).
Y. Hu is with the Beijing University of Technology, Beijing, China (e-mail: Following this spirit, we are interested to explore the
huyongli@bjut.edu.cn). possibility of improving the robustness of CAPTCHA to-
J. Yu is with the School of Computer and Information Technology and the wards algorithm cracking without changing the traditional
Beijing Key Laboratory of Traffic Data Analysis and Mining, Beijing Jiaotong
University, Beijing, China (e-mail : jianyu@bjtu.edu.cn). character-based scheme. In other words, is it possible to
design character CAPTCHAs only friendly to human instead

1520-9210 (c) 2020 IEEE. Personal use is permitted, but republication/redistribution requires IEEE permission. See http://www.ieee.org/publications_standards/publications/rights/index.html for more information.
Authorized licensed use limited to: Carleton University. Downloaded on August 06,2020 at 01:46:57 UTC from IEEE Xplore. Restrictions apply.
This article has been accepted for publication in a future issue of this journal, but has not been fully edited. Content may change prior to final publication. Citation information: DOI 10.1109/TMM.2020.3013376, IEEE
Transactions on Multimedia
of simply increasing content complexity? The key lies in
finding the algorithm limitation compatible to the scheme of
character image. One candidate is the vulnerability to visual
distortions. We have conducted data analysis and observed
that human and algorithm exhibit different vulnerability to
visual distortions (the observations are detailed in Section III).
This inspires us to exploit those distortions friendly to human
but obstructing algorithm to pollute the original character
CAPTCHA. Specifically, adversarial perturbation [12] exactly
meets this requirement: adversarial attack1 and CAPTCHA
share the common intention that human is imperceptible to but
algorithm is significantly affected by the same distortion. The
notorious characteristic of adversarial perturbation for visual
understanding turns out to be the desired one for CAPTCHA
design.
Inspired by this, we employ adversarial perturbation to
design robust character-based CAPTCHA in this study. Cur-
rent state-of-the-art cracking solution views CAPTCHA OCR
(Optical Character Recognition) as a sequential recognition
problem [13]–[17]. To remove the potential distortions, further
image preprocessing operations are typically added before
OCR. Correspondingly in this study, we propose to simul-
taneously attack multiple targets to address the sequential
recognition issue (Section IV-A), differentiably approximate
image preprocessing operations (Section IV-C) and stochastic
image transformation (Section IV-D) in the adversarial exam-
ple generation process to cancel out their potential influence.
Moreover, since we have no knowledge about the detailed
algorithm the cracking solution used (e.g., neural network
structure), the generated adversarial examples are expected
to be resistant to unknown OCR algorithms in the black-box
cracking. This study resorts this issue to ensemble adversarial
training by generating adversarial examples effective towards
multiple algorithms (Section IV-B). In summary, the contribu-
tions of this study are two-fold:
• We have discovered the different vulnerability between
human and algorithm on visual distortions. Based on
the observations, adversarial perturbation is employed to
improve the robustness of character-based CAPTCHA.
• Corresponding to the characteristics of typical OCR
cracking solutions, we proposed a novel methodology
addressing issues including sequential recognition, indif-
ferentiable image preprocessing, stochastic image trans-
formation and black-box cracking.
II. R ELATED W ORK
A. character-based CAPTCHAs
In online services, character-based CAPTCHAs are the most
popular protection to deter character recognition programs.
Since the initial goal of CAPTCHA, friendly design and
1Adversarial attack refers to the process of adding small but
specially crafted perturbation to generate adversarial examples mis-
leading algorithm. To avoid confusion with the process of attacking
CAPTCHA, in this study, we use “adversarial attack” to indi-
cate the generation of adversarially distorted CAPTCHAs and use
“CAPTCHA crack” to indicate the attempt of passing CAPTCHA
with algorithms.
1520-9210 (c) 2020 IEEE. Personal use is permitted, but republication/redistribution requires IEEE permission. See http://www.ieee.org/publications_standards/publications/rights/index.html for more information.
Authorized licensed use limited to: Carleton University. Downloaded on August 06,2020 at 01:46:57 UTC from IEEE Xplore. Restrictions apply.
2
security of CAPTCHAs have been studied. A fundamental
requirement of CAPTCHAs necessitates that be designed to
be easy for humans but difficult for computers. In traditional
CAPTCHA design, the trade-off between usability and se-
curity is difficult to balance. The three traditional designs
are most common: background confusion, using lines and
collapsing [18]. But there are some studies that use auto-
generated methods to synthesis CAPTCHA images, e.g., using
GANs, instead of manual design [19]. These auto methods,
which are applied to both character-based CAPTCHA and
image-based CAPTCHA, are novel approaches for generating
CAPTCHAs, but they still attempt to increase content com-
plexity of CAPTCHAs.
To overcome the limitations of traditional character-based
CAPTCHAs, other designs have been proposed, e.g., 3D-based
CAPTCHAs, Animated CAPTCHAs [18]. 3D approaches to
CAPTCHA design involve the rendering of 3D models to an
image [20], [21]. However, it has been demonstrated that this
approach is easy to attacks [22], [23]. Animated CAPTCHAs
attempt to incorporate a time dimension into the design. The
addition of a time dimension is assumed to increase the
security of the resulting CAPTCHA. Nevertheless, techniques
that can successfully attack the CAPTCHAs design have been
developed [24].
The last few years have witnessed deep learning plays
an important role in the field of artificial intelligence. The
recognition rate of character-based CAPTCHAs increases
year by year. George et al. proposed a hierarchical model
called the Recursive Cortical Network (RCN) that incorporates
neuroscience insights in a structured probabilistic generative
model framework, which significantly improved the recogni-
tion rate [25]. To remove the interference in the background,
Ye et al. proposed the GAN-based approach for automati-
cally transforming training data and constructing solvers for
character-based CAPTCHAs [26]. The convolutional neural
network shows a powerful performance in the recognition of
various characters, including Chinese characters [27]. But the
convolutional neural network has low recognition accuracy in
confusion class. To solve this problem, Chen et al. proposed
a novel method of selective learning confusion class for
character-based CAPTCHAs recognition [28]. As the com-
plexity of character-based CAPTCHAs increases, the methods
based on combining convolutional neural network and recur-
rent neural network achieve state-of-the-art performance [13]–
[17]. In this paper, we employ the architecture consists of
convolution neural network (CNN) layers and long short-term
memory (LSTM) as the default OCR algorithm. We also test
our CAPTCHAs on the latest method [17] in Section V-D,
which is an attention-based model that also consists of CNN
layers and LSTM.
B. adversarial example
While deep learning has achieved great performance, it
also has some security problems. Recent work has discovered
that the existing machine learning models, not just deep
neural networks, are vulnerable to adversarial example [12].
Given a trained classifier F with model parameters W , a
This article has been accepted for publication in a future issue of this journal, but has not been fully edited. Content may change prior to final publication. Citation information: DOI 10.1109/TMM.2020.3013376, IEEE
Transactions on Multimedia
valid input x and with corresponding ground truth prediction
y, i.e., y = F (x) with model parameters W . It is often
to get a similar input x0 is close to x according to some
distance metric d(x, x0 ), and cause y 6= F (x0 ) with model
parameters W . An example x0 with this property is known
as a untargeted adversarial example. A more powerful but
difficult example called targeted adversarial example is more
than a misclassification example, i.e., t is a target label and
t 6= y, t = F (x0 ) with model parameters W .
Prior work that considers adversarial examples can be
generally classified into two categories: white-box attack and
black-box attack. White-box attack has full knowledge of
the trained classifier F including the model architecture and
model parameters W . Black-box attack has no or limited
knowledge of the trained classifier F . Black-box setting is
apparently harder than white-box setting for attackers because
of the leaked gradient information. It seems that black-box
attack is impossible, but adversarial examples that affect
one model can often affect another model, which is called
transferability [29]. In the paper, we rely on the transferability
and deploy ensemble-based approaches to generate adversarial
CAPTCHAs.
Szegedy et al. [12] first pointed out adversarial exam-
ples and proposed a box-constrained LBFGS method to find
adversarial examples. To decrease expensive computation,
Goodfellow et al. [30] proposed the fast gradient sign method
(FGSM) to generate adversarial examples by performing a
single gradient step. Kurakin et al. [31] extended this method
to an iterative version, and find out that adversarial examples
can influence physical world. Dong et al. [32] further extended
the fast gradient sign method family by proposing momentum-
based iterative algorithms. In addition, there are some more
powerful methods called optimization-based attack methods.
Deepfool [33] is an attack technique optimized for the L2
distance metric. This method is based on the assumption that
the decision boundary is partly linear, then the distance and
direction of the data points to the decision boundary can
be calculated approximately. C&W [34] is another targeted
optimization-based method. It achieves its goal by increasing
the probability of target label.
To defend against adversarial examples, several adversarial
defensive methods have been proposed, which has been an ac-
tive field of AI research. Referring to [35], we generally divide
adversarial defensive methods into two categories. Athalye
et al. [35] identify gradient masking, or called obfuscated
gradients, which leads to a false sense of security in defenses
against adversarial examples. The authors addressed that the
reason why many adversarial defenses can defend against
adversarial examples is that the fast and optimization-based
methods cannot succeed without useful gradient information.
The most common gradient masking methods include input
transformation and stochastic gradients. Input transformation
techniques, e.g., image cropping and image binarization, cause
the gradients to be non-existent or incorrect. In this paper,
image binarization will definitely result in non-differentiable
if gradient masking is not overcome. Some adversarial defense
methods cause the network itself is randomized or the input
is randomly transformed. These methods based on stochastic
1520-9210 (c) 2020 IEEE. Personal use is permitted, but republication/redistribution requires IEEE permission. See http://www.ieee.org/publications_standards/publications/rights/index.html for more information.
Authorized licensed use limited to: Carleton University. Downloaded on August 06,2020 at 01:46:57 UTC from IEEE Xplore. Restrictions apply.
3
gradients lead adversarial attack methods using a single sample
of the randomness to incorrectly estimate the true gradient.
Goodfellow et al. [30] first proposed adversarial training
method, adversarial examples are regarded as training samples
to fit the model until these samples are classified correctly. The
idea is effective and general for all types of adversarial attacks.
This makes the network more robust against the adversarial
examples, but cost expensive computation, especially at a large
scale, e.g., the ImageNet [36] scale. In general, the existing
defensive methods cannot completely eliminate adversarial
attacks.
Many researchers have found that adversarial example can
be applied in other tasks, such as semantic segmentation [37],
face detection [38], and even speech recognition [39] and
translation [40]. The majority of the published papers have fo-
cused on how to eliminate the impact of adversarial examples
in application. Li et al. [41] evaluated adversarial examples
among different detection services, such as violence, politi-
cian, and pornography detection. Ling et al. [42] proposed a
uniform platform for comprehensive evaluation on adversarial
attacks and defenses in application, which can benefit future
adversarial examples research. In contrast, studies on employ-
ing adversarial examples against the malicious algorithm are
relatively limited. Osadchy et al. [43] employed adversarial
examples to design CAPTCHAs and analyzed security and
good usability of CAPTCHAs. But they only considered these
data types like MNIST and ImageNet instead of CAPTCHA
data types. Zhang et al. [44] studied the robustness of adver-
sarial examples on different types of CAPTCHA and gave the
suggestions that how to improve the security of CAPTCHA
using adversarial examples. Shi et al. [45] improved the
effectiveness of the adversarial example by using the Fourier
transform to generate CAPTCHA images in the frequency
domain. However, they only considered generating adversarial
examples on CNN systems, which is essentially the adversarial
attack algorithm based on the classification task. In contrast,
the current state-of-the-art CAPTCHA cracking system con-
sists of feature extraction module and sequential recognition
module (CNN + LSTM). Shi et al. [46] deployed character-
based adversarial CAPTCHAs on a large-scale online platform
and tested the proposed CAPTCHAs on convolutional recur-
rent neural networks [47]. However, they ignored experiments
and discussions on adversarial defense technologies, such as
image binarization and adversarial training. In Section V-D, we
compare our method with ACs [45] to prove that considering
the sequential recognition is essential. In Section V-B, we
show the necessity of considering image preprocessing.
III. DATA A NALYSIS
To justify the feasibility of employing algorithm limitations
for CAPTCHA design and motivate our detailed solution, this
section conducts data analysis to answer two questions: (1)
Whether human and algorithm have different vulnerability to
visual distortion? (2) What characteristics to consider when
employing distortions to design robust CAPTCHA?
Text-based CAPTCHA is the most widely deployed scheme
requiring subjects to recognize characters from 0-9 and A-
Z. Due to its simplicity, character-based CAPTCHA is very
This article has been accepted for publication in a future issue of this journal, but has not been fully edited. Content may change prior to final publication. Citation information: DOI 10.1109/TMM.2020.3013376, IEEE
Transactions on Multimedia
4

(a)Character distorted by Gaussian white noise (b)Character distorted by adversarial perturbation

1.1 1.1
1 1
0.9 0.9
Recognition accuracy

Recognition accuracy
0.8 0.8
0.7 0.7
0.6 0.6
0.5 0.5
0.4 0.4
0.3 0.3
0.2 0.2
0.1 algorithm human 0.1 algorithm human
0 0
0 1 2 3 4 5 6 7 8 9 10 0 1 2 3 4 5 6 7 8 9 10
Distortion level Distortion level

(c)Recognition accuracy on Gaussian distorted characters (d)Recognition accuracy on adversarially distorted characters

Fig. 2. Human v.s. algorithm vulnerability analysis results on Gaussian and adversarial distortions

A. Vulnerability Analysis to Visual Distortion

This subsection designs character recognition competition
between human and algorithm to analyze their vulnerabil-
ity to visual distortions. We employed two types of visual
distortions: (1) Gaussian white noise is one usual distortion
to generate CAPTCHAs. In this study, the added one-time
Gaussian white noise follows normal distribution with mean
(a) µ̃ = 0, variance σ̃ = 0.01 and constant power spectral
density. (2) Adversarial perturbation has been recognized as
1.1
1
imperceptible to human but significantly confusing algorithm.
0.9 We employ the widely used F GSM [30] to add adversarial
Recognition accuracy

0.8 perturbation, where one-time perturbation is constituted with

0.7
0.6
0.5
0.4
0.3
0.2
0.1 before
0
0 1 2 3
Fig. 3. The affection of image preprocessing: (a) distorted characters before
(top row) and after (bottom row) image binarization; (b)Recognition accuracy
on adversarially distorted characters.
effective to examine the robustness towards cracking algo-
rithm as well as friendliness to human. Therefore, this study
employs character-based CAPTCHA as the example scheme
to conduct data analysis, develop solution and implement
experiments. Specifically, during data analysis, we assume that
each CAPTCHA question is constituted by single character
in an RGB image with unique resolution of 48 × 64px. The
character font is fixed as DroidSansMono. The remainder of
the section will report the observations regarding human and
character recognition performance in different scenarios.
step size of 0.02. To examine the change of recognition
performance with increasing distortion difficulty, we added
8 levels of distortions onto the original character images
accumulatively: each level corresponds to 5 one-time Gaus-
after
sian white noises and adversarial perturbations respectively.
4 5 6 7 8 9 10 Examples for derived distorted CAPTCHA images in different
Distortion level levels are illustrated in Fig. 2(a) and (b).
(b) Regarding the human side, we recruited 77 master workers
from Amazon Mechanical Turk (MTurk). Each subject was
asked to recognize 450 character CAPTCHAs with Gaussian
and adversarial distortions in different levels respectively.
Regarding the algorithm side, we employed the state-of-the-
art OCR (Optical Character Recognition) algorithm, which
is the segmentation-based approach for OCR works by seg-
menting a text line image into individual character images
and then recognizing the characters [48]. The resultant av-
erage recognition accuracies for Gaussian and adversarially
distorted CAPTCHAs are shown in Fig. 2(c) and (d). We
can see that, for Gaussian distorted CAPTCHAs, human’s
recognition accuracy consistently declines as the distortion
level increases, indicating that Gaussian white noise tends
to undermine human’s vision. On the contrary, the examined
OCR algorithm demonstrates good immunity to Gaussian
white noise, possibly due to the noise removal effect by
multiple convolutional layers [49]. It is easy to imagine that

1520-9210 (c) 2020 IEEE. Personal use is permitted, but republication/redistribution requires IEEE permission. See http://www.ieee.org/publications_standards/publications/rights/index.html for more information.
Authorized licensed use limited to: Carleton University. Downloaded on August 06,2020 at 01:46:57 UTC from IEEE Xplore. Restrictions apply.
This article has been accepted for publication in a future issue of this journal, but has not been fully edited. Content may change prior to final publication. Citation information: DOI 10.1109/TMM.2020.3013376, IEEE
Transactions on Multimedia
if we design CAPTCHA by adding Gaussian white noise,
as the noise level increases, the resultant CAPTCHAs will
critically confuse humans instead of obstructing the cracking
OCR algorithms.
For adversarially distorted CAPTCHAs, we observed quite
opposite recognition results. Fig. 2(d) shows that humans
are more robust to the adversarial perturbations, while OCR
algorithm is highly vulnerable as the adversarial distortion
increases. This is not surprising since adversarial perturbation
is specially crafted to change the algorithm decision under
the condition of not confusing human. This characteristic of
adversarial perturbation demonstrates one important limitation
of algorithm regards to human ability, which perfectly satisfies
the requirement of robust CAPTCHA: algorithm tends to fail,
while human remains successful. Therefore, we are motivated
to employ adversarial examples to design robust CAPTCHA
to distinguish between algorithm and human.
B. Characteristics Affecting Robust CAPTCHA Design
The previous subsection observes that adversarial pertur-
bation is effective to mislead state-of-the-art OCR algorithm,
which shows its potential to be employed to design robust
CAPTCHA. However, typical CAPTCHA cracking solution
involves beyond OCR, e.g., image preprocessing operations
like binarization and Gaussian filtering will be applied to
remove distortions before issuing to the OCR module. Fig. 3(a)
illustrates the adversarially distorted CAPTCHA images before
and after binarization preprocessing. It is easy to conceive that
the effectiveness of adversarial perturbation will be critically
affected by image preprocessing operations.
We further quantified this affection by analyzing the OCR
performance on the same adversarially distorted CAPTCHA
images from previous subsection. The recognition accuracies
on the CAPTCHAs before and after binarization preprocessing
are plotted and compared in Fig. 3(b). It is shown that
after removing most distortions via image binarization, OCR
algorithm demonstrates basically stable performance in recog-
nizing CAPTCHAs with different levels of adversarial pertur-
bation. This tells us that standard adversarial perturbation is
insufficient to obstruct the cracking method. It is necessary
to design the robust CAPTCHA solution considering the
characteristics (like preprocessing operations) of CAPTCHA
cracking method.
IV. M ETHODOLOGY
As shown on the left of Fig. 4, typical cracking of character-
based CAPTCHA consists of two stages as image prepro-
cessing and OCR. The above data analysis has demonstrated
that image preprocessing has the effect of distortion removal,
making it not possible to straightforwardly employ adversarial
perturbation for robust CAPTCHA design. In addition to the
image preprocessing stage, the OCR stage also possesses char-
acteristics obstructing CAPTCHA: (1) sequential recognition,
disabling the traditional single character-oriented adversarial
perturbation; and (2) black-box crack, making it ineffective to
attack one specific OCR model. To address the above char-
acteristics of CAPTCHA cracking, our proposed CAPTCHA
1520-9210 (c) 2020 IEEE. Personal use is permitted, but republication/redistribution requires IEEE permission. See http://www.ieee.org/publications_standards/publications/rights/index.html for more information.
Authorized licensed use limited to: Carleton University. Downloaded on August 06,2020 at 01:46:57 UTC from IEEE Xplore. Restrictions apply.
5
generation framework consists of three modules: multi-target
attack, ensemble adversarial training, and image preprocessing
differentiable approximation. The proposed framework and its
relation to CAPTCHA cracking are illustrated on the right of
Fig. 4.
A. Multi-target Attack towards Sequential Recognition
Typical CAPTCHAs usually contain more than one char-
acter for recognition, e.g. the example CAPTCHAs contain
4 characters. Therefore, state-of-the-art CAPTCHA cracking
solutions are forced to address a sequential character recogni-
tion problem at the OCR stage [48]. Specifically, OCR stage
consists of three sub-modules as feature extraction, sequential
recognition, and output decoding. Feature extraction is basi-
cally realized by a convolutional neural network to encoding
the input image as neural feature. Sequential recognition is
typically realized by a recurrent neural network to process the
issued image neural feature and output multiple tokens includ-
ing characters (0-9, A-Z) and blank token ∅2 Output decoding
serves to transform the sequential tokens into final character
recognition results, by merging sequentially duplicated tokens
and removing blank ∅ tokens. For example, the original token
sequence “aa∅b∅∅ccc∅dd” will be transformed to “abcd”.
While CAPTCHA cracking views OCR as a sequential
recognition problem, standard adversarial perturbation is de-
signed to attack single target. In this study, we propose to
attack multiple targets corresponding to the multiple tokens
derived from OCR sequential recognition. The generated ad-
versarial CAPTCHA image is expected to simultaneously mis-
classify all the character tokens. For specific token sequence
t, all the characters appearing in t constitute the original set
Θ, while the remaining characters from (0-9, A-Z) constitute
the adversary set Θ̄. Denoting the raw image as x and the
corresponding adversary image as x0 , the multi-target attack
is formulated as the following optimization problem:
min d(x, x0 ) + λ ·
X
[max F (x0 )θj i − F (x0 )θθ̄i ]+ (1)
0
x
θi ∈Θ
j6=θ̄i i
where d(·, ·) is distance function to minimize the modification
from x to x03 , λ is the weight parameter balancing between
the image modification and the misclassification confidence in
the second term. Within the second term, θi is the character
appearing in the original set Θ, θ̄i is its one-to-one mapping
character in the adversary set Θ̄, F (x0 )θi denotes the output of
the second-to-last layer, the logits, corresponding to token θi
after sequential recognition, F (x0 )θj i denotes its j dimension,
and [f ]+ is the positive part function denoting max(f, 0).
Note that the one-to-one mapping from θi to θ̄i can be
either random or fixed. Random one-to-one mapping leads to
targeted adversarial attack, and fixed mapping leads to non-
targeted adversarial attack4 .
2 For typical 4 character-based CAPTCHAs, recurrent neural net-
work usually outputs 12-token sequence to improve tolerance for
segmentation and alignment [48].
3 Alternative choices for the distance function are allowed. In our
experiment, we use L2 distance.
4 The reported experimental results in Section V are based on
random one-to-one mapping.
This article has been accepted for publication in a future issue of this journal, but has not been fully edited. Content may change prior to final publication. Citation information: DOI 10.1109/TMM.2020.3013376, IEEE
Transactions on Multimedia
6

recognized
characters “R4GQ” Output decoding ① Multi-target Attack ② Ensemble
Adversarial Training
“RR 444 GGQQ” 𝜃𝑖 𝜃𝑖 + 𝐾
min 𝑑 𝑥, 𝑥 ′ + 𝜆 ෍[max 𝐹 𝑥 ′ 𝑗 − 𝐹 𝑥′ ഥ𝑖 ]
𝜃
𝑥′ ഥ𝑖 𝐹෨ 𝑥 ′ = ෍ 𝛼𝑘 𝐽𝑘 (𝑥′)
Optical Character Sequential recognition 𝜃𝑖
𝑗≠𝜃
𝑘=1
Recognition “RR 444 G GQQ”

“AA BBB C CDD”

Feature extraction GoogLeNet ResNet DenseNet 4ConvNet

Gaussian ③ Differentiable ④ Expectation

filtering Approximation
Image Preprocessing
Image
binarization
original Stochastic adversarial
𝑥′2
CAPTCHA transformation s 𝑥′ =
1
′
1+𝑒 −(𝑥 −𝜏) g 𝑥′ =
1
2𝜋𝜎
−
𝑒 2𝜎2 CAPTCHA image
image CAPTCHA cracking CAPTCHA generation
“ABCD”
Fig. 4. The proposed robust CAPTCHA designing framework. The left represents the process of CAPTCHA cracking, including sequential recognition,
feature extraction, image binarization (Gaussian filtering) and stochastic transformation. The right represents our solution of CAPTCHA generation, including
the corresponding multi-target attack, ensemble adversarial training, differentiable approximation and expectation, respectively.

When the original set Θ contains only one character, the
multi-target attack reduces to single-target attack as the stan-
dard adversarial perturbation. In fact, according to the mech-
anism of output decoding in CAPTCHA cracking, we only
need to misclassify any one of the character tokens to invalid
the final recognition result. The above equation in Eqn. (1)
provides a general case of attacking flexible numbers of
character tokens. In practice, the number of attacked characters
is one important parameter to control the model performance.
More attacked characters guarantee higher success rate to
resist crack, yet leading to more derived distortions and human
recognition burden. The quantitative influence of attacked
character number on the image distortion level and algorithm
recognition rate is discussed in Section V-C.
B. Ensemble Adversarial Training towards Black-box Crack
As mentioned in Section I, CAPTCHA cracking may
employ multiple OCR algorithms for character recognition.
At the stage of designing CAPTCHA, it is impractical to
target one specific OCR algorithm, which requires to design
adversarial CAPTCHA images that are effective to as many
OCR algorithms as possible. Fortunately, it is recognized that
adversarial perturbation is transferable between models: if
an adversarial image remains effective for multiple models,
it is more likely to transfer to other models as well [29].
Inspired by this, in order to improve the resistance to unknown
cracking models, we propose to generate adversarial images
simultaneously misleading multiple models.
Specifically, given K white-box OCR models with their cor-
responding the output of the second-to-last layer as J1 , ..., JK ,
we re-formulate the objective function in Eqn. 1 by replacing
F (x0 ) with F̃ (x0 ) defined as follows:
K
F̃ (x0 ) =
X
αk Jk (x0 )
k=1
PK
where αk is the ensemble weight with k=1 αk = 1. In most
cases, αk = 1/K except that one model is more important than
others. Among the three sub-modules of OCR stage, feature
extraction has the most model choices (e.g. various CNN
structures as GoogLeNet [50], ResNet [51]) which can be eas-
ily implemented into different CAPTCHA cracking solutions.
Therefore, this study addresses the black-box cracking issue
by attacking multiple feature extraction models. Specifically,
the training data and basic structure of Ji (x0 ) and F (x0 ) are
identical except for the different CNN structures in the feature
extraction sub-module. On the number of CNN structures, the
larger the value of K, the stronger the generalization capability
of the derived adversarial CAPTCHA images. However, an
excessive K value will lead to high computational complexity
and trivial weight αk to underemphasize single model. Refer-
ring to previous studies on ensemble adversarial attack [52],
3 ∼ 5 models achieve a good balance between transferability
and practicality. In this study, we select K = 3 and evenly set
αk = 1/3. The experimental results in [52] show that under
the same training set, the adversarial examples can achieve
stronger transferability when the network structure is more
similar, and it is reasonable to choose the model with large
structure difference to employ ensemble adversarial training.
The performance of employing ensemble adversarial training
to resist different OCRs is reported in Section V-D.
C. Differentiable Approximation towards Image Preprocess-
ing
The data observations in Section III-B demonstrate the
distortion removal consequences from binarization operation,
requiring us to consider the affection of image preprocessing
in adversarial image generation. To address this, we regard
image preprocessing operation as part of the entire end-to-
(2) end solution so that we can generate corresponding adversarial
images effectively to mislead the whole cracking solution.

1520-9210 (c) 2020 IEEE. Personal use is permitted, but republication/redistribution requires IEEE permission. See http://www.ieee.org/publications_standards/publications/rights/index.html for more information.
Authorized licensed use limited to: Carleton University. Downloaded on August 06,2020 at 01:46:57 UTC from IEEE Xplore. Restrictions apply.
This article has been accepted for publication in a future issue of this journal, but has not been fully edited. Content may change prior to final publication. Citation information: DOI 10.1109/TMM.2020.3013376, IEEE
Transactions on Multimedia
According to the usability to be incorporated into the end-to-
end solution, image preprocessing operations can be roughly
divided into two categories as either differentiable or non-
differentiable. For each category, we select one representative
operation to address in this study, i.e., Gaussian filtering
and image binarization. Regarding the differentiable Gaussian
0
x 2
filtering operation, g(x0 ) = √2πσ 1
e− 2σ2 , we can readily
incorporate it into the OCR model (Eqn. (1), Eqn. (2)) by
replacing the input image x0 with the preprocessing image
g(x0 ). Both forward and backward propagation are conducted
on the replaced function F (g(x0 )), leading to the generated
adversarial images expected to eliminate the affection from
Gaussian filtering.
Regarding the non-differentiable image binarization , we
cannot straightforwardly incorporate it into the objective func-
tion. Instead, we find a differentiable approximation s(x0 ) to
image binarization and incorporate the approximated function
into the end-to-end solution. In this study, s(x0 ) is defined as
follows:
1
s(x0 ) = x0 −τ
(3)
1 + e− ω
where τ denotes the threshold of image binarization, ω denotes
the degree of lateral expansion of the curve. Note that to
guarantee that the generated adversarial images are resistant to
image binarization, we only employ the approximated s(x0 )
at the backward propagation stage to update the generated
image, while the forward propagation still use the actual x0 to
calculate ∇x F (x).
To simultaneously resist to the affections from Gaussian
filtering and image binarization, we concatenate s(·) and g(·)
in the final objective function. Therefore, the overall optimiza-
tion problem incorporating the three proposed modules is as
follows:
X
min d(x, x0 ) + λ · [max F̃ (φ(x0 ))θj i − F̃ (φ(x0 )))θθ̄i ]+
x 0
j6=θ̄i
θi ∈Θ
(4)
where F̃ (·) denotes the ensemble of multiple OCR models
defined in Eqn. (2), and φ(x0 ) = s(g(x0 )) denotes the approx-
imated image preprocessing operations defined in Eqn. (3).
D. Expectation towards Stochastic Image Transformation
The above three subsections are more than enough for gen-
eral CAPTCHAs generation towards OCR cracking solution.
However, a potential cracker could use a number of trans-
formations to make the adversarial perturbations meaningless,
e.g., the cracker could slightly rotate the image, doing so
entirely bypasses general adversarial examples. Prior work
has shown that these adversarial examples fail to remain
effective under image transformations, and the gradient of the
expected value can increase adversarial even under various
image transformations [53]. To integrate image preprocessing
issues with potential transformations, we compute expectation
of stochastic image transformation including different angles
rotation. The expectation allows us for the construction of
adversarial examples that remain adversarial over a chosen
transformation distribution T . More concretely, we use a
chosen distribution T of transformation functions t taking an
1520-9210 (c) 2020 IEEE. Personal use is permitted, but republication/redistribution requires IEEE permission. See http://www.ieee.org/publications_standards/publications/rights/index.html for more information.
Authorized licensed use limited to: Carleton University. Downloaded on August 06,2020 at 01:46:57 UTC from IEEE Xplore. Restrictions apply.
7
input x controlled by the adversary to the “true” input t(x)
perceived by the OCR rather than optimizing the objective
function of a single example. We then re-formulate the second
term in Eqn. (4) by replacing x, x0 with t(x), t(x0 ) defined as
follows:
X
Et∼T [max F̃ (φ(t(x0 )))θj i − F̃ (φ(t(x0 ))))θ̄θi ]+ (5)
θi ∈Θ
j6=θ̄i i
Furthermore, rather than simply taking the d(·, ·) to constrain
the solution space, we instead aim to constrain the expected
effective distance between the adversarial and original inputs.
The first term in Eqn. (4) is replaced by d(t(x), t(x0 )) defined
as follows:
Et∼T [d(t(x), t(x0 ))] (6)
In practice, the distribution T can model perceptual trans-
formation such as color change, image translation, random
rotation, or addition of noise. These transformations amount
to a set of the random linear combinations, which are more
thoroughly described in Section V-E. Then we can approx-
imate the gradient of the expected value through sampling
transformations independently at each gradient descent step in
optimizing the objective function and differentiating through
the transformation. Given its ability to generate robust ad-
versarial CAPTCHA images, we use the gradient of the ex-
pected value to directly eliminate the affection from stochastic
transformation for differentiable image transformation. But
regarding some non-differentiable image transformation, we
cannot straightforwardly differentiate through the transforma-
tion. Instead, we can use the same category as mentioned
in Section IV-C that finding a differentiable approximation
and incorporate the approximated function into the end-to-end
solution.
V. E XPERIMENTS
i
We examined CAPTCHA images with 4 characters for ex-
periments. The CAPTCHAs are RGB images with resolution
of 192 × 64px. Regarding the cracking method, we consid-
ered image binarization and Gaussian filtering (kernel size:
3 × 3, σ = 0.8) at the image preprocessing stage. The OCR
stage is instantiated with CNN structures for feature extraction
and LSTM+softmax for sequential recognition. Regarding our
proposed CAPTCHA generation method, image binarization
is approximated with τ = 0.8, ω = 0.05, and 4 CNN
structures are employed for ensemble adversarial training. All
experiments are conducted on Nvidia GTX 1080Ti GPU with
11G memory.
A. Qualitative Attention Analysis
Visual attention has been widely used to explain which
region of image contributes much to the model decision [54].
In this study, we extracted the attention map using Grad-
CAM [55] to understand the change of recognition perfor-
mance under different visual distortions.
The first and second columns of Fig. 5 visualize the at-
tention map of the raw image and the image with Gaussian
white noise. It can be found that Gaussian white noise brings
This article has been accepted for publication in a future issue of this journal, but has not been fully edited. Content may change prior to final publication. Citation information: DOI 10.1109/TMM.2020.3013376, IEEE
Transactions on Multimedia
TABLE I
T HE RECOGNITION OF DIFFERENT COMPLEXITY LEVELS OF CAPTCHA S IN THE DIFFERENT SETTINGS . T HE RESULTS OF ALGORITHMS ARE OBTAINED
AFTER G AUSSIAN FILTERING AND IMAGE BINARIZATION .
Raw rCAPTCHA parallel
algorithm 100.0% 95.6%
Easy
human 99.0% 94.0%
algorithm 91.0% 88.0%
Medium
human 73.0% 51.0%
algorithm 81.0% 83.0%
Hard
human 56.0% 36.0%
Fig. 5. Example images (top row) and theri attention maps (bottom row).
From left to right, we show the original image, the image with Gaussian
white noise, the adversarial generated by our method and the adversarial image
generated by our method but without considering image preprocessing.
Fig. 6. Example CAPTCHAs with different complexity levels (from top to
bottom: easy, medium, hard). Each row from left to right shows the different
settings of Raw, rCAPTCHA parallel, rCAPTCHA w/o preprocessing and
rCAPTCHA.
trivial attention change from the original image. Both attention
maps keep to the region where characters exist. This well
explains the data observations in Section III-A that algorithm
is generally robust to Gaussian white noise. We also visualized
the attention map of the CAPTCHA images generated from
our proposed method on the third column of Fig. 5. It is shown
that the attention maps deviate much from the original image
and focus on unrelated regions where there exist no characters.
This justifies our motivation to employ adversarial perturbation
to mislead the algorithm prediction result, and demonstrates
the effectiveness of our proposed CAPTCHA design method.
To further validate the necessity of considering image pre-
processing in robust CAPTCHA design, the attention maps for
the images generated from our method but without considering
image preprocessing are shown in the fourth column of Fig. 5
for comparison. It is easy to conceive that without considering
image preprocessing, the generated images fail to deviate the
attention from the character regions. This is consistent with
the fact that image preprocessing has the effect of weakening
or eliminating adversarial perturbation.
1520-9210 (c) 2020 IEEE. Personal use is permitted, but republication/redistribution requires IEEE permission. See http://www.ieee.org/publications_standards/publications/rights/index.html for more information.
Authorized licensed use limited to: Carleton University. Downloaded on August 06,2020 at 01:46:57 UTC from IEEE Xplore. Restrictions apply.
8
rCAPTCHA w/o preprocessing rCAPTCHA
68.4% 0.0%
94.0% 94.0%
58.0% 0.0%
67.0% 65.0%
45.0% 4.0%
51.0% 49.0%
B. Quantitative Performance Comparison
To compare the performance of the proposed robust
CAPTCHA (rCAPTCHA) designing method, we report the
recognition accuracies of state-of-the-art cracking solution
under the following settings:
• Raw: the original CAPTCHA images without adding
adversarial perturbations;
• rCAPTCHA parallel: the proposed solution to generated
adversarial images, expect that the sequential recognition
sequential recognition sub-module of OCR is replaced
by 4 parallel recognition networks (each realized by
one fully-connected layer) to address one character’s
recognition;
• rCAPTCHA w/o preprocessing: the proposed solution to
generated adversarial images, but without considering the
image preprocessing stages;
• rCAPTCHA: the proposed solution to generated adversar-
ial images, considering both sequential recognition and
image preprocessing operations.
The state-of-the-art cracking solution is trained over 20, 000
CAPTCHA images with batch size 128. To examine the
application scope of the proposed CAPTCHA generation
methods, we conducted experiments on the CAPTCHAs with
three levels of complexities: easy, medium, hard. Fig. 6
shows examples of different complexity levels of CAPTCHAs
in the above four settings. For each of the settings, we
selected/generated 500 CAPTCHA images for testing, and
summarize the derived average recognition accuracy in Table I.
Experimental observations include: (1) By adding adversarial
perturbations, the right 3 columns consistently obtain lower
accuracies than the first column, showing the usability of
employing adversarial perturbations in resisting CAPTCHA
cracking. (2) Without considering the sequential recognition
or image preprocessing characteristics, the resisting effect
of rCAPTCHA parallel and rCAPTCHA w/o preprocessing
is not as obvious as that of rCAPCHA. This validates the
necessity of multi-target attack and differentiable approxi-
mation modules. (3) Regarding CAPTCHAs with different
complexities, we observed consistent phenomenon among the
four settings, demonstrating the wide application scope of the
proposed CAPTCHA generation method.
The notable decrease in algorithm recognition accuracies
shows the effectiveness of employing adversarial perturbation
to mislead cracking solution. To facilitate the correlation un-
derstanding between misleading cracking solution and friendly
This article has been accepted for publication in a future issue of this journal, but has not been fully edited. Content may change prior to final publication. Citation information: DOI 10.1109/TMM.2020.3013376, IEEE
Transactions on Multimedia
9

300 300

250
250

Image distortion
Image distortion

200
200
150
150
100

50 100
10 12 14 16 18 20 22 24 26 28 30 1 2 3 4
Distortion level Number of characters attacked
(a) Image distortion (a) Image distortion

1 1
0.9 0.9
Algorithm recognition

Algorithm recognition
0.8 0.8
0.7 0.7
0.6 0.6
0.5 0.5
0.4 0.4
0.3 0.3
0.2 0.2
0.1 0.1
0 0
10 12 14 16 18 20 22 24 26 28 30 1 2 3 4
Distortion level Number of characters attacked
(b) Algorithm recognition accuracy (b) Algorithm recognition accuracy
Fig. 7. The influence of λ on derived image distortion and cracking Fig. 8. The influence of |Θ| on derived image distortion and cracking
recognition accuracy. recognition accuracy.

to human recognition, we also provide the human recognition

accuracy for each experimental setting in Table I. Similar
to the data analysis, we have recruited 164 workers from
MTurk to recognize 4 character-based CAPTCHA images.
The reported accuracies are averaged over 1, 200 CAPTCHAs.
By comparing different rows, it is shown that the increasing
content complexity brings slight decrease of algorithm recog-
nition accuracy but causes huge trouble to human recognition.
Among different setting columns, while the algorithm recog-
nition accuracy fluctuates a lot, the human recognition perfor-
mance basically remains stable, validating the different distor-
tion vulnerability between human and algorithm. In summary,
regarding CAPTCHA images with different complexities, the
proposed CAPTCHA generation method succeeds to invalid
the cracking algorithm without increasing human recognition
burden.
C. Parameter Influence Analysis
The proposed robust CAPTCHA generation method mainly
involves with two parameters: the weight parameter λ in
Eqn. (1) and the number of attacked characters |Θ|. As in-
troduced in the methodology, the weight parameter λ controls
the relative importance between the visual distortion and
misclassification confidence. We adjusted λ within range of
[10, 30] with step of 1 and examined its influence on the
derived image distortion and cracking recognition accuracy.
The image distortion is measured as the sum of squared pixel-
wise difference between the original and adversarial images.
Fig. 9. Example CAPTCHAs with different image distortions: from left to
right shows images with distortions of 100, 200, 300 and 400.
The averaged distortion and recognition accuracy with the
change of λ are drawn in Fig. 7. It is shown that as λ increases,
more image distortion is observed in the derived CAPTCHA
images and cracking method tends to fail in recognizing the
generated CAPTCHAs. This is consistent with the definition
of λ in Eqn. (1). In practical applications, to prevent annoying
human subjects in recognizing the generated CAPTCHAs, an
appropriate λ is selected with moderate image distortion and
guaranteed cracking resistant performance. Our experimental
results reported in Section V-B are based on λ = 20.
Regarding the number of attacked characters, we set |Θ| to
{1, 2, 3, 4} respectively and examined the corresponding aver-
aged image distortion and algorithm recognition accuracy in
Fig. 8. As shown in Fig. 8(a), with |Θ| increases, more image
distortion is needed to misclassify the characters. In Fig. 9
we show example CAPTCHAs generated by rCAPTCHA with
different levels of image distortions. Combing with Fig. 8(a), it
is demonstrated that using the proposed rCAPTCHA method

1520-9210 (c) 2020 IEEE. Personal use is permitted, but republication/redistribution requires IEEE permission. See http://www.ieee.org/publications_standards/publications/rights/index.html for more information.
Authorized licensed use limited to: Carleton University. Downloaded on August 06,2020 at 01:46:57 UTC from IEEE Xplore. Restrictions apply.
This article has been accepted for publication in a future issue of this journal, but has not been fully edited. Content may change prior to final publication. Citation information: DOI 10.1109/TMM.2020.3013376, IEEE
Transactions on Multimedia
10

TABLE II
T RANSFERABILITY OF ADVERSARIAL IMAGES GENERATED BETWEEN PAIRS OF MODELS . T HE ELEMENT (i, j) REPRESENTS THE ACCURACY OF THE
ADVERSARIAL IMAGES GENERATED FOR MODEL i ( ROW ) TESTED OVER MODEL j ( COLUMN ).

Testing Model
Training Model
4ConvNet ResNet DenseNet GoogLeNet GoogLeNet w/ Average
Method No Attack 93% 84% 95% 97% 94% 93%
4ConvNet 54% 68% 82% 71% 70% 69%
ACs ResNet 47% 63% 73% 53% 64% 60%
DenseNet 53% 70% 60% 71% 71% 65%
4ConvNet 1% 3% 13% 7% 16% 8%
ResNet 8% 0% 3% 12% 11% 7%
rCAPTCHA DenseNet 16% 2% 3% 23% 27% 14%
Ensemble 0% 2% 1% 3% 7% 3%
GoogLeNet w/ 12% 11% 18% 3% 1% 9%

to attack even all 4 characters, the derived CAPTCHAs are
generally friendly to human and not bringing extra recognition
burden. As shown in Fig. 8(b), the increase of |Θ| enhances
the confidence to mislead the cracking algorithm and obtains
consistently lower recognition accuracy. With the introduction
of multi-attack towards sequential recognition, the proposed
rCAPTCHA method possess the flexibility to attack arbitrary
number of characters. In our experiments, to guarantee the
resistance capability, we fixed |Θ| = 4.
D. Robustness towards Different OCRs
To compare the generalization and transferability of our
proposed rCAPTCHA method and ACs [45], we imple-
mented different cracking methods and examined their recog-
nition accuracy on the generated CAPTCHAs. For gener-
ating CAPTCHAs of our method, we respectively trained
3 OCR models with different CNN structures, which are
denoted as 4ConvNet, mini-ResNet and mini-DenseNet. For
generating CAPTCHAs of [45], we trained the same OCR
as above, except that the sequential recognition sequential
recognition sub-module (LSTM) of OCR is replaced by 4 par-
allel recognition networks (fully-connected layer). For testing
CAPTCHAs, we trained 2 OCR models. One is the same OCR
with CNN structure of mini-GoogLeNet, the other is mini-
GoogLeNet w/ attention [17], which also uses mini-GoogLeNet
but adopts the attention mechanism in LSTM. 4ConvNet uses
four convolutional layers for feature extraction. The LSTM
input required a fixed-size feature vector, so we modified the
native network. mini-XNets are employed due to the quicker
convergence times and low resolution of CAPTCHA images:
mini-ResNet consists of five ResBlocks and two convolutional
layers, mini-DenseNet consists of four DenseBlocks with four
convolutional layers, and mini-GoogLeNet consists of two
Inception modules with six convolutional layers.
Three models of 4ConvNet, mini-ResNet, mini-DenseNet are
selected as white-boxs, with the remaining mini-GoogLeNet
model and mini-GoogLeNet w/ attention as the black-boxs.
The black-box models are regard as the potential OCR crack-
ing to simulate the alternative cracking choices in real-world
applications. Averaged over 100 tested CAPTCHAs, Table II
summarizes the black-box cracking recognition accuracy under
different training-testing pairs. For example, the value of
91% at element (1, 1) represents the recognition accuracy of
original CAPTCHA images on 4ConvNet. The value of 0%
at element (8, 1) indicates the recognition accuracy trained
with ensembled 3 white-box models and tested on 4ConvNet.
Lower accuracy value means superior resistant performance to
cracking solutions and better transferability of the method.
In the top half of Table II (ACs), the adversarial CAPTCHAs
without considering the sequential recognition obtain higher
average accuracies than rCAPTCHA. The accuracies of ACs
are no lower than 60% when image preprocessing and im-
age transformation are not involved. It is expected that the
employing image preprocessing and image transformation can
increase the accuracy, and 60% means that these adversar-
ial CAPTCHAs are almost recognized by the OCR. This
demonstrates when excluding controlled variables, considering
the sequential recognition problem is more important than
adopting Fourier transform during the generation of adversarial
CAPTCHAs. In the bottom half of Table II (rCAPTCHA),
we can observe that the adversarial images generated with
one model perform well on their own models but generally
perform poorly on other models. However, if we generate
the CAPTCHA images with ensemble training of 3 models,
the testing recognition accuracies for all 5 models are no
higher than 7%. This demonstrates the transferability of the
proposed rCAPTCHA method in employing ensemble training
towards black-box cracking. Specially, the value of 7% at
element (8, 4) demonstrates our method can perform well
on GoogLeNet w/ attention (GoogLeNet w/). The reason
is that the network structure of the current state-of-the-art
OCR model is similar (CNN + LSTM). We also generate
the adversarial images on GoogLeNet w/ attention (the last
row), which validates the generalization of our proposed
mechanism. It is expected with more models implemented in
ensemble training, the resistant performance towards arbitrary
black-box cracking methods will be guaranteed. In practical
applications, we can carefully select white-box models with
typically different structures to improve the generalization and
transferability to specific models.

1520-9210 (c) 2020 IEEE. Personal use is permitted, but republication/redistribution requires IEEE permission. See http://www.ieee.org/publications_standards/publications/rights/index.html for more information.
Authorized licensed use limited to: Carleton University. Downloaded on August 06,2020 at 01:46:57 UTC from IEEE Xplore. Restrictions apply.
This article has been accepted for publication in a future issue of this journal, but has not been fully edited. Content may change prior to final publication. Citation information: DOI 10.1109/TMM.2020.3013376, IEEE
Transactions on Multimedia
11

TABLE III
D ISTRIBUTION OF TRANSFORMATIONS

Transformation Minimum Maximum

Rotation -15° 15°
X-axis Translation − 14 width 1
4 width

Y-axis Translation − 14 height 1

4 height

Color Change(RGB) -50 50

Fig. 10. Example CAPTCHAs with different complexity levels (from top to
Rescale 0.15 1.65 bottom: easy, medium, hard). Each row from left to right shows the different
settings of image transformation.

TABLE IV
T HE RECOGNITION OF RAW IMAGES AND ADVERSARIAL IMAGES . T HE
RESULT ARE OBTAINED AFTER STOCHASTIC TRANSFORMATION .

Images Easy Medium Hard

Raw 95.0% 91.6% 73.1%
Adversarial 1.3% 5.4% 9.0%

E. Robustness towards Stochastic Image Transformation

To justify the performance of our method for stochastic im-
age transformation, we implemented a set of transformations Fig. 11. The recognition accuracy of relax adversarial training.
on different complexity levels of CAPTCHAs and examined
their recognition accuracies on the generated CAPTCHAs.
increases along with complexity levels of CAPTCHAs are the
Specifically, we considered the distribution of transformations
noises associated with “hard” CAPTCHA images mix with
that includes rotation, color change, rescale, and translation
the adversarial perturbation and thus adversarial perturbation
of the image. We selected/generated the 1, 000 images for
will be weakened as the image gets more complex. By com-
each of the settings, randomly chose an image transformation
paring the results in Table I, the larger the “approximation”
for each image, and examined whether our method is robust
component is, the more we can detect this trend.
over the chosen distribution. For each adversarial example,
Fig. 10 shows example CAPTCHAs of different complexity
we evaluated over 100 stochastic transformations sampled
levels after stochastic transformations, combing with images
from the distribution at evaluation time. We give the specific
of non-transformations, which observes that these images are
parameters we chose of stochastic transformations in Table III,
still adversarial after different image transformations. This
where each parameter is sampled uniformly at random from
demonstrates that our approach is effective in generating robust
the specified range. These stochastic transformations include
adversarial examples.
color change, image translation, rotation and rescale.
Nevertheless, when the number of transformations the
Table IV summarizes the results. Experimental observa-
cracker chose is too large, expectation will occasionally fail
tions include: (1) By applying stochastic transformations, the
to generate an effective adversarial example. In this study,
second row consistently obtains lower accuracies than the
the number we chose is 4 and the scope of transformations
first row, showing the usability of employing expectation
is also reasonable. Another case occurs when we replace
towards stochastic image transformation. (2) After stochastic
non-differentiable image preprocessing with differentiable ap-
transformations, the accuracies of raw images become lower
proximation. Because the essence of expectation is also an
than before transformations. The decrease in algorithm recog-
“approximation”, when the “double approximation” occurs,
nition accuracies shows that stochastic transformation does
the effectiveness of adversarial example will be weakened.
hurt performance of normal image, which is caused by the
loss of image information and the shortcoming of neural
network, e.g., lack of rotation-invariance. The increase from F. Discussion on Adversarial Defense
0.0%, 0.0%, 4.0% to 1.3%, 5.4%, 9.0% means that stochastic The image non-differentiable preprocessing and stochastic
transformation partially plays a role in eliminating effective- image transformation both are aimed at weakening the ef-
ness of adversarial examples. Because the stochastic trans- fectiveness of adversarial example, which are called adver-
formation is a kind of adversarial defense technology, more sarial defenses. However, the existence of robust adversarial
specific discussions about adversarial defense are reported in CAPTCHAs implies that defenses based on stochastic trans-
Section V-F. The reason that algorithm recognition accuracy forming input are not secure: adversarial examples generated

1520-9210 (c) 2020 IEEE. Personal use is permitted, but republication/redistribution requires IEEE permission. See http://www.ieee.org/publications_standards/publications/rights/index.html for more information.
Authorized licensed use limited to: Carleton University. Downloaded on August 06,2020 at 01:46:57 UTC from IEEE Xplore. Restrictions apply.
This article has been accepted for publication in a future issue of this journal, but has not been fully edited. Content may change prior to final publication. Citation information: DOI 10.1109/TMM.2020.3013376, IEEE
Transactions on Multimedia
by using differentiable approximation and expectation can
circumvent these defenses. Prior work has shown that most
of adversarial defenses are based on obfuscated gradients,
in this study: (1)image non-differentiable preprocessing is a
kind of shattered gradients, which are nonexistent or incorrect
gradients caused either intentionally through non-differentiable
operations or unintentionally through numerical instability;
(2) stochastic image transformation is a kind of stochastic
gradients, which depend on test-time randomness [35]. But
if the obfuscated gradient information can be approximated, it
can only provide a false sense of security.
There is another kind of adversarial defense technology
called adversarial training, which is not dependent on obfus-
cated gradient. Adversarial training solves a min-max game
through a conceptually simple process: train on adversarial
examples until the model learns to classify them correctly [30].
To further validate the adversarial defense, we study the ad-
versarial training approach of [56] in this subsection. For this
scenario, we generated/selected 500 adversarial CAPTCHA
images for testing. Then we started to fine-tune the model with
the 50, 000 steps. However, due to the complexity of OCR
model, compared with the common CNN model, standard
adversarial training does not show the effectiveness as usual.
After training with 50, 000 steps, the accuracy of OCR is
still 0%. So we relax the constraints of standard adversarial
training to examine whether the idea of adversarial training
will work. Then we fine-tune the model on the same 500
adversarial images as testset. The results are shown in Fig. 11,
from combined results we make the following observations.
As training data increases, adversarial training significantly
improves the accuracy of the OCR model. But its shortcom-
ings also obvious that the time cost is gradually increasing.
Moreover, if a cracker wants to use adversarial training, he is
supposed to have access to the training dataset we use and the
parameter in algorithm we choose, e.g., distance function to
minimize the modification and so on.
We discussed adversarial defense technology on adversarial
CAPTCHAs we proposed. The defense technologies based on
obfuscated gradients cannot hinder the type of CAPTCHAs.
The adversarial training based on non-obfuscated gradients is
still effective but limited to practicality.
VI. CONCLUSION
This study designs robust character-based CAPTCHAs to
resist cracking algorithms by employing their unrobustness to
adversarial perturbation. We have conducted data analysis and
observed human and algorithm’s different vulnerabilities to
visual distortions. Based on the observation, robust CAPTCHA
(rCAPTCHA) generation framework is introduced with three
modules of multi-target attack, ensemble adversarial training,
differentiable approximation to image preprocessing, and ex-
pectation to stochastic image transformation. Qualitative and
quantitative experimental results demonstrate the effectiveness
of generated CAPTCHAs in resisting cracking algorithms.
We ascribe the main contribution not as proposing a specific
CAPTCHA system, but as introducing the idea of exploiting
algorithm unrobustness to increase the robustness of automated
1520-9210 (c) 2020 IEEE. Personal use is permitted, but republication/redistribution requires IEEE permission. See http://www.ieee.org/publications_standards/publications/rights/index.html for more information.
Authorized licensed use limited to: Carleton University. Downloaded on August 06,2020 at 01:46:57 UTC from IEEE Xplore. Restrictions apply.
12
system towards cracking. Character-based CAPTCHA, which
is most friendly and effective to human, to validate this idea
with the simplest scheme. This idea is expected to easily adopt
to generate robust image-based and other CAPTCHAs.
It is noted that similar to the game competition between ad-
versarial attack and defense, with more CAPTCHA designers
employing adversarial attack to resist cracking, future cracking
solutions are expected to employ adversarial defense tech-
niques for self-enhancement. We hope this study could draw
attention of future CAPTCHA designing on the competition
between adversarial attack and defense. Moreover, with the
development of deep learning and other AI algorithms, we
are confronted with critical security-related problems when
algorithms are maliciously utilized towards human. In this
case, it is necessary to get aware of the limitations of current
algorithms and appropriately employ them to resist the abuse
use of algorithms.
R EFERENCES
[1] A. M. Turing, “Computing machinery and intelligence-am turing,” Mind,
vol. 59, no. 236, p. 433, 1950.
[2] M. Naor, “Verification of a human in the loop or identification via the
turing test,” 1996.
[3] L. Von Ahn, B. Maurer, C. McMillen, D. Abraham, and M. Blum, “re-
captcha: Human-based character recognition via web security measures,”
Science, vol. 321, no. 5895, pp. 1465–1468, 2008.
[4] A. A. Chandavale, A. M. Sapkal, and R. M. Jalnekar, “Algorithm to
break visual captcha,” in 2009 International Conference on Emerging
Trends in Engineering & Technology, 2009, pp. 258–262.
[5] G. Mori and J. Malik, “Recognizing objects in adversarial clutter:
Breaking a visual captcha,” in 2003 IEEE Computer Society Conference
on Computer Vision and Pattern Recognition, vol. 1, 2003, pp. I–I.
[6] S. Sivakorn, I. Polakis, and A. D. Keromytis, “I am robot:(deep) learning
to break semantic image captchas,” in 2016 IEEE European Symposium
on Security and Privacy (EuroS&P), 2016, pp. 388–403.
[7] K. He, X. Zhang, S. Ren, and J. Sun, “Delving deep into rectifiers:
Surpassing human-level performance on imagenet classification,” in
2015 IEEE Conference on Computer Vision and Pattern Recognition
(CVPR), 2015, pp. 1026–1034.
[8] J. F. Gemmeke, D. P. Ellis, D. Freedman, A. Jansen, W. Lawrence, R. C.
Moore, M. Plakal, and M. Ritter, “Audio set: An ontology and human-
labeled dataset for audio events,” in 2017 IEEE International Conference
on Acoustics, Speech and Signal Processing (ICASSP), 2017, pp. 776–
780.
[9] P. Rajpurkar, J. Zhang, K. Lopyrev, and P. Liang, “Squad: 100,000+
questions for machine comprehension of text,” 2016.
[10] M. R. Ogiela, N. Krzyworzeka, and L. Ogiela, “Application of
knowledge-based cognitive captcha in cloud of things security,” Con-
currency and Computation: Practice and Experience, vol. 30, no. 21, p.
e4769, 2018.
[11] D. Geman, S. Geman, N. Hallonquist, and L. Younes, “Visual turing test
for computer vision systems,” Proceedings of the National Academy of
Sciences, vol. 112, no. 12, pp. 3618–3623, 2015.
[12] C. Szegedy, W. Zaremba, I. Sutskever, J. Bruna, D. Erhan, I. Goodfellow,
and R. Fergus, “Intriguing properties of neural networks,” in 2014
International Conference on Learning Representations (ICLR), 2014.
[13] Q. Liu, L. Wang, and Q. Huo, “A study on effects of implicit and
explicit language model information for dblstm-ctc based handwriting
recognition,” in 2015 International Conference on Document Analysis
and Recognition (ICDAR), 2015, pp. 461–465.
[14] T. M. Breuel, “High performance text recognition using a hybrid
convolutional-lstm implementation,” in 2017 IAPR International Con-
ference on Document Analysis and Recognition (ICDAR), vol. 1, 2017,
pp. 11–16.
[15] M. Jenckel, S. S. Bukhari, and A. Dengel, “Transcription free lstm ocr
model evaluation,” in 2018 International Conference on Frontiers in
Handwriting Recognition (ICFHR), 2018, pp. 122–126.
[16] H.-R. Shin, J.-S. Park, and J.-K. Song, “Ocr for drawing images using
bidirectional lstm with ctc,” in 2019 IEEE Student Conference on
Electric Machines and Systems (SCEMS 2019), 2019, pp. 1–4.
This article has been accepted for publication in a future issue of this journal, but has not been fully edited. Content may change prior to final publication. Citation information: DOI 10.1109/TMM.2020.3013376, IEEE
Transactions on Multimedia
[17] Y. Zi, H. Gao, Z. Cheng, and Y. Liu, “An end-to-end attack on text
captchas,” IEEE Transactions on Information Forensics and Security,
vol. 15, pp. 753–766, 2019.
[18] Y.-W. Chow, W. Susilo, and P. Thorncharoensri, “Captcha design and
security issues,” in Advances in Cyber Security: Principles, Techniques,
and Applications, 2019, pp. 69–92.
[19] H. Kwon, Y. Kim, H. Yoon, and D. Choi, “Captcha image generation
systems using generative adversarial networks,” IEICE TRANSACTIONS
on Information and Systems, vol. 101, no. 2, pp. 543–546, 2018.
[20] M. E. Hoque, D. J. Russomanno, and M. Yeasin, “2d captchas from
3d models,” in Proceedings of the IEEE SoutheastCon 2006, 2006, pp.
165–170.
[21] C. R. Macias and E. Izquierdo, “Visual word-based captcha using 3d
characters,” 2009.
[22] V. D. Nguyen, Y.-W. Chow, and W. Susilo, “On the security of text-based
3d captchas,” Computers & Security, vol. 45, pp. 84–99, 2014.
[23] Q. Ye, Y. Chen, and B. Zhu, “The robustness of a new 3d captcha,”
in 2014 IAPR International Workshop on Document Analysis Systems,
2014, pp. 319–323.
[24] V. D. Nguyen, Y.-W. Chow, and W. Susilo, “Breaking an animated
captcha scheme,” in International Conference on Applied Cryptography
and Network Security, 2012, pp. 12–29.
[25] D. George, W. Lehrach, K. Kansky, M. Lázaro-Gredilla, C. Laan,
B. Marthi, X. Lou, Z. Meng, Y. Liu, H. Wang et al., “A generative
vision model that trains with high data efficiency and breaks text-based
captchas,” Science, vol. 358, no. 6368, p. eaag2612, 2017.
[26] G. Ye, Z. Tang, D. Fang, Z. Zhu, Y. Feng, P. Xu, X. Chen, and Z. Wang,
“Yet another text captcha solver: A generative adversarial network based
approach,” in Proceedings of the 2018 ACM SIGSAC Conference on
Computer and Communications Security, 2018, pp. 332–348.
[27] Y. Lv, F. Cai, D. Lin, and D. Cao, “Chinese character captcha recognition
based on convolution neural network,” in 2016 IEEE Congress on
Evolutionary Computation (CEC), 2016, pp. 4854–4859.
[28] J. Chen, X. Luo, Y. Liu, J. Wang, and Y. Ma, “Selective learning
confusion class for text-based captcha recognition,” IEEE Access, vol. 7,
pp. 22 246–22 259, 2019.
[29] N. Papernot, P. McDaniel, and I. Goodfellow, “Transferability in ma-
chine learning: from phenomena to black-box attacks using adversarial
samples,” arXiv preprint arXiv:1605.07277, 2016.
[30] I. J. Goodfellow, J. Shlens, and C. Szegedy, “Explaining and harnessing
adversarial examples,” in 2015 International Conference on Learning
Representations (ICLR), 2015.
[31] A. Kurakin, I. J. Goodfellow, and S. Bengio, “Adversarial examples
in the physical world,” in 2017 International Conference on Learning
Representation Workshop, 2017.
[32] Y. Dong, F. Liao, T. Pang, H. Su, J. Zhu, X. Hu, and J. Li, “Boosting
adversarial attacks with momentum,” in 2018 IEEE Conference on
Computer Vision and Pattern Recognition (CVPR), 2018, pp. 9185–
9193.
[33] S.-M. Moosavi-Dezfooli, A. Fawzi, and P. Frossard, “Deepfool: a simple
and accurate method to fool deep neural networks,” in 2016 IEEE
Conference on Computer Vision and Pattern Recognition (CVPR), 2016,
pp. 2574–2582.
[34] N. Carlini and D. Wagner, “Towards evaluating the robustness of neural
networks,” in 2017 IEEE Symposium on Security and Privacy (SP),
2017, pp. 39–57.
[35] A. Athalye, N. Carlini, and D. A. Wagner, “Obfuscated gradients give a
false sense of security: Circumventing defenses to adversarial examples,”
2018 International Conference on Machine Learning (ICML), pp. 274–
283, 2018.
[36] J. Deng, W. Dong, R. Socher, L.-J. Li, K. Li, and L. Fei-Fei, “Imagenet:
A large-scale hierarchical image database,” in 2009 IEEE Conference on
Computer Vision and Pattern Recognition (CVPR), 2009, pp. 248–255.
[37] C. Xie, J. Wang, Z. Zhang, Y. Zhou, L. Xie, and A. Yuille, “Adversarial
examples for semantic segmentation and object detection,” in 2017 IEEE
International Conference on Computer Vision (ICCV), 2017, pp. 1369–
1378.
[38] M. Sharif, S. Bhagavatula, L. Bauer, and M. K. Reiter, “Accessorize to
a crime: Real and stealthy attacks on state-of-the-art face recognition,”
in 2016 ACM Sigsac Conference on Computer and Communications
Security, 2016, pp. 1528–1540.
[39] N. Carlini and D. Wagner, “Audio adversarial examples: Targeted attacks
on speech-to-text,” in 2018 IEEE Security and Privacy Workshops
(SPW), 2018, pp. 1–7.
[40] J. Li, S. Ji, T. Du, B. Li, and T. Wang, “Textbugger: Generating
adversarial text against real-world applications,” in 2019 Annual Network
and Distributed System Security Symposium, 2019.
1520-9210 (c) 2020 IEEE. Personal use is permitted, but republication/redistribution requires IEEE permission. See http://www.ieee.org/publications_standards/publications/rights/index.html for more information.
Authorized licensed use limited to: Carleton University. Downloaded on August 06,2020 at 01:46:57 UTC from IEEE Xplore. Restrictions apply.
13
[41] X. Li, S. Ji, M. Han, J. Ji, Z. Ren, Y. Liu, and C. Wu, “Adversarial
examples versus cloud-based detectors: A black-box empirical study,”
IEEE Transactions on Dependable and Secure Computing, 2019.
[42] X. Ling, S. Ji, J. Zou, J. Wang, C. Wu, B. Li, and T. Wang, “Deepsec: A
uniform platform for security analysis of deep learning model,” in 2019
IEEE Symposium on Security and Privacy (SP), 2019, pp. 673–690.
[43] M. Osadchy, J. Hernandez-Castro, S. Gibson, O. Dunkelman, and
D. Pérez-Cabo, “No bot expects the deepcaptcha! introducing immutable
adversarial examples, with applications to captcha generation,” IEEE
Transactions on Information Forensics and Security, vol. 12, no. 11, pp.
2640–2653, 2017.
[44] Y. Zhang, H. Gao, G. Pei, S. Kang, and X. Zhou, “Effect of adver-
sarial examples on the robustness of captcha,” in 2018 International
Conference on Cyber-Enabled Distributed Computing and Knowledge
Discovery (CyberC), 2018, pp. 1–109.
[45] C. Shi, X. Xu, S. Ji, K. Bu, J. Chen, R. Beyah, and T. Wang, “Adversarial
captchas,” arXiv preprint arXiv:1901.01107, 2019.
[46] C. Shi, S. Ji, Q. Liu, C. Liu, Y. Chen, Y. He, Z. Liu, R. Beyah, and
T. Wang, “Text captcha is dead? a large scale deployment and empirical
study,” in Proceedings of the 2020 ACM Conference on Computer and
Communications Security, 2020.
[47] B. Shi, X. Bai, and C. Yao, “An end-to-end trainable neural network
for image-based sequence recognition and its application to scene
text recognition,” IEEE Transactions on Pattern Analysis and Machine
Intelligence, vol. 39, no. 11, pp. 2298–2304, 2015.
[48] T. M. Breuel, A. Ul-Hasan, M. A. Al-Azawi, and F. Shafait, “High-
performance ocr for printed english and fraktur using lstm networks,”
in 2013 12th International Conference on Document Analysis and
Recognition, 2013, pp. 683–687.
[49] F. Liao, M. Liang, Y. Dong, T. Pang, X. Hu, and J. Zhu, “Defense against
adversarial attacks using high-level representation guided denoiser,” in
2018 IEEE Conference on Computer Vision and Pattern Recognition
(CVPR), 2018, pp. 1778–1787.
[50] C. Szegedy, W. Liu, Y. Jia, P. Sermanet, S. E. Reed, D. Anguelov,
D. Erhan, V. Vanhoucke, and A. Rabinovich, “Going deeper with con-
volutions,” in 2015 IEEE Conference on Computer Vision and Pattern
Recognition (CVPR), 2015, pp. 1–9.
[51] K. He, X. Zhang, S. Ren, and J. Sun, “Deep residual learning for image
recognition,” in 2016 IEEE Conference on Computer Vision and Pattern
Recognition (CVPR), 2016, pp. 770–778.
[52] Y. Liu, X. Chen, C. Liu, and D. Song, “Delving into transferable adver-
sarial examples and black-box attacks,” 2017 International Conference
on Learning Representations (ICLR), 2017.
[53] A. Athalye, L. Engstrom, A. Ilyas, and K. Kwok, “Synthesizing robust
adversarial examples,” in 2018 International Conference on Machine
Learning (ICML), 2018, pp. 284–293.
[54] M. D. Zeiler and R. Fergus, “Visualizing and understanding convolu-
tional networks,” in 2014 European Conference on Computer Vision
(ECCV), 2014, pp. 818–833.
[55] R. R. Selvaraju, M. Cogswell, A. Das, R. Vedantam, D. Parikh, and
D. Batra, “Grad-cam: Visual explanations from deep networks via
gradient-based localization,” in 2017 IEEE International Conference on
Computer Vision (ICCV), 2017, pp. 618–626.
[56] A. Madry, A. Makelov, L. Schmidt, D. Tsipras, and A. Vladu, “To-
wards deep learning models resistant to adversarial attacks,” in 2018
International Conference on Learning Representations (ICLR), 2018.