IEEE TRANSACTIONS ON INFORMATION FORENSICS AND SECURITY, VOL.

15, 2020 753

An End-to-End Attack on Text CAPTCHAs

Yang Zi, Haichang Gao , Member, IEEE, Zhouhang Cheng, and Yi Liu

Abstract— Text-based CAPTCHAs are the most widely used aimed at hollow text CAPTCHAs; [3] used a brute-force
CAPTCHA scheme. Most text-based CAPTCHAs have been approach with a high cost; and [4] utilized a Gabor filter,
cracked. However, previous works have mostly relied on a series but the component combination was complex, and the graph
of preprocessing steps to attack text CAPTCHAs, which was
complicated and inefficient. In this paper, we introduce a simple, search was time-consuming. With the development of deep
generic, and effective end-to-end attack on text CAPTCHAs learning, the application of deep learning to CAPTCHA break-
without any preprocessing. Through a convolutional neural ing has become a trend. Reference [5] introduced a convo-
network and an attention-based recurrent neural network, our lutional neural network (CNN) in their attack on Microsoft’s
attack broke a wide range of real-world text CAPTCHAs that CAPTCHA in 2015. In 2018, Tang et al. [6] claimed that they
are deployed by the top 50 most popular websites ranked by
Alexa.com. In addition, this paper comprehensively analyzed the proposed a generic method, and they adopted a CNN as well.
security of most resistance mechanisms of text-based CAPTCHAs However, these methods were mostly based on preprocessing
through experiments. Experimental results prove that the anti- and segmentation techniques, which were complicated and
segmentation principle can be completely broken under deep inefficient.
learning attacks without any segmentation or preprocessing steps In this paper, we propose an end-to-end method that is
in contrast to commonly held beliefs.
simple, generic and effective in breaking text CAPTCHAs.
Index Terms— CAPTCHA, text-based, security, CNN, RNN, This method utilizes an attention-based model that consists of
attention, deep learning. a CNN and a recurrent neural network (RNN). We tested our
I. I NTRODUCTION attack on text CAPTCHAs deployed by the top 50 most popu-
lar websites (ranked by Alexa.com). The experimental results
C APTCHA (Completely Automated Public Turing Test to
Tell Computers and Humans Apart) is used to distinguish
malicious bots from legitimate users [1] by automatically
demonstrated that our model obtained high success rates
without any segmentation or other preprocessing techniques.
Additionally, we conducted a systematic analysis of the
setting up specified tests that are difficult for computers
effectiveness of all common resistance mechanisms under deep
but easy for humans. This technology has almost become a
learning attacks. Our targeted schemes cover almost all real-
standard security mechanism used to prevent automatic voting,
world CAPTCHA design features. The analysis proved that the
registration, spam and dictionary attacks on passwords on
segmentation-resistance principle may no longer be applicable
websites. The most commonly used type of CAPTCHA is
under deep learning attacks. This paper also analyzed the
a text-based CAPTCHA that requires the user to decipher
security of a series of uncommon CAPTCHAs. All target
characters within an image.
CAPTCHAs were broken with high success rates, indicating
To enhance the security of text CAPTCHAs, CAPTCHA
that our model is generic for various CAPTCHA schemes.
designers have made every effort to design CAPTCHAs that
The rest of this paper is organized as follows: Section II
are robust to different automated attacks, and their measures
first briefly introduces the most commonly used CAPTCHA
have included distorting or rotating characters and applying
resistance mechanisms, then summarizes previous methods in
noise or a complex background. In early studies, the process
text-based CAPTCHA breaking; Section III introduces the
of mainstream segmentation-based attacks consisted of three
network structure of our model and the training process in
main steps: preprocessing, segmentation and recognition. Con-
detail; Section IV evaluates the security of a range of real-
sidering that different CAPTCHA schemes may have distinct
world text CAPTCHAs; Section V makes a comprehensive
features due to their unique designs and generation algorithms,
analysis of the effectiveness of existing resistance schemes;
attackers must accordingly design different preprocessing and
Section VI outlooks the development direction of text-based
segmentation methods. It is clear that the whole attack process
CAPTCHAs in the future; and Section VII concludes the
is very tedious and lacks generalization capacity. Some pre-
paper.
vious studies claimed to propose generic methods [2]–[4],
but these methods still have limitations. Reference [2] only
II. BACKGROUND
Manuscript received July 11, 2018; revised April 7, 2019 and May 25,
2019; accepted July 9, 2019. Date of publication July 15, 2019; date of current A. Summary of Resistance Mechanisms
version September 24, 2019. This work was supported by the National Natural
Science Foundation of China under Grant 61472311. The associate editor Text CAPTCHAs are the most commonly used CAPTCHA
coordinating the review of this manuscript and approving it for publication schemes. The intuitiveness and low cost of generation are
was Prof. Karen Renaud. (Corresponding author: Haichang Gao.) the main reasons for the popularity of this type. However,
The authors are with the Institute of Software Engineering, Xidian Univer-
sity, Xi’an 710071, China (e-mail: hchgao@xidian.edu.cn). since the development of image processing techniques, early
Digital Object Identifier 10.1109/TIFS.2019.2928622 text CAPTCHAs are vulnerable to rogue programs due to
1556-6013 © 2019 IEEE. Personal use is permitted, but republication/redistribution requires IEEE permission.
See http://www.ieee.org/publications_standards/publications/rights/index.html for more information.

Authorized licensed use limited to: NATIONAL INSTITUTE OF TECHNOLOGY SURATHKAL. Downloaded on September 13,2022 at 15:13:58 UTC from IEEE Xplore. Restrictions apply.
 754
Fig. 1. Samples of real-world CAPTCHA schemes with different resistance
mechanisms: (a) character overlapping, (b) rotation, (c) distortion, (d) hol-
low, (e) varied-length, (f) multi-fonts, (g) noisy arcs, and (h) background
interference.
their simple structure. A series of resistance mechanisms have
been adopted to enhance the security of text CAPTCHAs.
We summarize the most widely used schemes as follows:
1) Character Overlapping: The segmentation-resistance
principle was a commonly accepted guideline for CAPTCHA
designers, and having the characters overlap is a good appli-
cation of the principle, as shown in Figure 1(a).
2) Rotation: Rotation means to rotate the characters at a
certain angle, as shown in Figure 1(b).
3) Distortion: Distortion, which means that the text of
a CAPTCHA is locally or globally warped, as shown
in Figure 1(c).
4) Hollow: This design uses contour lines to form charac-
ters (see Figure 1(d)) to increase the difficulty of extracting
an individual character.
5) Variable Length: The string lengths of most text
CAPTCHAs are fixed. However, some CAPTCHAs adopt a
variable string length. For the Microsoft CAPTCHA shown
in Figure 1(e), the text length varies from 8 to 10.
6) Multi-Font: Multi-font means that different fonts are
used in the same CAPTCHA image, as shown in Figure 1(f).
7) Noisy Arcs: Using noisy arcs is an anti-segmentation
defense. As shown in Figure 1(g), noisy arcs include thin
foreground arcs, thick foreground arcs and thin background
arcs.
8) Background Interference: Background interference
embeds characters into a complex background to prevent
attackers from separating and segmenting the CAPTCHA,
as shown in Figure 1(h).
In practice, a CAPTCHA scheme always combines several
resistance mechanisms to guarantee its security.
B. Previous Works
1) Ad Hoc Method: This type of method aims to break
a specific CAPTCHA with particular features. Early studies
including [7], [8] and [9] used shape feature descriptors,
distortion estimation techniques or different preprocessing
techniques to break a specific CAPTCHA. In contrast to
Authorized licensed use limited to: NATIONAL INSTITUTE OF TECHNOLOGY SURATHKAL. Downloaded on September 13,2022 at 15:13:58 UTC from IEEE Xplore. Restrictions apply.
IEEE TRANSACTIONS ON INFORMATION FORENSICS AND SECURITY, VOL. 15, 2020
the early sophisticated algorithms, Yan and El Ahmad [10]
used the pixels counting method to attack Captchaservice.org
with a success rate of nearly 100%. Reference [11] proposed
new segmentation techniques and managed to crack several
CAPTCHA schemes but failed on CCT CAPTCHAs. In 2015,
Karthik and Recasens [5] broke the Microsoft CAPTCHA
by template matching with a success rate of 5.6%. They
also attempted to use a CNN as the character classifier and
achieved a success rate of 57.1%. The CNN they used is a
variation of LeNet-5, which consists of three convolutional
layers and three-fully connected layers. However, Karthik’s
segmentation algorithm is based on the assumption that the
string length of each CAPTCHA is constant. If the string
length and character size are variable, their segmentation
method will lose efficacy. In addition, Starostenko et al. [12]
broke four CAPTCHAs with success rates ranging from 40.4%
to 94.3%. And Gao et al. [13] attacked a Microsoft two-layer
CAPTCHA by transforming the CAPTCHA into two single-
layer CAPTCHAs and achieved a success rate of 44.6%.
2) Toolbox Method: This kind of method refers to apply-
ing corresponding techniques in the toolbox to specific
CAPTCHAs. In 2009, PWNtcha [14] broke a large range
of CAPTCHAs. The success rates ranged from 49% to
100%. However, all of the methods are cumbersome and
result in a slow attack speed. In 2010, Li et al. [15] built
a set of CAPTCHA-breaking tools to break 3 e-banking
CAPTCHAs and 41 login CAPTCHA schemes. In 2011,
Bursztein et al. [16] proposed Decaptcha, which broke 13 out
of 15 of schemes with success rates ranging from 4.9% to
66.2%. The whole process includes five stages and seven
techniques. Various techniques were selected based on the
features of each CAPTCHA.
3) Generic Method: Gao et al. [2] broke five hollow
CAPTCHA schemes with success rates ranging from 36%
to 89%, but their method lost effectiveness when dealing
with non-hollow schemes. Their team also proposed another
method that was based on the Gabor filter [4] that effectively
broke a variety of text CAPTCHAs with success rates ranging
from 5% to 77%. The disadvantage of their method is that the
preprocessing step is not efficient enough, especially when
dealing with CAPTCHAs adopting complicated background
interference. Bursztein et al. [3] applied their approach to
multiple CAPTCHAs. The success rates ranged from 5.3%
to 55.2%. The drawback to this method is that as the length
of the CAPTCHA increases, the computation cost increases
exponentially.
Tang et al. [6] used two CNNs in their pipeline method
and broke eleven real-world CAPTCHA schemes, with success
rates ranging form 10.1% to 90.0%. The CNNs in their
method are both based on LeNet-5 network and consist of four
convolutional layers, two max pooling payers and three fully-
connected layers. The first CNN in their method is responsible
for predicting the CAPTCHA length, and the other is used
for recognizing individual characters. However, the training
process of two individual CNNs is time-consuming. Mori and
Malik [17] managed to break CAPTCHAs in one step using a
single CNN. The CAPTCHA in their experiment is the most
primary and fixed-length scheme. Therefore, the robustness
 ZI et al.: END-TO-END ATTACK ON TEXT CAPTCHAs
Fig. 2.
of their method remains to be tested. In 2017, Le et al. [18]
combined a CNN and an RNN and proposed a end-to-end
model. The CNN structure consists of six convolutional layers,
three max-pooling layers and two fully-connected layers. First,
the CNN embeds a CAPTCHA image into a fixed-dimensional
embedding vector. At each time step, the LSTM concatenates
the image embedding, the hidden state and the one-hot pre-
diction of the prior time step as input to predict the one-
hot prediction of each character. But when solving real-world
CAPTCHAs, their model requires extra finetuning process,
making their method a two-stage attack. George et al. [19]
proposed a recursive cortical network (RCN) that broke four
schemes with success rates ranging from 57.1% to 66.6%.
For each scheme, nine parameters of this network must be set
manually. Additionally, the average attack speed is 94 seconds,
which is extremely slow.
III. N ETWORK S TRUCTURE
Our goal is to recognize the complete character sequence
without any segmentation. We use a sequence-to-sequence
model similar to that in [20], which was originally designed for
the text recognition task in natural scenes, in our experiment.
The model is based on a CNN, a long short-term memory
(LSTM) network and a new attention mechanism. The archi-
tecture is briefly illustrated in Figure 2. The model mainly
consists of two parts: an encoder and a decoder.
A. Feature Extraction
The encoder of our model is essentially a CNN. The CNN
is used to extract features from the whole CAPTCHA image.
Specifically, the function of the CNN is to embed the original
CAPTCHA image into an embedding vector that contains
the global information of the original CAPTCHA image. The
feature vector extracted by the encoder is denoted as f = fi, j,c
(i and j index the location in the feature map, and c indexes
the channels).
The Inception-v3 network [21], which is the key component
of GoogLeNet and achieves excellent performance in
Authorized licensed use limited to: NATIONAL INSTITUTE OF TECHNOLOGY SURATHKAL. Downloaded on September 13,2022 at 15:13:58 UTC from IEEE Xplore. Restrictions apply.
755
Architecture overview.
the ILSVRC2014 image classification task [22], was used as
the CNN feature extractor. Another reason that we choose the
Inception-v3 network is that the inception block increases the
depth and width of network. According to the empirical
CNN design principles proposed by [21], increasing the
network depth and width tends to contribute to better network
performance.
B. Character Recognition
The core of our model is an LSTM. It works as a decoder
to transfer the feature vector f into a text sequence. Compared
to a traditional RNN, which performs poor in storing infor-
mation over extended time intervals, the LSTM overcomes
the weakness of the RNN in long-term dependencies and
explicitly learns when to store information. In a traditional
sequence-to-sequence model, at each time step, the decoder
takes the whole embedding vector f as the input directly.
This paper introduces the attention mechanism into our model.
The bottleneck of the traditional encoder-decoder structure
is that the input is constant, which limits the representation
capability of the model. The attention mechanism allows the
decoder to ignore irrelevant information while preserving the
most significant information of the feature vector f. In essence,
the attention mechanism assigns different weights to different
parts of the feature vector, so that the model can focus on a
specific part of the feature vector at each time step, making the
predictions more accurate. This is the fundamental reason why
our method can recognize every individual character without
segmentation.
Bahdanau’s [23] team adopted the attention mechanism in
the RNN encoder-decoder framework to address the machine
translation task. They first calculated the relevance of different
parts of the input and output. Based on this intermediate result,
they assigned different weights to the context vectors when
predicting each target word. The attention mechanism helped
them achieve a good performance.
In [24], Chorowski et al. proposed a location-aware attention
mechanism. Their method measured the relevance of each part
of the feature map by integrating information of the hidden
 756
state at the current time step, the attention distribution at the
prior time step and the global feature map. They adopted their
attention mechanism in a speech recognition task and achieved
good results.
In our task, the model sometimes needed to skip some
unwanted character recognition, as described in Section V.G
later. Therefore, the attention mechanisms above are not
applicable to our task. Based on prior works, we proposed a
new attention mechanism and describe the details as follows.
We denote the hidden state of the LSTM at time t as st .
At each time step t, the input to the LSTM decoder is
computed by
x̂ t = We et −1 + Wu1 u t −1
where et −1 represents the one-shot encoding of the pre-
dicted character at time t-1 (see the red line of the Decoder
in Figure 2 ) and ut is the masked embedding feature vectors
(see the green line of Decoder of Figure 2), computed as

ut = αt,i, j f i, j,c (2)
i, j
We refer to α t as the attention masks at time step t.
First, we compute the relevant importance of each part of the
feature vector f, which is obtained by combining the image
information in f with a time offset st (see the dotted blue line
in Figure 2) and the feature vector f.
αt = so f tmax i, j (tanh(st , f i, j,c ))
The previous hidden state st −1 and the current input x̂ t are
employed to compute the hidden output ot and the next hidden
state st .
Finally, in order to obtain the final probability distribution
over the letters at time step t, we combine the hidden output
information ot of the LSTM with the attentional feature vector
ut , computed as
Pt = so f tmax(Wo ot + Wu2 u t )
According to the probability distribution, we use the argmax
function to choose the most likely character as the final result.
C. Implementation Details
For the encoder, we did not use the complete network
structure of Inception-v3. Instead, we cropped the Inception-
v3 from the first layer to the “Mixed_6d” block for use as
our encoder. The decoder consists of T LSTM cells, each of
which contains 256 hidden units. The number of LSTM cells is
equal to the max string length of the CAPTCHA. For the entire
network structure, the number of trainable parameters reaches
8,459,676. Depending on the original size of the different
CAPTCHAs categories, we rescaled each CAPTCHA image
so that the short edge was in the range of 100 to 200 pixels.
In training phase, the model is trained by stochastic gradient
descent (SGD) with a momentum of 0.9. The learning rate is
set to 0.004. In addition, we trained 200 epochs with a batch
size of 64.
After each epoch, the model is tested on the validation set.
If the performance of the model is better than that of the
Authorized licensed use limited to: NATIONAL INSTITUTE OF TECHNOLOGY SURATHKAL. Downloaded on September 13,2022 at 15:13:58 UTC from IEEE Xplore. Restrictions apply.
IEEE TRANSACTIONS ON INFORMATION FORENSICS AND SECURITY, VOL. 15, 2020
Fig. 3. Samples of the difficult version of the Google CAPTCHA.
current best model, the weights of the best model will be
updated, and the number of the current epoch will be recorded.
In the testing phase, the best model is used to run an attack
on the test samples.
IV. E XPERIMENTS
(1)
A. Sample of a Large-Scale Attack on the Google CAPTCHA
We collected a typical crowding-characters-together (CCT)
CAPTCHA, a difficult version of the Google CAPTCHA, as a
representative subject for our attack, as shown in Figure 3.
We summarize the features of the Google CAPTCHA as
follows:
• Every challenge is derived from the lowercase alphabet
(26 lowercase letters)
• Varied CAPTCHA lengths are used, and the CAPTCHA
lengths range from 8 to 10.
• CCT is the main segmentation resistance mechanism.
• Large degrees of rotation and distortion are applied.
This version contains many user-unfriendly challenges. For
(3) example, Figure 3(a) might be recognized as “shmomnc”,
while the answer is “shmoranc”, and 3(b) might be recognized
as “deremas”, while the answer is “cleremas”. As a result,
the difficulty of recognizing this version of Google CAPTCHA
is extremely high.
Previous studies have tried to attack this version of the
Google CAPTCHA. In 2011, [16] successfully broke 13 of
15 text CAPTCHAs but failed (0% success rate) to break
the Google CAPTCHA. In 2015, [12] broke the Google
(4) CAPTCHA with a success rate of 94.5%. However, the method
involves seven steps before the recognition stage and is very
complicated. In 2016, [4] broke this CAPTCHA using log-
Gabor filters, but their success rate was only 7.8%.
Our attack consists of two phases: the training phase and
the testing phase. During the training phase, we collected
10,000 CAPTCHA samples from the Google website in
July 2017, which were divided into a training set and a
validation set at a ratio of 3:1. Then, the original dataset
was expanded 5, 10, 15 and 20 times by including newly
collected samples to build another four datasets. The answer
of each CAPTCHA sample in these datasets was labeled
manually. To avoid human error, we double checked the
answers. For each of these datasets, an individual model was
trained independently.
In the testing phase, another 3,000 samples were used.
We used the full sequence accuracy to benchmark the per-
formance of our model. In other words, for a CAPTCHA
sample in the test set, the full predicted character sequence was
considered to be correctly resolved only if it was identical to
the manually labeled answer. We trained five individual models
with different numbers of samples during the training phase.
 ZI et al.: END-TO-END ATTACK ON TEXT CAPTCHAs 757

TABLE I
R ESISTANCE M ECHANISMS OF TARGETED CAPTCHA S CHEMES

The testing samples were input to each of the five trained
models respectively to output the final prediction.
With 10,000 training samples, a success rate of 42.7%
was achieved. As the number of image samples of the train-
ing set increased to 50,000, 100,000, 150,000, and 200,000,
the success rate increased to 87.9%, 94.5%, 97.4%, and 98.3%
respectively. The average processing time per image was less
than 0.13 seconds, which is 2 times faster than [12] and
62 times faster than [4]. Obviously, despite the complexity and
user-unfriendliness of the Google CAPTCHA, our method has
achieved excellent results in both success rate and speed.
B. Large-Scale Attack
To evaluate the effectiveness of our attack more compre-
hensively, we chose sixteen CAPTCHA schemes deployed
by eleven websites, including Wikipedia, Baidu, QQ, Sina,
Weibo, Jd, 360, Apple, Yandex, Alipay and Microsoft, which
are ranked in the top 50 according to the Alexa.com traffic
rankings as of February 2018. Their design features cov-
ered almost all current mainstream resistance mechanisms,
as shown in Table I.
For each scheme, we collected 10,000 CAPTCHA images
from the corresponding website and manually labeled their
answers. Of these, 8500 samples were used for training,
1000 for validating and the remaining 500 for testing. We used
the method described in Section IV(A) to calculate the final
success rates. Table II summarizes the attack results for every
CAPTCHA scheme. Specifically, Baidu, Jd and 360 deployed
more than one kind of CAPTCHA, so we performed our attack
on their different schemes separately. The final success rates
ranged from 74.8% to 97.3%, proving we broke all target
schemes successfully. The average attack speeds ranged from
0.08 to 0.23 seconds. These results not only far exceeded the
speed of humans but also surpassed those of most traditional
methods that are based on segmentation (6.22 seconds per
image for [3], 15 seconds for [4], and 9.05 seconds for [13]).
C. Comparison With Prior Works
Table III compares the results of our attack with those
of previous methods. Obviously, our results outperformed
all other CAPTCHA crackers. Moreover, our attack doesn’t
need any segmentation or preprocessing process, unlike tradi-
tional text CAPTCHA breaking techniques, thus making our

Authorized licensed use limited to: NATIONAL INSTITUTE OF TECHNOLOGY SURATHKAL. Downloaded on September 13,2022 at 15:13:58 UTC from IEEE Xplore. Restrictions apply.
 758
TABLE II
ATTACK R ESULTS ON THE CAPTCHA S CHEMES
D EPLOYED BY 11 W EBSITES
TABLE III
C OMPARISON W ITH P REVIOUS W ORKS
approach simple and efficient. For any type of target text
CAPTCHA scheme, we only need to collect 10,000 samples
to train the model. Most importantly, our method is highly
robust. No previous work has achieved such great success in
such large scale text CAPTCHA schemes.
Compared to Karthik and Tang’s works, the biggest advan-
tage of our approach is that it recognizes a CAPTCHA
image without any preprocessing or segmentation. Therefore,
the errors introduced by these two steps can be avoided.
Through training, our model is able to focus on the font
features and filter out redundant visual information such as
noisy arcs or rotation angles. Therefore, our method far
surpasses prior pipeline methods in both accuracy and speed.
Compared to Le’s work, our method performed better for
two reasons. First, we used a more advanced CNN structure,
Inception-v3, which is wider and deeper in network structure.
Therefore, our encoder is better in its representation capability.
Second, the attention mechanism is combined with the LSTM
during the decoding phase. This allows our model to focus on
the most relevant information and makes the output at each
time step more explicit. This explains our better success rate
than that of Le’s work for Wikipedia’s CAPTCHA.
D. Attack With Different Number of Training Samples
Despite the good results in both accuracy and speed,
our approach has one drawback: manually labeling
10,000 CAPTCHA samples is not an easy task. In theory,
Authorized licensed use limited to: NATIONAL INSTITUTE OF TECHNOLOGY SURATHKAL. Downloaded on September 13,2022 at 15:13:58 UTC from IEEE Xplore. Restrictions apply.
IEEE TRANSACTIONS ON INFORMATION FORENSICS AND SECURITY, VOL. 15, 2020
TABLE IV
ATTACK R ESULTS W ITH D IFFERENT N UMBER OF T RAINING S AMPLES
deep learning models perform better when using more training
samples. To strike a balance between efficiency and accuracy,
we conducted a series of supplementary experiments. For
each CAPTCHA scheme in Table I, we set the number
of samples used for training to 2,000, 4,000, 6,000 and
8,000 respectively. The ratio between the training set and the
validation set is 3:1. With these different numbers of training
samples, we ran our attack on each scheme respectively.
The attack results are listed in Table IV. Table IV shows
that, with 2,000 real-world samples, out method achieved
success rates over 40% on 13 out of 16 forms of CAPTCHAs.
With 4,000 real-world samples, our method achieved suc-
cess rates over 70% on 13 out of 16 CAPTCHAs. The
CAPTCHA schemes with lower success rates tend to adopt
more resistance mechanisms. For example, each of the two
CAPTCHA schemes of the 360 company adopted a total
of seven resistance mechanisms, which greatly increased the
variations of character features.
To make the trend of the model’s success rates more
explicit, we present the relationship curve between the success
rate and the number of training samples of each CAPTCHA
scheme in Figure 4. Overall, the success rate increases as
the number of training samples increases. For some schemes,
as the number of samples increases, the success rates decreases
slightly, which is due to the difference between the samples
distributions of training set and testing set.
All current CAPTCHA cracking works require the manual
labeling of samples. The sample size of 2,000 to 4,000 are
relatively small-scale for deep learning techniques. If we are
pursuing a balance between accuracy and efficiency, maybe
the use of 4,000 samples is a better choice.
 ZI et al.: END-TO-END ATTACK ON TEXT CAPTCHAs 759

Fig. 4. The effect of the number of training samples on the attack success
rate.

V. C OMPREHENSIVE A NALYSIS
In this section, we first introduced an automatic CAPTCHA
generation system. By imitating the features of real-world
CAPTCHA schemes, we generated five different CAPTCHA
schemes and implemented our attacks on the corresponding
real-world CAPTCHAs. Then, we explored whether it is
possible to train the model with one CAPTCHA scheme and
use the trained model to break another scheme. Next, we com-
prehensively studied the security of the most commonly used Fig. 5. The framework of our CAPTCHA generation system. The system
resistance mechanisms and some special mechanisms such as first generates single characters according to pre-defined parameters; next,
it embeds generated characters into a background image; finally, the system
the mix-background scheme, stylization and two-layer struc- selectively adds global interference.
ture. Taking the Chinese CAPTCHAs as examples, we also
studied the security of large alphabet CAPTCHAs. Finally, TABLE V
we trained a generic model applicable to multiple CAPTCHA ATTACK R ESULTS W ITH S YNTHETIC CAPTCHA S AMPLES
categories.

A. CAPTCHA Generation System

Synthetic data is often used as a replacement for real-world
CAPTCHA samples to reduce costs. For example, [13], [17]
and [26] achieved good results by using their own generated
CAPTCHAs: [13] mimicked the Microsoft CAPTCHA; [17]
used the Cool PHP CAPTCHA framework to generate black
and white images; and [26] generated single Chinese character
images with noisy arcs.
Compared to the above methods, our generation system
can imitate most of real-world CAPTCHA schemes, includ-
ing all mechanisms listed in Section II. The automatically
production process is shown in Figure 5. The basic working Character sets. For CAPTCHAs that are based on Roman
principle of our system is simple. First, generate individual characters, the character set contains 62 categories (10 digits,
characters. Then, local interference can be added to individual 26 lowercase letters and 26 uppercase letters). For Chinese
characters, or individual characters can be inserted into the CAPTCHAs, we used the GB2312 character set which con-
background image. Finally, the global interference can be tains 3755 commonly used Chinese characters.
added to the whole CAPTCHA image. Fonts. Most of the CAPTCHAs that are based on Roman
It should be noted that for different CAPTCHA schemes, characters uses the bold Microsoft YaHei font as the default
we used different character sets and character fonts. font, except those adopting the multi-font mechanisms.

Authorized licensed use limited to: NATIONAL INSTITUTE OF TECHNOLOGY SURATHKAL. Downloaded on September 13,2022 at 15:13:58 UTC from IEEE Xplore. Restrictions apply.
 760
TABLE VI
ATTACK R ESULTS OF BASELINE M ODELS AND F INE -T UNED M ODELS
For Chinese CAPTCHAs, the font library contains three fonts:
KaiTi, SimSun and Microsoft YaHei.
B. Attack With Synthetic CAPTCHA Samples
In this section, we attempted to attack the real-world
CAPTCHA schemes with synthetic samples. We chose the
five most complicated CAPTCHAs in Table I as our tar-
get schemes, including those of QQ, Sina, 360, Apple, and
Microsoft. According to the resistance mechanisms of each
scheme, our CAPTCHA generation system imitated the fea-
tures of these CAPTCHAs and generated corresponding sam-
ples. To improve the model’s generalization ability, for each
scheme, we adopted 150,000 synthetic samples (100,000 as
the training set and 50,000 as the validation set) to train the
model.
The attack results of models trained on synthetic samples
are listed in the ‘Imitation model’ column in Table V. For
comparison, the attack results of models trained on real-world
samples are listed in another column named ‘Baseline model’.
The success rates of the imitation models ranged from 3.8%
to 85.7%. The accuracy of the QQ’s CAPTCHA was the
lowest. Obviously, the similarity between our synthetic QQ
scheme and the real-world scheme was not good enough for
our attack. Note that for the Apple’s CAPTCHA, our imitation
model’s performance even outperformed the baseline model.
The success rates of most imitation models are much lower
than those of the corresponding baseline models. The reason
behind this result was that the tiny difference between the
synthetic samples and the real-world samples will enlarge
greatly though multiple activation layers and cause an output
error.
To correct the error caused by the difference between the
synthetic and the real-world samples, we used a small number
(500) of real-world samples to fine-tune the imitation model.
The results are shown in the last column in Table V. The
success rates of the fine-tuned models range from 77.6%
to 90.9%. The performances of the fine-tuned model are
comparable to or even better than those of the baseline model.
Using synthetic samples to break real-world CAPTCHA
schemes requires fewer real-world samples than does the
baseline method. However, imitating real-world samples is
actually very time-consuming. For example, to imitate the
360’s CAPTCHA, we had to consider a series of factors,
including the angle of rotation, font style, degree of over-
lapping and background interference. The parameters of the
system had to be adjusted manually before generating the
Authorized licensed use limited to: NATIONAL INSTITUTE OF TECHNOLOGY SURATHKAL. Downloaded on September 13,2022 at 15:13:58 UTC from IEEE Xplore. Restrictions apply.
IEEE TRANSACTIONS ON INFORMATION FORENSICS AND SECURITY, VOL. 15, 2020
synthetic samples. Therefore, the time cost of using synthetic
samples to break CAPTCHAs may be even higher.
C. Generalization of the Baseline Models
To further test the generalization ability of our model,
we used a model trained on one CAPTCHA scheme to attack
another scheme. The model trained on the original scheme
is called the baseline model. According to the attack results
in Table II, we selected the two schemes with the lowest
success rates (QQ and Microsoft) and the two schemes with
the highest success rates(Jd and Baidu) as our experimental
subjects.
First, the baseline model of each scheme was used
directly to attack the other three targeted schemes. As shown
in Table VI, without exception, the success rates were all
zero. In fact, the features of these four schemes are very
distinct. Therefore, it is challenging for our baseline models
to address totally different CAPTCHAs. Next, using the fine-
tuning strategy, each baseline model was optimized with
1,000 real-world samples. The attack results of the fine-tuned
models are also listed in Table VI.
After fine-tuning, the performance of the model was sig-
nificantly improved. We found that the growth rates of the
simple targeted schemes were much higher than those of the
complicated schemes due to the diversity between the different
schemes.
In conclusion, only if the target scheme is used to train the
model, or the training samples are very similar to the target
samples, can our method achieve a good performance.
D. Security of Normal Resistance Mechanisms
To further examine the effectiveness of the mainstream
resistance mechanisms in text-based CAPTCHAs, we com-
bined different resistance mechanisms to generated seven
kinds of CAPTCHA schemes. The character set contained
62 categories (10 digits, 26 lowercase letters and 26 uppercase
letters). As shown in Table VII, the number of resistance
mechanisms increased from 2 to 8.
For each scheme in Table VII, we generated 150,000 sam-
ples as the training set and 50,000 samples as the validation
set. By using the samples from different schemes to train the
model, we obtained seven trained models. Then, 3,000 testing
samples of each scheme were input to the corresponding
model to output the prediction answers. The experimental
results are listed in Table VII. As the complexity of the
CAPTCHA increased, the success rate decreased from 96.1%
 ZI et al.: END-TO-END ATTACK ON TEXT CAPTCHAs 761

TABLE VII
ATTACK R ESULTS ON D IFFERENT R ESISTANCE M ECHANISM C OMBINATIONS

to 3.3%, indicating that the use of combinations of multiple TABLE VIII

resistance mechanisms dose enhance the security of the text- E FFECT OF H OLLOW AND VARIABLE -L ENGTH R ESISTANCE M ECHANISM
based CAPTCHA. According to the Bursztein [16] criterion,
all CAPTCHA schemes listed in Table VII were broken by
our method successfully.
The first five resistance mechanisms (overlapping, rota-
tion, distortion, hollow and variable length) have little effect
on the recognition success rate. However, as the multi-font
mechanism was adopted, the success rate dropped sharply
from 88.7% to 13.5%. Such a huge reduction indicated that
using multiple fonts in CAPTCHA design is an effective
strategy. Finally, when the CAPTCHA adopted noisy arcs
and background interference, the accuracy dropped slightly,
proving that these CAPTCHAs were almost useless to defend
against our deep learning based attack.
Among all the schemes, the adoption of the hollow and
variable-length strategies slightly increased the attack success
rate, while the adoption of the multi-font scheme led to a sharp
drop in attack accuracy. To analyze the reason behind these
anomalies, we performed some further experiments.

1) Effect of Hollow and Variable-Length Mechanism: To

eliminate the effects of other resistance mechanisms, we only
combined two resistance mechanisms, at most, in our supple-
mentary experiments. First, the CAPTCHA generation system
generated four basic CAPTCHA schemes, each of which only
adopted one resistance mechanism, including the overlapping,
distortion, multi-font and rotation mechanisms. Then, the hol-
low or variable-length mechanism was added to each of these
basic CAPTCHA schemes, respectively, to obtain another
eight CAPTCHA schemes.
Experimental results concerning the effectiveness of the hol-
low and variable-length mechanisms are listed in Table VIII.
Without exception, for every basic CAPTCHA scheme that
used one resistance mechanism, once the hollow mechanism
was added, the success rate increased to some extent. The
reason for this anomaly is the inner working mechanism of our
method. Unlike traditional segmentation-based approaches,
our character recognition process relies on the outline features
of each character. The hollow mechanism made the outline of
each character category clearer, which reduced the difficulty
of the recognition task for our method. In other words,
the hollow mechanism actually weakens the security of text-
based CAPTCHAs against deep learning attacks.
According to the results in Table VIII, the variable-length
mechanism also seemed to have a negative effect on the secu-
rity of CAPTCHAs. However, by comparing the statistics of
variable-length and fixed-length CAPTCHA samples, we real-
ized that when we adopted the variable-length mechanism in
our experiment, the average length of the CAPTCHA was

Authorized licensed use limited to: NATIONAL INSTITUTE OF TECHNOLOGY SURATHKAL. Downloaded on September 13,2022 at 15:13:58 UTC from IEEE Xplore. Restrictions apply.
 762
Fig. 6. The attack success rates on CAPTCHAs that adopt different number
of fonts.
shorter. For fixed-length CAPTCHAs, the length was set to
8, while the variable-length CAPTCHAs had lengths ranging
from 4 to 10, with an average of 7. In fact, in the decoding
process of our attack, the model recognizes one character
at a time and sequences all predicted characters as the final
answer. Therefore, the final recognition accuracy is inversely
proportional to the CAPTCHA length.
2) Effect of the Multi-Font Mechanism: In Table VII, a total
of 230 fonts was used to generate CAPTCHA samples to study
the effect of the multi-font mechanism. In the real world, it is
unlikely to adopt so many fonts in CAPTCHAs. We tested
the security of six CAPTCHA schemes that adopted 1, 10,
50, 100, 200 and 230 fonts, respectively. Figure 6 shows the
attack results. When the number of fonts increased from 1 to
230, the success rate dropped from 99.2% to 42.9%. The
adoption of the multi-fonts mechanism made the shape of
each character category more diverse, greatly increasing the
difficulty of character recognition. The experimental results
showed that the multi-font mechanism is indeed an effective
mechanism to resist deep learning attacks.
3) Usability of Complex CAPTCHA Schemes: Note that the
success rates of the last two CAPTCHA schemes in Table VII
were only 7.7% and 3.3%, respectively, but these two schemes
were also very unfriendly and unrecognizable to human
beings. To comprehensively analyze the usability of the last
two schemes in Table VII, we did a supplementary experiment
and evaluated the usability of these two schemes from two
aspects:
• Accuracy. It is measured by the percentage of passed
tests in all attempts.
• Response time. It means the time cost for the users to
recognize the characters in the CAPTCHA and submit
their answers.
The two schemes were deployed online respectively. For
each scheme, 50 volunteers were invited to take partici-
pate in at least 30 tests. The experimental results are listed
in Table IX. According to [27], the user accuracies of most
Authorized licensed use limited to: NATIONAL INSTITUTE OF TECHNOLOGY SURATHKAL. Downloaded on September 13,2022 at 15:13:58 UTC from IEEE Xplore. Restrictions apply.
IEEE TRANSACTIONS ON INFORMATION FORENSICS AND SECURITY, VOL. 15, 2020
TABLE IX
U SABILITY A NALYSIS OF THE T WO C OMPLEX CAPTCHA S CHEMES
TABLE X
ATTACK R ESULTS ON S PECIAL R ESISTANCE M ECHANISMS
visual perception based CAPTCHAs range from 70% to 98%
and the response time range from 6.8 seconds to 13 seconds.
Despite the response time of these two schemes is acceptable,
the user accuracy of 9.9% and 5.2% reflect that the two
schemes are awful in usability, which means such kind of
CAPTCHAs is unlikely to be used in the real world.
E. CAPTCHAs With Special Resistance Mechanisms
In addition to the common resistance mechanisms summa-
rized above, we also studied three special resistance mecha-
nisms, including multiple background interference, stylization
and two-layer structure.The experimental results are listed
in Table X.
1) Multiple Background Interference CAPTCHAs: For each
CAPTCHA image, two interferences were adopted in the back-
ground image. Our success rate on this scheme was 93.8%,
and average attack speed was 0.14 seconds, demonstrating that
the multiple background interference mechanism still cannot
resist our attack.
2) Stylized CAPTCHAs: Neural style transfer changes the
texture of a whole image, while preserving its content.
Tang [6] first applied this idea to click-based CAPTCHAs
to better fuse foreground objects into the background image.
We combined the networks proposed by [28] and [29] as
our neural style transfer network. We selected the well-known
artwork Composition X as the style image to generate stylized
CAPTCHA.The success rate of 97.8% shows that the neural
style transfer had little effect on improving the security of
text-based CAPTCHAs.
3) Two-Layer CAPTCHAs: Two-layer scheme means to
arrange characters into two rows. This design aims at increas-
ing the difficulty of segmentation. To study the security of the
two-layer scheme against our deep learning attack, we imitated
two-layer CAPTCHA scheme of Microsoft and ran our attack
on it. The final success rate we achieved was 90.5%, and the
 ZI et al.: END-TO-END ATTACK ON TEXT CAPTCHAs
TABLE XI
ATTACK R ESULTS ON C HINESE CAPTCHA S
average processing time was 0.14 seconds. Therefore, the two-
layer structure is also ineffective against deep learning attacks.
F. Large-Alphabet CAPTCHAs
The character sets of large alphabets, such as Korean,
Japanese and Chinese, are much larger than the Roman
alphabet. Intuitively, a larger alphabet will increase the solu-
tion space, thus making it much more difficult to break the
CAPTCHA.
Taking Chinese CAPTCHAs as an example, the total num-
ber of Chinese characters exceeds 20,000. Compared with
those of Roman characters, the Chinese character strokes are
more complicated, and many characters are very similar. This
further increases the difficulty of the automatic recognition
task. In this subsection, we examined whether larger alphabets
are resistant to deep learning attacks.
We used the GB2313 character set, which includes 3755 of
the most commonly used characters, as the character library.
Three Chinese CAPTCHAs were generated (see the first
column of Table XI), which mimicked those of Baidu,
BotDetectTM-Chess and Tianya, respectively. Their charac-
teristics are listed in the second column of Table XI.
We achieved 96.5%, 99.6% and 99.9% success rates on the
three Chinese CAPTCHAs, respectively, with a small number
of training epochs. The results showed that although there
are many more Chinese characters than Roman characters,
the Chinese CAPTCHAs are also easy to break. Compared
with [30], which required a huge cost to extract individual Chi-
nese characters, our attack had better accuracy and efficiency.
G. Selective Recognition of Text-Based CAPTCHAs
In general, common text-based CAPTCHA schemes require
users to recognize all characters to pass the test. In some
cases, however, the mechanism requires that only some of the
characters in the image be identified.
Figure 7 displays two different CAPTCHAs schemes. The
left one is a Roman-character CAPTCHA that requires users to
only input uppercase letters. The right is a Chinese CAPTCHA
scheme. Some of the Chinese characters are inverted. The
requirement is to recognize all upright characters to pass the
test.
For this type of mechanism, we do not need to make
any changes to our network. We just ignore the unwanted
Authorized licensed use limited to: NATIONAL INSTITUTE OF TECHNOLOGY SURATHKAL. Downloaded on September 13,2022 at 15:13:58 UTC from IEEE Xplore. Restrictions apply.
763
Fig. 7. Visualization of the attention mask.
characters when labeling the training samples. For example,
the label of the left sample in Figure 7 was ‘QCY’. The success
rates for the two schemes were 99.7% and 99.9%, respectively.
There are two reasons why we achieved such a high success
rate. First, the attention mechanism allows our model to
focus on the most relevant feature information and filter out
irrelevant information. Second, our model relies on the font
features of training samples, so our model can recognize
specific characters and ignore other unwanted characters.
To describe how the attention mechanism works in pre-
dicting individual characters, we utilized a method similar
to that proposed by [31] to visualize the attention masks
on the two CAPTCHAs. The attention visualization result is
shown in Figure 7, with the highlighted parts representing the
most relevant image regions for the current prediction. For the
sample in 7 (a), the requirement was to output only uppercase
letters, and the answer was ‘QCY’. In the first step, our model
focused attention on the uppercase letter ‘Q’. In the second
step, the attention was focused on the next uppercase character
‘C’ rather than the lowercase character ‘a’. In the training
phase, our model did not learn the font feature of character
‘a’. Therefore, in the testing phase, our model treated character
‘a’ as background and ignored it directly. In the third step, our
model shifted attention to the next uppercase character ‘Y’ and
ignored the numeric character ‘3’. The recognition process
of the scheme in 7 (b) is similar. At each step, our model
only focused attention on upright characters and bypassed all
inverted ones directly.
H. A Generic Model
Our attack has achieved great success on various
CAPTCHA schemes. However, for different CAPTCHA
schemes, we need to collect the corresponding samples and
train the model separately. Considering that there are hundreds
of text-based CAPTCHA schemes in the real world, it would
be desirable to train a generic model that can break multiple
CAPTCHA schemes.
To improve the robustness of the attack model to various
text CAPTCHAs, we applied all the resistance mechanisms
listed in Section II to generate ten CAPTCHA schemes with
distinct features as our training samples (as listed in Table XII).
We used 200,000 samples of each CAPTCHA to train the
model. The model was trained for 300 epochs. For each
scheme, we prepared another 3000 samples for testing.
 764
TABLE XII
ATTACK R ESULTS OF THE G ENERIC M ODEL ON S YNTHETIC S AMPLES
The success rates on different schemes ranged from 58.7%
to 96.7%, with an average speed of 0.12 seconds. The experi-
mental results demonstrated that it is possible to break multiple
kinds of CAPTCHA with one generic model. However, it is
unclear whether the existing generic model is applicable to
the new CAPTCHAs that have not been included in the
training process. To examine this idea, we used the other five
CAPTCHA schemes in Table II. For each scheme, we prepared
500 test samples and ran the attack respectively. Unfortu-
nately, the generic model did not perform well on the new
CAPTCHAs.
Then, for each scheme, we fine-tuned the generic model
with 1,500 corresponding real-world samples. Unsurprisingly,
the fine-tuned generic model achieved a higher success rate
again, ranging from 65.0% to 86.4%, as shown in Table XIII.
This result indicated that a generic model can indeed attack
a variety of CAPTCHA schemes, but when dealing with new
CAPTCHA schemes, the generic model needs to be fine-tuned
with the target scheme samples first.
VI. CAPTCHA S IN THE F UTURE
Currently, most of the CAPTCHAs used are still text-
based CAPTCHAs. Since its inception, the text CAPTCHA
has been continuously update in the confrontation between
CAPTCHA designers and attackers. In 2005, [32] established
Authorized licensed use limited to: NATIONAL INSTITUTE OF TECHNOLOGY SURATHKAL. Downloaded on September 13,2022 at 15:13:58 UTC from IEEE Xplore. Restrictions apply.
IEEE TRANSACTIONS ON INFORMATION FORENSICS AND SECURITY, VOL. 15, 2020
TABLE XIII
ATTACK R ESULTS OF G ENERIC M ODEL , BASELINE M ODEL AND
F INE -T UNED M ODEL
the segmentation-resistance principle, which has become the
cornerstone for designing text CAPTCHAs. For a long time,
how to extract individual characters from a CAPTCHA image
has been the primary task for attackers. However, using deep
learning techniques, we overturned this principle. Under the
premise of ensuring usability, the text CAPTCHA has little
room for increased security.
Based on the experimental results, we presented the possible
future directions of text-based CAPTCHAs. According to the
experimental results shown in Table VII, the multi-font mech-
anism dramatically reduced the success rates of our attack.
The variation of character fonts increased the difficulty of the
recognition process. This indicates that resisting recognition,
rather than resisting segmentation, will be the likely develop-
ment direction in the future. If we can find an effective method
to increase the recognition difficulty, then the text CAPTCHA
will still be applicable. For example, [33] and [34] described a
pixel-label distortion process that produced adversarial exam-
ples which are almost visually identical to the original. The
difference is not obvious to humans, but all neural networks
in their study misclassified these images. Upon adding slight
perturbation to images, pandas are classified as gibbons. This
process can be applied to text-based CAPTCHAs, as it can
mislead a classifier, while the image remains easily identifiable
to humans.
Other studies have shown that synthetic artifacts can be
created to confuse a recognition network. Recently, [35]
proposed a misclassification attack against state-of-the-art face
recognition systems by producing physical eyeglasses on the
face in the image, leading a face recognition system that is
based on a deep neural network to misclassify the subject.
This method can also be applied to text CAPTCHAs by pasting
some physical artifacts to fool the recognition systems.
VII. C ONCLUSION
This paper systematically analyzed the security of text
CAPTCHAs. Our proposed attack is a generic, effective
and end-to-end solution. Using deep learning techniques,
we have successfully broken the difficult version of the
 ZI et al.: END-TO-END ATTACK ON TEXT CAPTCHAs
Google CAPTCHA with a success rate of 98.3%, thereby
outperforming all previous attacks. We also cracked a large
amount of real-world text CAPTCHAs that are deployed by
the top 50 websites as ranked by Alexa.com. The success rates
ranged from 74.8% to 97.3%. The average speed was less than
0.23 seconds per challenge.
We comprehensively analyzed the security of eight com-
mon resistance mechanisms by attacking seven generated
CAPTCHA schemes adopting these mechanisms. We also
performed our attack on some special resistance mechanisms
including the mixed-background scheme, stylized scheme and
two-layer scheme. In addition, our method is not only applica-
ble to common text CAPTCHAs that are based on Roman
characters but also to CAPTCHAs that are based on large
alphabets, such as Chinese. Experiments also proved that
our method is able to perform selective recognition. Finally,
we examined the feasibility of using one generic model to
attack multiple CAPTCHA schemes.
The experimental results presented in this paper proved
that most real-world text CAPTCHAs are not secure, and
the existing resistance mechanisms are also not as effective
as expected. Our experimental results also overturned the
segmentation-resistance principle. Instead, the recognition-
resistance principle may be the development direction for
text-based CAPTCHAs. In addition, new techniques such as
adversarial examples could be used to improve the security of
text-based CAPTCHAs.
We expect that our work will help to promote the security
of text CAPTCHAs. Current text CAPTCHAs are no longer
secure. How to design CAPTCHAs that are resistant to deep
learning attacks while maintaining user friendliness is an open
problem and the subject of our ongoing work.
R EFERENCES
[1] L. Von Ahn, M. Blum, and J. Langford, “Telling humans and computers
apart automatically,” Commun. ACM, vol. 47, no. 2, pp. 56–60, 2004.
[2] H. Gao, W. Wang, J. Qi, X. Wang, X. Liu, and J. Yan, “The robustness of
hollow CAPTCHAs,” in Proc. ACM SIGSAC Conf. Comput. Commun.
Secur., 2013, pp. 1075–1086.
[3] E. Bursztein, J. Aigrain, A. Moscicki, and J. C. Mitchell, “The end is
nigh: Generic solving of text-based CAPTCHAs,” in Proc. WOOT, 2014,
pp. 1–15.
[4] H. Gao et al., “A simple generic attack on text CAPTCHAs,” in Proc.
NDSS, 2016, pp. 1–26.
[5] C. Hong, B. Lopez-Pineda, K. Rajendran, and A. Recasens, “Breaking
microsoft’s CAPTCHA,” Comput. Netw. Secur. Term Projects, MIT,
Cambridge, MA, USA, Tech. Rep. 6.857, 2015.
[6] M. Tang, H. Gao, Y. Zhang, Y. Liu, P. Zhang, and P. Wang, “Research
on deep learning techniques in breaking text-based CAPTCHAs and
designing image-based CAPTCHA,” IEEE Trans. Inf. Forensics Security,
vol. 13, no. 10, pp. 2522–2537, Oct. 2018.
[7] G. Mori and J. Malik, “Recognizing objects in adversarial clutter:
Breaking a visual CAPTCHA,” in Proc. IEEE Comput. Soc. Conf.
Comput. Vis. Pattern Recognit., vol. 1, Jun. 2003, p. 1.
[8] G. Moy, N. Jones, C. Harkless, and R. Potter, “Distortion estimation
techniques in solving visual CAPTCHAs,” in Proc. IEEE Comput. Soc.
Conf. Comput. Vis. Pattern Recognit. (CVPR), vol. 2, Jun./Jul. 2004,
p. 2.
[9] K. Chellapilla and P. Y. Simard, “Using machine learning to break visual
human interaction proofs (HIPs),” in Proc. Adv. Neural Inf. Process.
Syst., 2005, pp. 265–272.
[10] J. Yan and A. S. El Ahmad, “Breaking visual CAPTCHAs with naive
pattern recognition algorithms,” in Proc. 23rd Annu. Comput. Secur.
Appl. Conf. (ACSAC), Dec. 2007, pp. 279–291.
Authorized licensed use limited to: NATIONAL INSTITUTE OF TECHNOLOGY SURATHKAL. Downloaded on September 13,2022 at 15:13:58 UTC from IEEE Xplore. Restrictions apply.
765
[11] J. Yan and A. S. El Ahmad, “A low-cost attack on a microsoft
CAPTCHA,” in Proc. 15th ACM Conf. Comput. Commun. Secur., 2008,
pp. 543–554.
[12] O. Starostenko, C. Cruz-Perez, F. Uceda-Ponga, and V. Alarcon-Aquino,
“Breaking text-based CAPTCHAs with variable word and charac-
ter orientation,” Pattern Recognit., vol. 48, no. 4, pp. 1101–1112,
2015.
[13] H. Gao, M. Tang, Y. Liu, P. Zhang, and X. Liu, “Research on the secu-
rity of microsoft’s two-layer CAPTCHA,” IEEE Trans. Inf. Forensics
Security, vol. 12, no. 7, pp. 1671–1685, Jul. 2017.
[14] PWNtcha. Pretend We’re Not a Turing Computer but a Human Antag-
onist. Accessed: Dec. 4, 2009. [Online]. Available: http://caca.zoy.
org/wiki/PWNtcha
[15] S. Li, S. A. H. Shah, M. A. U. Khan, S. A. Khayam, A.-R. Sadeghi,
and R. Schmitz, “Breaking e-banking CAPTCHAs,” in Proc. 26th Annu.
Comput. Secur. Appl. Conf., 2010, pp. 171–180.
[16] E. Bursztein, M. Martin, and J. Mitchell, “Text-based CAPTCHA
strengths and weaknesses,” in Proc. 18th ACM Conf. Comput. Commun.
Secur., 2011, pp. 125–138.
[17] F. Stark, C. Hazırbas, R. Triebel, and D. Cremers, “CAPTCHA recog-
nition with active deep learning,” in Proc. GCPR Workshop New
Challenges Neural Comput., 2015, pp. 1–8.
[18] T. A. Le, A. G. Baydin, R. Zinkov, and F. Wood, “Using synthetic data
to train neural networks is model-based reasoning,” in Proc. Int. Joint
Conf. Neural Netw. (IJCNN), May 2017, pp. 3514–3521.
[19] D. George et al., “A generative vision model that trains with high
data efficiency and breaks text-based CAPTCHAs,” Science, vol. 358,
no. 6368, p. eaag2612, 2017.
[20] Z. Wojna et al., “Attention-based extraction of structured information
from street view imagery,” 2017, arXiv:1704.03549. [Online]. Available:
https://arxiv.org/abs/1704.03549
[21] C. Szegedy, V. Vanhoucke, S. Ioffe, J. Shlens, and Z. Wojna, “Rethinking
the inception architecture for computer vision,” in Proc. IEEE Conf.
Comput. Vis. Pattern Recognit., Jun. 2016, pp. 2818–2826.
[22] O. Russakovsky et al., “ImageNet large scale visual recognition
challenge,” Int. J. Comput. Vis., vol. 115, no. 3, pp. 211–252,
Dec. 2015.
[23] D. Bahdanau, K. Cho, and Y. Bengio, “Neural machine translation by
jointly learning to align and translate,” 2014, arXiv:1409.0473. [Online].
Available: https://arxiv.org/abs/1409.0473
[24] J. K. Chorowski, D. Bahdanau, D. Serdyuk, K. Cho, and Y. Bengio,
“Attention-based models for speech recognition,” in Proc. Adv. Neural
Inf. Process. Syst., 2015, pp. 577–585.
[25] H. Gao et al., “Robustness of text-based completely automated public
turing test to tell computers and humans apart,” IET Inf. Secur., vol. 10,
no. 1, pp. 45–52, 2016.
[26] J. Chen, X. Luo, Y. Guo, Y. Zhang, and D. Gong, “A survey on breaking
technique of text-based CAPTCHA,” Secur. Commun. Netw., vol. 2017,
Dec. 2017, Art. no. 6898617.
[27] E. Bursztein, S. Bethard, C. Fabry, J. C. Mitchell, and D. Juraf-
sky, “How good are humans at solving CAPTCHAs? A large
scale evaluation,” in Proc. IEEE Symp. Secur. Privacy, May 2010,
pp. 399–413.
[28] L. A. Gatys, A. S. Ecker, and M. Bethge, “A neural algorithm of artistic
style,” 2015, arXiv:1508.06576. [Online]. Available: https://arxiv.org/
abs/1508.06576
[29] J. Johnson, A. Alahi, and L. Fei-Fei, “Perceptual losses for real-time
style transfer and super-resolution,” in Proc. Eur. Conf. Comput. Vis.
Cham, Switzerland: Springer, 2016, pp. 694–711.
[30] A. Algwil, D. Ciresan, B. Liu, and J. Yan, “A security analysis of
automated chinese turing tests,” in Proc. 32nd Annu. Conf. Comput.
Secur. Appl., 2016, pp. 520–532.
[31] K. Simonyan, A. Vedaldi, and A. Zisserman, “Deep inside convolu-
tional networks: Visualising image classification models and saliency
maps,” 2013, arXiv:1312.6034. [Online]. Available: https://arxiv.org/
abs/1312.6034
[32] K. Chellapilla, K. Larson, P. Y. Simard, and M. Czerwinski, “Computers
beat humans at single character recognition in reading based human
interaction proofs (HIPs),” in Proc. CEAS, 2005, pp. 1–8.
[33] C. Szegedy et al., “Intriguing properties of neural networks,” 2013,
arXiv:1312.6199. [Online]. Available: https://arxiv.org/abs/1312.6199
[34] I. J. Goodfellow, J. Shlens, and C. Szegedy, “Explaining and harness-
ing adversarial examples,” 2014, arXiv:1412.6572. [Online]. Available:
https://arxiv.org/abs/1412.6572
[35] M. Sharif, S. Bhagavatula, L. Bauer, and M. K. Reiter, “A general frame-
work for adversarial examples with objectives,” 2017, arXiv:1801.00349.
[Online]. Available: https://arxiv.org/abs/1801.00349
 766 IEEE TRANSACTIONS ON INFORMATION FORENSICS AND SECURITY, VOL. 15, 2020

Yang Zi is currently pursuing the master’s degree in Zhouhang Cheng is currently pursuing the master’s
computer science with Xidian University. His current degree in computer science with Xidian University.
research interest is captcha. Her current research interest is captcha.

Haichang Gao is currently a Professor with the Yi Liu is currently pursuing the master’s degree in
Institute of Software Engineering, Xidian University. computer science with Xidian University. His current
He has published over 30 papers. He is currently research interest is captcha.
in charge of the National Natural Science Founda-
tion of China Project. His current research interests
include captcha, computer security, and machine
learning.

Authorized licensed use limited to: NATIONAL INSTITUTE OF TECHNOLOGY SURATHKAL. Downloaded on September 13,2022 at 15:13:58 UTC from IEEE Xplore. Restrictions apply.