Breaking NoCaptcha Captcha using

Browser Automation

CGS402A Project-1 Report

Prof. Nisheeth Srivastava & Prof. Harish

Karnick

GitHub Repository

190965 — 190923 — 190166

22 January 2023
Contents

1 Problem Statement and Introduction 2

2 Installation 3

2.1 Clone GitHub Repository . . . . . . . . . . . . . . . . . . . . . . . . . . . 3

2.2 Install Node JS to run ReactJS . . . . . . . . . . . . . . . . . . . . . . . . 3

2.3 Install Microsoft Edge . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3

2.4 Install PyAutoGUI and Dependencies . . . . . . . . . . . . . . . . . . . . 3

2.5 Running the Project . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4

3 Approach and Execution 5

3.1 Set-up Testing Website . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5

3.2 Configuring Mouse Movements . . . . . . . . . . . . . . . . . . . . . . . . 5

3.2.1 WindMouse algorithm . . . . . . . . . . . . . . . . . . . . . . . . . 6

3.2.2 Start position: start x, start y . . . . . . . . . . . . . . . . . . . . 6

3.2.3 Target position: dest x, dest y . . . . . . . . . . . . . . . . . . . . 6

3.2.4 Gravity: G 0 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7

3.2.5 Wind: W 0 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7

3.2.6 Maximum step size: M 0 . . . . . . . . . . . . . . . . . . . . . . . 7

3.2.7 Distance of closeness: D 0 . . . . . . . . . . . . . . . . . . . . . . . 7

3.2.8 Resultant motion . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8

3.3 Browser Automation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8

3.3.1 Starting Local Server . . . . . . . . . . . . . . . . . . . . . . . . . . 8

3.3.2 Opening the Browser . . . . . . . . . . . . . . . . . . . . . . . . . . 9

0
 3.3.3 Overall Algorithm . . . . . . . . . . . . . . . . . . . . . . . . . . . 9

4 Results 11

5 Future Work 11

6 Conclusion 12

7 References 12

1
1 Problem Statement and Introduction

By ensuring that only a person with the correct password may access your ac-
count, CAPTCHA protects you from remote digital entry. Computers are able to
generate distorted images and process user input, but they are unable to under-
stand or solve problems in the same way as a human would in order to pass the test.

As bot-based threats have evolved, so have the CAPTCHA mechanisms intended

to stop them. In its early forms, users were asked to read distorted text and submit
it in a form.

This project addresses the issue of utilizing browser automation to overcome

the NoCaptcha captcha. Here, we design automation that nearly simulates the
mouse movement of a human while still having a slight overshoot that explains the
course of the mouse arrow. We have integrated some necessary modules in python
to stimulate the problem’s scenario. We have used the indexing method to locate
the location of the mouse in different instances. We have attempted to address
every scenario that could possibly arise.

2
2 Installation

NOTE: The project can only be reproduced using the microsoft-edge browser Run the
following commands sequentially in the terminal as instructed:

2.1 Clone GitHub Repository

$ git clone https://github.com/AtharvaUmbarkar/CGS-Project-1.git

$ cd CGS-Project-1

2.2 Install Node JS to run ReactJS

$ curl -sL https://deb.nodesource.com/setup_18.x | sudo -E bash -

$ sudo apt update
$ sudo apt install nodejs
$ sudo apt install npm

2.3 Install Microsoft Edge

$ sudo apt update

$ sudo apt install software-properties-common apt-transport-https wget
$ wget -q https://packages.microsoft.com/keys/microsoft.asc -O- |
sudo apt-key add -
$ sudo add-apt-repository "deb [arch=amd64]
https://packages.microsoft.com/repos/edge stable main"
$ sudo apt install microsoft-edge-stable

2.4 Install PyAutoGUI and Dependencies

$ pip install PyAutoGUI

$ pip install opencv-python
$ pip install numpy
$ sudo apt install scrot

3
2.5 Running the Project

From the root of the cloned repository, exeute:

$ python3 ./main.py

4
3 Approach and Execution

3.1 Set-up Testing Website

For testing, we installed the NoCaptcha(ReCaptchaV2 Tickbox) using the Google Re-
captcha API on a plain website created using ReactJS. The captcha was installed using
the following code:

Figure 1: Code to install Recaptcha on the site

The Website was divided into 9 sections as shown below and the ReCaptcha component
is rendered into one of the sections randomly. This allows us to test check-box detection
and mouse automation across multiple iterations.

Figure 2: ReactJS Website for Testing

3.2 Configuring Mouse Movements

The nocaptcha algorithm, along with various other factors, takes into account the move-
ment pattern of the cursor to decide whether the response is from a human or a bot. To
tackle this issue, we had to automate the movement of the mouse cursor the way a human
might move. Looking into various algorithms we first tried the Bezier curve method, but
the speed and trajectory did not give a human-like movement, so we discarded it and
moved to the second algorithm, the Wind-Mouse algorithm.

5
3.2.1 WindMouse algorithm

The algorithm focuses on the fact that a human moving a mouse rarely follows a straight
line. Instead, the motion has significant amount fluctuations throughout. This can be
easily seen by using some drawing program like MS Paint and choosing a brush to trace
the movement from one point to another.

Figure 3: Some examples of a real human moving the mouse across the screen,
captured in GIMP.

The WindMouse algorithm is inspired by the idea of the effect of gravitational force
and wind on the motion of an object. The target acts as a huge mass that provides
acceleration to the object (here cursor) towards itself due to gravitational pull. While
the wind exerts random force in a random direction, and smoothly changes both direction
and speed of motion.

The function consists of the following parameters:

3.2.2 Start position: start x, start y

These coordinates are the current position of the cursor during the function is being
called.

3.2.3 Target position: dest x, dest y

These coordinates specify where the cursor should finally move and settle. In our case,
these are the coordinates of the center of the captcha.

6
3.2.4 Gravity: G 0

This parameter signifies the amount of gravitational pull provided by the destination.

The gravitational force is calculated as:

⃗ = G0 ⃗xf − ⃗x
G
|⃗xf − ⃗x|
Where ⃗xf is the destination while ⃗x is the cursor position at that instant. Tweaking this
parameter value controls the acceleration with which the cursor moves in the direction
of the destination.

3.2.5 Wind: W 0

The modeling of wind is what gives the wind mouse algorithm its characteristic behavior.
The wind force is resolved into two separate orthogonal forces, Wx and Wy . Wind force
decides the random disturbance in the motion of the cursor.

The wind force updates depending on the distance of √ the cursor from the target. Far
from the target, the wind force reduces by a factor of 3 along with the addition of a
random fluctuation in value in the range (− W √0 , + W
5
√ 0 ). When the cursor reaches closer
5 √
to the target, the wind force starts reducing
√ with a factor of 5 only, with no random
addition. This value was originally set to 3 by the author, but we increased it as there
were unnecessarily high overshoots and oscillations due to a slower rate of decrease in
wind force near the target.

3.2.6 Maximum step size: M 0

The gravitational force provides constant acceleration to the cursor, which results in
non-humanly high speed to the cursor, which cannot be achieved through human mouse
movement. To tackle this, a velocity clip is applied to limit the highest speed the cursor
can achieve.

3.2.7 Distance of closeness: D 0

The cursor has to at the end converge the motion at the destination. But a high amount
of wind fluctuation and constant gravitational pull results in the cursor oscillating around
the target rather than converging. This problem is resolved by using a proximity region

7
in which the wind force starts converging without any random addition and allows the
cursor to converge without unnecessary oscillations.

Figure 4: Representation of proximity region

3.2.8 Resultant motion

The WindMouse algorithm resulted in a non-machine-like motion with random fluctua-

tions in direction during motion and occasional overshoots.

Figure 5: Some examples of a WindMouse generating random paths across the screen.

3.3 Browser Automation

3.3.1 Starting Local Server

We have used Python Subprocesses module to start the react website using npm(node
package manager) scripts in the background, as follows:

8
 Figure 6: Starting Local Host

The first process installs any dependencies that our testing website has by executing
’npm install’, then we start the local host where the website is hosted by exeuting ’npm
run start’.

3.3.2 Opening the Browser

Figure 7: Opening the Browser

Using the webbrowser module in Python, by passing the browser executable path to the
module we autonomously open the browser session.

3.3.3 Overall Algorithm

The following loop demonstrates the overall flow of our algorithm.

9
 Figure 8: Overall Algorithm

1. Open the target URL. ie. http://localhost:3000/

2. Refresh the window to allow the recaptcha script to load properly, using PyAuto-
GUI’s hotkey function.

3. Locate the center of the desired element ie. the captcha box using locateCen-
terOnScreen which uses OpenCV under the hood and takes a confidence value
and a target image as inputs.

Figure 9: Target Image

4. Using the WindMouse Algorithm move the mouse from current location to the
target location and click on the desired element.

5. Close the current tab using the Close Tab(’ctrl’ + ’w’) hotkey and repeat the loop.

Once all the iterations are finished, we close the browser window using the Close Win-
dow(’ctrl’ + ’shift’ + ’w’) hotkey, and kill the react subprocess.

10
4 Results

Using the above methodology we were able to pass the Recaptcha Challenge successfully
and were able to mimic human mouse trajectory and overshooting accurately.

Heres a Google Drive link to the entire testing video: Link

5 Future Work

These are some of the issues that we faced and also some potential improvements that
can be made on top of our existing stack.

1. The Google Recaptcha API tracks your IP at each captcha verification request.
Hence if we try to verify it rapidly in a short amount of time, it will start asking
for image-based verification. Options like IP routing/masking can be explored to
mitigate this.
2. This algorithm can be integrated with an image-based captcha-breaking algorithm
to handle cases when the captcha system asks for image identification for second-
step verification.
3. Due to the use of PyAutoGUI, the algorithm restricts mouse usage for its duration.
Our initial approach was to use Selenium to fully automate the process without
restricting any GUI, but the attempts were unsuccessful due to the IP tracking as
mentioned above
4. Instead of a controlling algorithm like WindMouse we can explore the implementa-
tion of a mathematical model which takes into account actual human movements
as parameters and trains/builds upon it for better trajectories.

11
6 Conclusion

In this report, we have discussed how we have mimicked the mouse movement of a human
being and are able to break the Nocaptcha. As we have added the future work section to
show some areas where potential improvement can be done like handling a high number
of requests at a single time. If we look into the consumer and business website then it
is clear that a captcha can’t stand to interrupt the user flow and that would ultimately
lead to lower conversions on websites. So, these realities are clearly showing the demand
for new development of advanced mechanisms that don’t frustrate the users and are
difficult for bots to solve.

7 References

1. Google Recaptcha API: https://developers.google.com/recaptcha/docs/display

2. BezMouse, 2017: https://github.com/vincentbavitz/bezmouse.

3. WindMouse, an algorithm for generating human-like mouse motion, 2021:

https://ben.land/post/2021/04/25/windmouse-human-mouse-movement/.

4. PyAutoGUI: https://pyautogui.readthedocs.io/en/latest/quickstart.html

5. Insights into the workings of NoCaptcha: https://stackoverflow.com/questions/27286232/how-

does-google-recaptcha-v2-work-behind-the-scenes

6. Sivakorn, Suphannee. “I’m Not a Human: Breaking the Google reCAPTCHA.”

(2016). Link to Paper

12