Computer Juggling Trainer

CST 499

Chris Carson, John Coffelt, Michael Rose

April 28, 2019

 1

Executive Summary

The project, currently titled Computer Juggling Coach will be a first of its kind. At this
time, there are no other programs that are designed to track the motion of juggling balls as they
are juggled and provide the user feedback on this motion. In addition to being a unique program,
it is also slated to be free to use. As a team, it is hoped that this will be an enjoyable piece of
software that can be used as a training tool, or just for simple entertainment by jugglers of all
skill levels. The goal is to effectively utilize the free and very efficient OpenCV library to
capture data in real time through the user’s webcam. This data will be analyzed by the software
in order to correctly identify catches and throws as well as some of the most common juggling
patterns. This analysis will yield information that will be provided to the user so that he will be
able to better understand what they may be doing correctly and incorrectly. The user should then
be able to modify their form or correct their throws as recommended to successfully achieve
their juggling goals. Basic instruction on how to get started juggling will also be provided in this
application. An additional feature of this software is that it will be able to count consecutive
throws of a pattern and give feedback to the user on how to improve their technique.
Consecutive throws are key to successful juggling. A juggler is not a juggler if they are unable
to throw and catch their juggling balls more than once. The finished project will be the
milestone achievement to-date for the team developing it, and will act as a springboard into
understanding computer vision and its capabilities of being used on the client side in web
browsers. With luck, the experience will help to open doors in the field of computer vision for
the team members involved.

The doors the team members are opening into the field of computer vision will hopefully
be beneficial to other programmers as well. The ability to track a user’s expected and repetitive
motion and compare it to one that is common or standard will hopefully find multiple uses. With
minor changes or even just portions of this newly developed software, it is hoped that other
programmers will be able to use it to effectively develop their own programs. The analysis
portion should also be transportable as well with minor changes. The possibilities are many and
it is the hope of the team members that the Computer Juggling Coach will be just the beginning.

People from all over the globe are invited to dabble with, or consistently use this software
for entertainment and learning. Outside of books and videos that provide no actual feedback to
the end user, the only current way to get feedback is to be in the presence of an experienced
juggler. Since juggling is a niche sport, it’s often difficult for someone with interest in it to find
a competent juggler to provide instruction and feedback. The online Computer Juggling Coach
will solve this dilemma and will provide a fun way to track a juggling student’s progress as they
refine their skills. It also provides a way for people of all ages who are shy, easily embarrassed,
or reluctant to practice in front of others to learn and enjoy the sport. People seem to understand
that an electronic device, in this case a computer, is non-judgmental and can therefore provide
feedback without fear of being laughed at or ridiculed. Juggling is an inexpensive, healthy, and
ancient art that many have not learned due to the difficulties of learning basic juggling patterns.
With this resource available to all, there is bound to be gained interest in this activity by people
from all demographics and ages. The team will monitor the website usage in hopes of
confirming that the site is a fun and valuable resource for all to enjoy.
 2

Table of Contents
Description 3
II. Goals and Objectives 4
III. Environmental Scan 6
IV. Project Objectives and Methodology 7
Primary Objectives and Methodology: 7
Secondary Objectives and Methodology (Time Permitting): 8
V. Stakeholders, Community, and Clients 8
VI. Ethical Considerations 9
VII. Legal Considerations 11
VIII. Timeline and Budget 1​1
i. Timeline/Budget 1​1
a. Timeline 1​1
b. Budget 12
ii. Resources Needed 1​2
iii. Milestones 13
iv. Risks/Dependencies 1​3
IX. Final Deliverables 14
X. Usability Testing/Evaluation Plan 15
i. Executive Summary. 15
ii. Phase 1 Scope 1​5
a. Methodology 1​5
b. Scope 16
c. Equipment 16
d. Metrics 1​6
ii. Phase 2 Scope 17
a. Methodology 17
b. Scope 17
c. Equipment 1​7
d. Metrics 1​7
XI. Team Members 18
References 20
 3

I. Description

Juggling is a form of entertainment that has a very long history. Some historians date it

back almost 4000 years from depictions of jugglers in ancient Egyptian hieroglyphics (The

History of Juggling, 2018). Whatever the history of juggling, it is a skill that requires great

hand-eye coordination and manual dexterity. It is not an easy skill to master. Many people seem

to be interested in juggling from the many books, videos, and juggling artifacts that are available

for sale in stores and on the internet. You buy a book. You buy the video. You buy the juggling

balls. You open the book and read it. You play the video and watch it. You throw the balls up

into the air and you catch one. You drop one. You try again. Most people find that dropping the

ball is a much more common occurrence than catching the ball. You watch the video again.

You read the book again. You try a few more times. You drop the balls a few more times.

Everything gets put away. At your next garage sale, your juggling kit is for sale for one eighth

of the price you paid.

The above scenario is very common when it comes to learning to juggle. What the team

found to be beneficial is to have feedback. Am I throwing the ball high enough? Am I throwing

the ball straight up? Are my hands in the right place to catch the ball? It helps to have a friend

watching you and letting you know, but it is often hard for a person to determine how straight

that ball is going up or if it is truly high enough. It is also often embarrassing to practice and

continue dropping the ball in front of someone else.

The team’s project will help to avoid the embarrassment of learning to juggle in front of a

friend. It will also help to answer the question about how high, straight, and accurately each ball
 4

is thrown. This movement and this ability are not only useful as a form of entertainment, but it

can also be used to help refine motor skills. Throwing objects up and catching them is a

coordination skill that is often used in physical therapies and rehabilitations. With the team’s

project, users can avoid the additional frustration of not knowing how they are throwing the

object because they will be able to see and track the objects being thrown.

II. Goals and Objectives

It is the main goal of this project that the team members learn how to successfully utilize

the OpenCV computer vision library for real world applications. Computer vision is an

exploding new technology that is allowing for endless possibilities of software capturing video

and processing it quickly and accurately. It’s enabling easy access for developers to use facial

recognition, environmental analysis such as self-driving cars or the tracking of customer

behaviors. The future programming market will be in high demand for developers who

understand computer vision libraries and the associated technical terminology and methodology

utilized by them. The team expects to have this project not only listed on their resume for future

employers, but to have a working application widely utilized by the public to demonstrate the

technical skills acquired by the team’s during their time at CSUMB.

An additional goal of the project is to have a public and free to use piece of software that

is both enjoyable, functional and useful. The team is not aware of any resource on the web that

currently provides any sort of feedback for jugglers. This site will be a first, which is very

difficult to achieve in today’s internet culture. Adding to this goal is the hope by the team that
 5

the effort that will be put into this project might help to peek people’s interest in the fun and

healthy sport of juggling across the globe.

There are three main objectives associated with this project. The first being proper

implementation of the OpenCV library in JavaScript so that the application can be run in any

modern browser with no need to download software and no dependencies on specific end user

hardware. The use of the library for capturing data in real time should be highly optimized for

the specific task at hand. This means high frame rate and quick analysis, as juggling is a very fast

paced activity. Initial experimentation with the library has convinced the team this objective is

achievable.

The second objective will be to correctly analyze the data captured from the OpenCV

library in hopes of recognizing throws, catches and many of the basic patterns of juggling. The

team hopes to accomplish this through algorithms of their own design as well as researching data

analysis techniques already in use by the industry. This objective is critical to the overall success

of the application, as faulty or inefficient tracking will severely reduce the quality of the final

end user experience.

The third and final objective is to package the software into an easy and fun to use

website that people of all ages can use and that functions sufficiently well on a wide range of

hardware. The achievement of this objective will rely on the successful completion of two

smaller objective. The first, on creating a clean and intuitive user interface and paying close

attention to the usability testing feedback to further enhance the UI in ways that were not
 6

originally apparent during the initial design. Secondly, on having highly optimized code that

does not require relatively new or fast hardware to run on.

III. Environmental Scan

Jugglers have been developing software to be used a learning tool or to otherwise aid in

exploration of the hobby for decades. This trend has only grown as the internet and easily

shareable videos have become available allowing for juggling to increase in popularity.

Juggling relies on repeatable patterns and these patterns have been committed to formulas

in a juggling specific notation that allows for describing and documenting juggling patterns.

Many software programs have been developed that can receive typed pattern notations and turn

them into visual demonstrations of the pattern. There are also already programs that can visualize

existing patterns and then churn out the notation for that pattern. These programs have been a

great assistance and tool for jugglers, regardless of skill level.

Over the last few years, as the technology has grown, open source software frameworks

for recognizing patterns using real-time video has become available and widely documented.

Following the trend, many jugglers have already applied this software to the hobby and have

used it to track and analyze their juggling patterns in interesting ways. A quick exploration of

any video sharing site will reveal at least a handful of videos demonstrating this. In this sense,

the team’s project will not tread new ground. However, the technology, at this time, has largely

been underutilized in applying this technology outside of prototypes or basic video

demonstrations. There are few, if any, applications that apply these open-sourced libraries in a

way that is both user-friendly and useable by jugglers of various skill levels. It is the intent of
 7

this project to package a user-friendly web-based interface that will receive real-time video of

juggling as input and output analysis in a way that is useful to the user as both a training tool and

as an enhancement that increases enjoyment of the hobby. This sets this project apart from any

similar, currently existing applications.

IV. Project Objectives and Methodology

Primary Objectives and Methodology:

- Create a web-based, user-friendly juggling practice assistance application that

implements the OpenCV (Open Source Computer Vision Library) or similar library.

- User will be able to stream video via webcam or, possibly, mobile camera to the

website. Video will not be recorded or saved.

- Application will track motion of common juggling instruments (juggling balls only

initially) through space via the real-time video stream. For the objective to satisfy other

user-friendliness requirements, this will need to be realistically useable in most home

environments under acceptable lighting conditions with minor to moderate configuration

required by the user.

- Record metrics from the recognition of juggling movement. These metrics would be

related to the x/y positioning of the juggling instruments through time. Analyze those metrics for

data regarding the nature of the juggling pattern the user is performing.

- Output this analysis to the user in a way that is useful and/or interesting. This would

include information on deviation of throws in the pattern over time, showing how consistent each
 8

user’s throw is. A throw in this context represents one toss of a juggling instrument from one

hand to the other. It would also include the pattern that the program believes the user to be

juggling with some level of confidence.

Secondary Objectives and Methodology (Time Permitting):

- The program will also provide visual examples of common juggling patterns to assist

users.

- The tool will include various games that the user can play. This will include games that

score a user based on how closely they could adhere to certain patterns and how consistent the

user is with each of their throws.

- The program will also aid the user in developing consistency by displaying the peak of

each throw for each instrument on the screen, giving the user a target to hit on the following

throws.

V. Stakeholders, Community, and Clients

The project should appeal to many different people. This includes people trying to learn

how to juggle, but also existing jugglers. Those that already know how to juggle should find the

team’s project entertaining and useful for practicing, regardless of their current skill level.

However, since the team’s project will track consistency of motion, it may have appeal outside

of the juggling community or those trying to learn. It could potentially also include those trying

to improve or regain their hand-eye coordination and physical therapists may find it to be a

useful tool. It is also possible that the application of the technology in the team’s project will
 9

inspire others to apply it in ways that were not intended. In this way, the team is hoping to

expand their clientele to a community of not just jugglers, but other programmers that may find

similar uses for the program with just minor modifications. Motion capture and analysis may

have existed, but has not been prevalent in the motion of objects being tossed up in the air and

caught.

There are no external stakeholders in this project at this time. The internal stakeholders,

John Coffelt, Christopher Carson, and Michael Rose, are looking to achieve several goals. The

success of this project will result in the passing of a course at California State University

Monterey Bay and will earn each one a degree. It will also provide great experience and a

positive example of their individual and team skills in the creation of a software project.

Through this achievement, each team member will gain the opportunity to take their skills to

companies where it is hoped that each team member will make significant impacts in the work

they do. The losses from the project should it not be successful would be difficult to bear. The

team’s confidence in the ability to provide a useful tool for jugglers to learn and improve their

skills will be forever shaken.

VI. Ethical Considerations

The following ethical considerations have been noted as part of this project.

1. The application will receive a live video feed from the user’s personal web camera. It is

reasonable to question whether this could intrude on the user’s privacy. As the internet and

social media have become interwoven into the fabric of society, the way that these intrude on

individual privacy has been an increasingly growing concern. Many of the companies behind the
 10

technology platforms that have become ubiquitous in everyday life have shown, at times, a

distinctly cavalier attitude towards protecting the privacy of their users. It is not the intent of this

project for it to contribute towards the erosion of privacy.

While the application does require the live video feed to work, the team will take steps to

mitigate any perception that the user’s desire to maintain their privacy is at the forefront of the

developer’s concerns. The user will have to authorize the application to use their webcam feed.

No video will be saved from the webcam feed nor will it be shared with any outside actors. The

video feed will purely be used for facilitating the features of the application. The metric data

gathered by the application is positional data only and will not include any identifying

information. Also, no long-term usage analytics will be gathered by the application. The team

hopes that this will mitigate any concerns about how the application is handling the user’s

privacy.

2. A significant amount of physical movement is required to use the application effectively.

This could be perceived as the application being ableist in nature. While the team is not in any

way dismissive of this, there is little that can be done to make the application more useable by

those with disabilities that affect the motor skills required to juggle. Juggling is inherently a

physically taxing hobby. Unfortunately, the internet and computer games have historically done

a poor job of serving the disabled community.

This application will not be able to correct those issues. However, it is a hope that, because the

application tracks repeated movements of objects in space, that it could potentially be used as a
 11

tool for physical therapy. The technology being explored in the project could also be applied in

ways that would better serve those with disabilities in the future.

3. Algorithms for recognizing human features in photos and video have shown bias towards

those with lighter skin tones or have not worked as well on those with darker skin tones. This

could be due to how the algorithms were developed and trained over time. This could be

discriminatory towards people of color. To mitigate issues with this, the application is planned

to rely purely on visually tracking physical objects in space and will not attempt to track any of

the physical features of the user.

VII. Legal Considerations

The application will use the OpenCV software library as its foundation. The OpenCV

software is open source and the team will ensure that the application’s usage of the software does

not violate the OpenCV open source licensing agreement in any way.

VIII. Timeline and Budget

i. Timeline/Budget

a. Timeline

Week 1 Implementation of OpenCV - confirm over-all base requirements

Week 2 Design of initial data analysis algorithm(s) / Construction of simple web-based

UI for testing

Week 3 Usability testing feedback / Refinement of algorithm(s) and UI

 12

Week 4 Enhancement of UI / Development of game and training programs

Week 5 Final database and platform migration / Final usability testing

Week 6 Final enhancements and modifications as needed

Week 7 Wrap up any remaining tasks

Week 8 Present Project

b. Budget

The entire project can be completed with zero expense. There is no cost to use the

OpenCV database and the application will run on any browser. The user can use any balls

(regular tennis balls work great) so there should be no cost for user equipment. The only

possible cost might be long term hosting of the finished product which will be decided by the

team after the completion of the class.

ii. Resources Needed

The only specialized resource needed for full implementation is the OpenCV.js library.

The library is fully available under the 3-clause BSD open source license, which places very

minimal restrictions on re-distribution and none on the general use of the library. The project

will be composed of a modern JavaScript web-based application so the development IDE’s, git

usage, and initial hosting will be easily accessible at no expense. The application will be

designed to be calibratable to any juggling ball; tennis balls, oranges, hacky sacks, or any other

juggle-able items already owned by the user, so no special materials are required for use of the

software.
 13

iii. Milestones

The major milestones will correspond to the end of each full development iteration of the project.

Milestone 1:​ Estimated to take place during or around week 2. This will mark the end of the

first full iteration of base functionality and design, making the application ready for Phase 1

testing. The team will have a fully working prototype to further develop and refine based upon

usability and functionality.

Milestone 2: ​ Estimated to take place during or around the end of week 4. This will be the end

of the second full iteration of development. The application should be well tested and fully

functional, including initial game and instruction modes.

Milestone 3: ​ Estimated to take place during week 6. This will be the end of the final and third

full iteration of development. All modifications based upon usability testing should be complete.

The product should be stable enough for deployment to the general public.

iv. Risks/Dependencies

There are two obvious risks to timely completion. The first is not being able to gather the

data from the tracked juggling balls with enough precision to be practically usable for games or

training. The second major risk would be the inability to design an algorithm set that can

consistently recognize throws and juggling patterns. The failure or success of the algorithms will

be highly dependent on the quality and consistency of the data collected. If the team fails at

either of these risk related tasks, the team will be forced to divert their plans and find an

alternative approach. If the team can get past these risks, there are no other notable dependencies
 14

since the framework will simply be a modern JavaScript driven web-based application, which all

three team members have prior experience with.

IX. Final Deliverables

The final deliverable will be a free to use website open to the general public where the

application can be accessed and used in any modern web browser with an attached webcam or

smartphone with camera activated. The application should be able to function in a variety of

environments and should be able to track most any brightly colored object, as long as the color

stands out from the color of the background and the user’s clothing. The only client approval the

team will be seeking is a satisfactory level of usability by the general public. Six weeks is not

enough time to have a highly refined juggling game and training application, but it is the goal of

the team to have a fully functional and easy to use version of the application completed within

the given timeframe.

It is the hope of the team that the website will stay active long after completion of the

course to as act as both; A) a virtual asset of the team’s time and effort at CSUMB and B) a

resource available to the general public to use free of charge. The team is willing to maintain

and continue to host the site after the complete of the Capstone. If there is enough interest in the

site, the team may continue to enhance the site’s capabilities and features, however it is not the

team’s intent to ever attempt to monetize the application.

Although accumulating statistical data is not a base requirement for the project, time

permitting, it is something the team would like to add in order to continuously track the usage of

the site. It would be very interesting to monitor where on the globe the site is accessed from, as
 15

well as the progress that repeat users make in developing their juggling skills. Users will have

the ability to log in to save their progress and high scores, however to keep the ease of use as

simple as possible, creating a user account will be optional.

X. Usability Testing/Evaluation Plan

i. Executive Summary.

The usability testing will be conducted in two phases. Phase one will be focused on making

enhancements to the first working prototype of the website. These enhancements will include

ease of use, refinements to the pattern recognition algorithms as well adjusting the setting of the

software in order for it to adapt to different environments and different trackable objects.

Phase two will be a broader usability test open to the general public via the web. The

core website and functionality will have been developed at this phase, so the feedback will be

used to make minor enhancements to both the functionality, user interface and optimization of

the code.

ii. Phase 1 Scope

a. Methodology

Participants will be limited to close friends and family members who can be closely

observed during testing. No initial instruction will be given, as the software should have clear

and concise directions for the user to follow. The team member conducting the test should be

taking notes continuously as to properly record the feedback during and after testing. Testing
 16

sessions can range from 20 minutes to 1 hour depending on the skill level and interest of the

participant.

b. Scope

Participants will be expected to explore the software and it’s features on their own. Initial

settings and adjustments will be the responsibility of the user. Participants will be encouraged by

the tester to give feedback during and after the testing.

c. Equipment

Testing will take place on a variety of different platforms, including PC, Mac and Smart

devices. Participants are encouraged to test multiple platforms in a single session. Tennis balls

will be provided by the tester for the participant to use, however they are free to use any juggling

objects of their choosing.

d. Metrics

Testing will be concluded after the user has made a reasonable effort to record throws

and catches and has been given feedback from the software. Critical Errors include not being

able to have the application provide any feedback or being unable to get the software

successfully tracking their juggling. Non-critical Errors will be any unexpected frustration or

difficulty on the part of the participant to effectively initialize and use the application.
 17

ii. Phase 2 Scope

a. Methodology

The nearly finished product will be made available to the general public online. The

team will use juggling forums, as well as YouTube to attempt to attract as many willing

participants as possible. There is not target demographic for this phase and the testing will not

be monitored by team members.

b. Scope

Similar to phase 1, participants will be expected to explore and use the application

without guidance. They will be encouraged to provide feedback via a link on the website.

c. Equipment

The participant is welcome to use any personal computer system that supports a webcam

and a modern browser. The juggling equipment will be at the discretion of the use; however.

tennis balls will be recommended due to their excellent visibility.

d. Metrics

Critical Errors for this phase shall be defined as any time a test subject is not able to get

the application functioning, or is not able to receive feedback from it. None-critical Errors

include difficulties using the application, frustrations with the user interface, problems getting it

to accurately track their juggling or over-all poor performance of the application.

 18

XI. Team Members

John Coffelt

Roles:

a) Developing algorithms that will successfully detect throws, as well as more

complex juggling patterns. John will be tasked with refining these

algorithms based upon usability testing and feedback from the team

b) (shared) Designing and coding tutorials and games that will enhance the

experience of the user

c) Code reviewing pull requests from other team members

d) Overseeing design patterns and coding best practices for the project

Michael Rose

Roles:

a) Developing the front-end user interface, including log-in functionality and

making refinements based upon testing

b) (shared) Designing and coding tutorials and games that will enhance the

experience of the user

c) Setup and maintain database for storing user information and tracking user

progress

d) Code reviewing pull requests from other team members

 19

Chris Carson

Roles:

a) Work with and optimize the OpenCV library and create functions capable

of tracking juggled objects in real time and outputting data-sets

b) (shared) Designing and coding tutorials and games that will enhance the

experience of the user

c) Choosing, setting up and maintaining the platform for hosting the website

d) Code reviewing pull requests from other team members

 20

References

The History Of Juggling and The Worlds Best Jugglers. (2018, February 18). Retrieved from
https://jugglealot.com/the-history-of-juggling-and-the-worlds-best-jugglers/

[Sketch of Man Juggling]. Retrieved from

https://www.vectorstock.com/royalty-free-vector/man-juggling-with-balls-sketch-icon-ve
ctor-13230721