See discussions, stats, and author profiles for this publication at: https://www.researchgate.

net/publication/334447010

Artificially Intelligent Assistant for Basketball Coaching

Chapter · January 2020

DOI: 10.1007/978-3-030-24986-1_33

CITATIONS READS

0 203

4 authors, including:

Baris Can Ustundag Tarik Uzunovic

Sabanci University University of Sarajevo
7 PUBLICATIONS   10 CITATIONS    48 PUBLICATIONS   111 CITATIONS   

SEE PROFILE SEE PROFILE

Edin Golubovic
Sabanci University
46 PUBLICATIONS   167 CITATIONS   

SEE PROFILE

Some of the authors of this publication are also working on these related projects:

COST Action CA16116 Wearable Robots for Augmentation, Assistance or Substitution of Human Motor Functions View project

COST Action CA15225 - Fractional-order systems - analysis, synthesis and their importance for future design View project

All content following this page was uploaded by Tarik Uzunovic on 16 August 2019.

The user has requested enhancement of the downloaded file.

 Artificially intelligent assistant
for basketball coaching

Yasin Acikmese1 , Baris Can Ustundag2 , Tarik Uzunovic3 , and Edin Golubovic4
1
Department of Industrial Eng., Galatasaray University, Istanbul, Turkey
yasin.acikmese@ogr.gsu.edu.tr
2
Department of Computer Eng., Istanbul Technical University, Istanbul, Turkey
ustundag16@itu.edu.tr
3
Faculty of Electrical Eng., University of Sarajevo, Bosnia and Herzegovina
tuzunovic@etf.unsa.ba, ORCID: 0000-0002-7979-7395
4
Inovatink, Istanbul, Turkey
edin@inovatink.com, ORCID: 0000-0001-7410-3369

Abstract. Technological advancements in wearable sensors, machine

learning and Internet of Things (IoT) is opening new perspectives on the
understanding of physiological, biomechanical, and psychological mecha-
nisms of human movement in sport disciplines. Utilization of technology
and field expertise allows the development of artificially intelligent (AI)
assistants for sport coaching. This paper considers the development of AI
assistant for basketball coaching. The assistant architecture and design
is explained in detail. The details about the development of knowledge
model for basketball exercise recognition are supplied and experimental
verification of proposed method is done.

Keywords: AI assistant, wearable, machine learning, IoT, basketball,

kinesiology, neural networks.

1 Introduction
The advent of IoT, wearable sensors and machine learning allows for holistic
study of kinesiology (physiological, biomechanical, and psychological mecha-
nisms of human movement). These technologies improve the study of human
movement in the context of rehabilitation, physical therapy, strength and con-
ditioning training, sport, exercise and workplace activity. Promising benefits of
faster progress and improvement through well measured, processed and ana-
lyzed movement, dictate the advancement of research in the application of IoT,
wearable sensors and machine learning to the kinesiology studies.
In kinesiology studies, wearable sensors can be used to quantify performed
movement and provide insight into performance. Machine learning algorithms
offer possibility to both quantify the movement and to provide reasoning assis-
tance to the field experts. Using IoT, large scale databases and cloud based data
analysis and knowledge generation tools can be explored. Ideally, the utilization
of these technology for kinesiology applications would lead to truly autonomous
artificially intelligent (AI) assistant.
 The advancement in AI and human-computer interaction research opened up
new areas of application for AI assistants in education [1], tele-rehabilitation [2],
robotics [3], homes [4], smart phones [5] and entertainment [6]. AI assistants are
also utilized in sports coaching with an aim to assist players during technical
training. Authors in [7] introduce a smart assistant for professional volleyball
training. The assistant utilizes combination of machine learning, statistical player
data, wearable sensors and cameras to address controlling exercise effort and
fatigue levels (technical-tactical) and exercise quality training. AI assistants are
used for encouraging of physical activity [8], [9]. In [8] AI assistant consists of an
activity monitor that tracks user’s location, a portal that provides an overview
of the user’s behavior and a reasoning engine built around computational model
of behavior change. In [9] developed system for personalized physical activity
coaching uses machine learning algorithms to automatize coaching based on the
information about the probability of meeting daily physical activity goal.
Work on automated assessment of tennis swings to improve performance
and safety is done in the context of qualitative coaching diagnostics in [10].
Authors demonstrate the possibility of automation of qualitative analysis of
human motion in context of various tennis training exercises. Similarly authors
in [11] develop coach for golf training using wearable sensor and machine learning
algorithm. The system is used to assist the player in developing golf skills.
In this paper, the architecture, design and implementation of artificially in-
telligent assistant for basketball coaching is presented. This work builds on our
previous research [12]. Developed AI assistant utilizes wearable sensors, machine
learning and IoT technologies to build a complete solution. This paper has two
major results. First, it presents in detail the software architecture necessary to
build AI assistant for basketball coaching. Second, it builds on that architecture
to demonstrate it’s use in basketball training type recognition by utilizing neural
network trained knowledge model. The rest of this paper is organized as follows;
next section gives the overview of the assistant architecture and implementation.
Section 3 gives details about developed knowledge model. Numerical results are
presented in Section 4 and discussion is done in Section 5. Section 6 concludes
the paper.

2 Design of AI Assistant for Basketball Coaching

2.1 System architecture
The architecture of artificially intelligent assistant for basketball coaching is
given in Fig. 1. Assistant perceives environment, the motion of trainee in par-
ticular, through wearable sensors. Data obtained through sensors is filtered to
remove noise and transformed to proper format by data preprocessing archi-
tectural block. Data is converted into knowledge and actionable intelligence by
processing block. Processing block employs knowledge models previously devel-
oped and stored in knowledge models database. Assistant decides which models
to use for data processing depending on the context of the training. Gathering
of short and long term training metrics is done in training metric accumulation
 Fig. 1. Artificially Intelligent Assistant for Basketball Coaching Architecture

block. Knowledge models make use of training metrics to produce useful results.
Feedback block provides audio and visual feedback to the trainees. The accom-
modation of new knowledge is done through new knowledge discovery element.
AI Assistant for basketball coaching makes use of knowledge models to rate
the basketball training exercises and to monitor the progress of trainee. The
knowledge models are developed under the assumption of certain context. For
a model to provide correct results the context must be known. In this work the
basketball coaching context has been divided into levels. The contextual map
of developed assistant is shown in Fig. 2. The developed assistant is used for
sports application of basketball coaching. The improvement in performance of
basketball player is reached through physical, technical, tactical and psychologi-
cal training. At this stage of progress of presented research the Level 3 of context
must be supplied by user. The knowledge models are developed for assistant to
be autonomous below this level, i.e. models must cover the autonomous progress
monitoring and feedback generation for contextual levels 4, 5 and 6.

2.2 System implementation

AI assistant for basketball coaching is implemented through three technological
elements; wearable device, hub and cloud service. Wearable device sends motion
data to the processing hub. Hub communicates with the wearable device, stores
data and prepares it for further processing. The hub runs knowledge models
and feedback algorithms. Hub is Internet connected device capable of storing
data and providing processing offload from cloud in distributed manner. Cloud
service offers data storage, long term monitoring capabilities and utilization of
large scale databases and cloud based data analysis tools. The implementation
of the system is shown in Fig. 3.

Wearable device The operation of AI assistant for basketball coaching depends

on the information about trainee’s activity during training. Necessary informa-
 Fig. 2. Artificial Assistant Contextual Map

tion can be obtained through utilization of motion sensors inside the basketball,
video capturing devices recording player during the training or wearable motion
sensors. The sensor of choice depends on the design requirements of the over-
all system. For the case of work presented in this paper following requirements
are considered; sensor must be cost effective and easily implemented; next to
the standard motion parameters such as position, velocity and acceleration, the
force that hand exerts on the ball should be easily measured or observed; trainee

Fig. 3. System implementation

should be able to utilize sensor both indoors and outdoors under different light-
ing conditions; trainee activity during rest/recovery period should be recorded.
The best fit for these requirements would be the wearable device with motion
sensor.
For the purposes of this work wearable device incorporating motion sensor
was implemented. The wearable device is shown in Figure 4. Sensor, microcon-
troller for signal acquisition and Bluetooth Low Energy (BLE) communication
are coupled together in battery powered wearable device [15] to provide unteth-
ered and trainee friendly way of data collection [12]. Wearable device is capable
of recording acceleration using a 3-axis accelerometer, angular velocity using
3-axis gyroscope and orientation in the form of quaternions. Recorded data is
sent to hub through Bluetooth Low Energy (BLE) communication protocol. The
device is intended to be worn on the arm as an armband. However, it might be
placed anywhere on the body using a different size strap. The processor reads
signals from sensor at a rate of 100Hz.

Fig. 4. Wearable Device

3 Knowledge model for recognition of training exercises

In this section, knowledge model development for basketball exercise recognition

is described in detail. Presented knowledge model is developed to autonomously
understand the type of technical training that is being perform and records
progress and gives feedback based on the perceived context. Such approach shows
that given enough of empirical data, any desired context in basketball coaching
can be perceived.

3.1 Model development

The main steps in the development of knowledge model are (1) data collection
to form learning dataset, (2) preprocessing of raw sensor data, (3) segmentation
and feature extraction, (4) modelling and (5) model verification.
Dataset collection Dataset used in this paper was formed using motion
data collected during training exercises performed by 5 amateur basketball play-
ers. Single wearable sensor was worn by players on their dominant arm between
 Fig. 5. Data Segmentation and Feature Extraction Process

wrist and elbow. Each player performed 6 different exercises (Table 1) in three
different attempts. Each exercise was performed for 30 seconds. Acceleration and
gyroscope signals were recorded and each exercise was labeled with appropriate
exercise class.
Data preprocessing After forming learning dataset [12] the data is pre-
processed. Acceleration and gyroscope signals are noisy and need to be filtered.
To achieve replicability of models, sensor data is scaled to range -/+ 1. Moving
Average Filter is used to eliminate noise from sensor signals.
Segmentation Raw sensor signals form timeseries segmented using Sliding
Window method, to enable consistent feature extraction from raw data [16]. In
his work 3s windows with 50% overlap factors were employed to extract features
from raw data.
Feature extraction Human activity recognition systems with accelerome-
ter and gyroscope sensors engage statistical and, frequency-domain features [13],
[14], [16] to represent raw data. The feature extraction process contains the iden-
tification of description variables that include all relevant sensor information. In
this work the abstraction of raw data has been done with Statistical Calculations
and Discrete Wavelet Transformation [14], [17]. Statistical features distinguish
and represent raw data with high consistency. On the other hand the energies of
6 Daubechies (1 mother wavelet and 5 scaling functions) Discrete Wavelets are
used as features [17], [18]. In addition, Hjorth Parameters (mobility and com-
plexity) are used [19]. The feature vector is formed for 3 s of each of sensor data
and learning instance is formed using combination of feature vector and labeled
exercise types. This process is depicted in Fig. 5.
Modelling Training type recognition process includes defining a target set
of exercises, collecting sensor data, and assigning sensor data to correct exercises.
Most of activity recognition research focus on the use of statistical machine learn-
ing techniques [16] to get a grip on raw data of human activities. In this work,
Supervised Artificial Neural Networks are used to classify different basketball
exercises throughout the context levels.
The approach to the development of recognition model is done in cascaded
fashion. Decisions are done for small number of classes and multiple machine
learning classifiers are developed. There are 6 different types of exercises as it
can be seen on Table 1 and all of them belong to technical training. Four of
them are Ball-Handling exercises (One-Hand Ball-Handling) and 2 of them are
scoring exercises (1 Open Shot and 1 Standing Layup). In recognition process,
Supervised Artificial Neural Networks (Multilayer Feed-Forward Neural Network
or Multilayer Perceptron) classification algorithm [20] runs for every context
level. One ANN is trained to recognize either the exercise is ball handling or
scoring (binary classifier). Scoring is further recognized as shooting or layup
(binary classifier). Ball handling is further recognized as forward backward, left
right, regular or two hands dribble (multi-class classifier).

Level 4 Context Level 5 & 6 Context

Forward Backward Dribble
Left - Right Dribble
Ball Handling
Regular Dribble
Two - Hands Dribble
Shooting
Scoring
Layup
Table 1. Exercise Types

Model verification Above described model is developed on relatively small

dataset. The evaluation of such models is done by utilization of random selection
when forming data partitions for learning and testing. Multiple repetition of the
evaluation process using different randomly selected training and testing sets,
as well as the averaging of the error estimation obtained, can provide satisfac-
tory results. Cross-validation is based on this principle, with the corresponding
substitution of the training set and the test set in each iteration. In this work
results are validated using 10-fold cross-validation method [21].

3.2 Model utilization in real-world training scenario

In the context of basketball training player wears motion sensor on his/her dom-
inant arm and performs exercises with the ball. Referring to the architecture
from Fig. 1, data obtained from sensors is received by hub in real-time and data
is preprocessed to form the same feature vector. Feature vector is supplied as
the input to developed classification model. Model classifies the current train-
ing exercise and records the results in the long term metric accumulation block.
Recorded metrics are compared against the current training plan supplied by
the context block and feedback is supplied to the user. The training plan can
simply consist of number of repetitions for each technical exercise with respect
to predefined rules in regard to the age, physical characteristics, level of profes-
sion, prefered in-game position, etc. Training plan, in the context of developed
AI basketball coach, serves as a set-point for accumulated metrics. Feedback is
generated in the form of motivational suggestions.
4 Results

4.1 Features

Using domain knowledge and review of relevant machine learning and signal pro-
cessing literature, a set of features is obtained to be used for model development
[12]. Features are listed in Table 2. These features are extracted from raw data
of all signals, total of 114 features create feature vector.

Max Skewness Wavelet C Energy Wavelet D4 Energy Variance

Min 25th percentile Wavelet D1 Energy Wavelet D5 Energy Mean abs. dev.
RMS 75th percentile Wavelet D2 Energy Hjorth complexity Mean cross. rate
Mean Median Wavelet D3 Energy Hjorth mobility
Table 2. Features List

4.2 Exercise Recognition

Exercise recognition process is designed as pipeline of 3 classifiers. First classifier

classifies the type of technical training as Ball-Handling vs. Scoring. Then, 2 dif-
ferent classifiers are used to differentiate sub-classes. Recognition using first clas-
sifier is done using only 2 neurons in one hidden layer of ANN. In Shooting and
Layup exercise recognition, ANN of 24 neurons in one hidden layer is used. There
are totally 132 instances, 67 Shooting and 65 Layup. Fig. 6 shows the perfect
classification capability of trained ANN in both Scoring vs. Ball-Handling and
Scoring. In Ball-Handling type recognition process, Forward-Backward Dribbling
and Left-Right Dribbling instances are confused due to the similarity between
exercises. To recognize different Ball-Handling movements, neurons in hidden
layer are set as 60 to reach satisfactory results. Total number of instances is 346,
and Forward-Backward, Left-Right, Regular and Two-Hands Dribbling have 88,
88, 85, 85 observations in respectively.

Fig. 6. Confusion Matrices

 The metrics that are used in validation process to evaluate performance of the
model may change in different studies. However, the widely-used performance
benchmarks in activity recognition research community are Accuracy, Precision,
Recall, F-score and Confusion Matrix [16]. The average accuracy is 94% and
worst performance is for Left-Right Dribbling which is 86%. The results are
summarized in Table 3.

Classes / Metrics Accuracy Precision Recall F1-Score Support

Forward-Backward Dribbling 90% 0.89 0.90 0.89 88
Left-Right Dribbling 86% 0.89 0.86 0.88 88
Regular Dribbling 100% 1.00 1.00 1.00 85
Two-Hands Dribbling 100% 0.98 1.00 0.99 85
Average / Total 94% 0.94 0.94 0.94 346
Table 3. Metric Results of Ball-Handling Classification

5 Discussion

t-Distributed Stochastic Neighbor Embedding (t-SNE) [22] algorithm was used

to understand data distribution of exercise type observations, which reduces
dimensionality to 2D or 3D for better representation. Data distributions can be
seen in Fig.7. As it is seen, both Ball-Handling vs. Scoring and Scoring type data
distributions can be distinguishable in 2D representation. However, Forward-
Backward and Left-Right Dribbling observations are not easily distinguishable in
2D. Developed ANNs also show poor performance with these two exercise types.
These results can be predictable considering the similarity between exercises,
movements closely resemble each other.
The performance of trained ANNs is compared against the real labels using
widely-used method of Stratified K-fold Cross-Validation [21] (Table 3). Average
of validation metrics are calculated to understand general performance of model.
Stratification ensures that every fold is a good representation of whole data. The
verification process shows that the developed model is useful for the task in mind.

6 Conclusion

Training sessions in basketball are prepared, evaluated and monitored by pro-

fessional coaches. Coaches are intensely involved in this process and have a huge
influence on the performance quality of trainees. Coach’s involvement in the
training process, ultimately, decides the competition outcome. The convergence
of technology for trainee monitoring and context processing with field expe-
rience opens up new horizons in basketball coaching. In this work artificially
intelligent assistant is presented for basketball coaching. Once provided with
 Fig. 7. t-SNE 2D Representation of Data Distribution

certain amount of context developed assistant can monitor athletes, understand

finer levels of supplied context (e.g. what exercise is being performed), evaluate
the performance of athletes under certain context and provide necessary feed-
back. The AI assistant operates on the basis of knowledge models that can be
developed for different training scenarios. The knowledge models allow embed-
ding of the field experience into analytic, computational intelligence or artificial
intelligence models. Presented knowledge model is developed to autonomously
understand the type of technical exercise that is being performed and to record
progress and give feedback based on the perceived context. Such approach shows
that given enough of empirical data, any desired context in basketball coaching
can be perceived. The ultimate goal of this research is to come up with fully
autonomous assistant that is able to understand and act upon most general
context of basketball training. The future work will consist of development of
knowledge models for estimation of quality of performed exercises and further
developments towards level 3 contextual training types.

References
1. Gregg, D. G.: E-learning agents. The Learning Organization, vol. 14(4), pp. 300–312.
(2007)
2. Solana, J., Caceres, C., Garcia-Molina, A., Chausa, P., Opisso, E., Roig-Rovira, T.,
Gomez, E. J.: Intelligent Therapy Assistant (ITA) for cognitive rehabilitation in
patients with acquired brain injury. BMC medical informatics and decision making,
vol. 14(1), pp. 58–. (2014)
3. Dragone, M., Saunders, J., Dautenhahn, K.: On the integration of adaptive and
interactive robotic smart spaces. Paladyn, Journal of Behavioral Robotics, vol. 6(1).
(2015)
4. Beetz, M., Jain, D., Mosenlechner, L., Tenorth, M., Kunze, L., Blodow, N., Panger-
cic, D.: Cognition-enabled autonomous robot control for the realization of home
chore task intelligence. Proceedings of the IEEE, vol. 100(8), pp. 2454–2471. (2012)
5. Biundo, S., Bercher, P., Geier, T., Mller, F., Schattenberg, B.: Advanced user assis-
tance based on AI planning. Cognitive Systems Research, vol. 12(3-4), pp. 219–236.
(2011)
 6. Bercher, P., Richter, F., Hornle, T., Geier, T., Holler, D., Behnke, G., Nothdurft,
F., Honold, F., Minker, W., Weber, M., Biundo, S.: A planning-based assistance
system for setting up a home theater. Proc. of the 29th national conference on
artificial intelligence (AAAI), pp. 4264-4265. (2015)
7. Vales-Alonso, J., Chaves-Dieguez, D., Lepez-Matencio, P., Alcaraz, J. J., Parrado-
Garca, F. J., Gonzalez-Castano, F. J.: SAETA: A smart coaching assistant for pro-
fessional volleyball training. IEEE Transactions on Systems, Man, and Cybernetics:
Systems, vol. 45(8), pp. 1138–1150. (2015)
8. Klein, M. C., Manzoor, A., Middelweerd, A., Mollee, J. S., te Velde, S. J.: Encour-
aging physical activity via a personalized mobile system. IEEE Internet Computing,
vol. 19(4), pp. 20–27. (2015)
9. Dijkhuis, T. B., Blaauw, F. J., van Ittersum, M. W., Velthuijsen, H., Aiello, M.:
Personalized physical activity coaching: a machine learning approach. Sensors, vol.
18(2), pp. 623. (2018)
10. Bacic, B., Hume, P. Computational intelligence for qualitative coaching diagnos-
tics: Automated assessment of tennis swings to improve performance and safety.
arXiv preprint arXiv:1711.09562. (2017)
11. Ghasemzadeh, H., Loseu, V., Jafari, R. Wearable coach for sport training: A quan-
titative model to evaluate wrist-rotation in golf. Journal of Ambient Intelligence
and Smart Environments, vol. 1(2), pp. 173–184. (2009)
12. Acikmese, Y., Ustundag, B. C., Golubovic, E. Towards an artificial training ex-
pert system for basketball. 10th International IEEE Conference on Electrical and
Electronics Engineering (ELECO), pp. 1300–1304. (2017)
13. Yang, C. C., Hsu, Y. L., A Review of Accelerometry-Based Wearable Motion De-
tectors for Physical Activity Monitoring. Sensors, vol. 10(8), pp. 7772-7788. (2010)
14. Ren, X., Ding, W., Crouter, S. E., Mu, Y., Xie, R., Activity recognition and in-
tensity estimation in youth from accelerometer data aided by machine learning.
Applied Intelligence, vol. 45(2), pp. 512–529. (2016)
15. Wearable Sensor v2, Inovatink, https://github.com/inovatink/ws-hardware
16. Lara, O. D., Labrador, M. A., A survey on human activity recognition using wear-
able sensors. IEEE Communications Surveys and Tutorials, vol. 15(3), pp. 1192–
1209. (2013)
17. Rioul, O., Vetterli, M., Wavelets and signal processing. IEEE signal processing
magazine, vol. 8(4), pp. 14–38. (1991)
18. Guo, L., Rivero, D., Seoane, J. A., Pazos, A., Classification of EEG signals using
relative wavelet energy and artificial neural networks. In Proceedings of the first
ACM/SIGEVO Summit on Genetic and Evolutionary Computation (ACM), pp.
177–184. (2009)
19. Khan, M., Ahamed, S. I., Rahman, M., Smith, R. O. , A feature extraction method
for realtime human activity recognition on cell phones. In Proceedings of 3rd Inter-
national Symposium on Quality of Life Technology (isQoLT 2011).
20. Russell, S. J., Norvig, P., Artificial intelligence: a modern approach. Malaysia;
Pearson Education Limited. (2016)
21. Kohavi, R., A Study of Cross Validation and Bootstrap for Accuracy Estimation
and Model Selection. International Joint Conference on Artificial Intelligence (IJ-
CAI). (1995)
22. Maaten, L. V. D., Hinton, G., Visualizing data using t-SNE. Journal of Machine
Learning Research 9, pp. 2579–2605. (2008)

View publication stats