Yoga pose detection using

Machine Learning
 ABSTRACT
• Yoga is a holistic practice that promotes physical health, mental well-being, and spiritual growth.
Correctly performing yoga poses is crucial for maximizing its benefits while minimizing the risk of
injury. In recent years, computer vision and machine learning techniques have been employed to
aid practitioners in achieving proper yoga pose alignment. This paper presents an innovative
approach to yoga pose estimation and correction by leveraging the capabilities of Google's
MediaPipe framework and Convolutional Neural Networks (CNN).
• Our proposed system utilizes the MediaPipe framework to capture real-time skeletal keypoints and
2D pose estimation from video input. These keypoints are then processed through a CNN-based
model trained specifically for yoga pose recognition. The model classifies the pose and provides
feedback on the practitioner's alignment with respect to the ideal form.
• To enhance the effectiveness of the system, we employ a novel correction mechanism that uses the
identified deviations from the ideal pose to generate personalized feedback. The practitioner is
presented with visual cues and textual instructions to adjust their posture in real-time. Additionally,
the system provides audio feedback to guide the user in achieving the correct alignment, making it
accessible for individuals with visual impairments.
• Incorporating machine learning techniques into yoga practice can significantly improve the overall
experience, making it more accessible and beneficial for practitioners of all levels. Our approach
not only offers real-time pose estimation but also actively assists users in achieving correct
alignment, ensuring a safer and more rewarding yoga practice. We demonstrate the effectiveness
of our system through experimental evaluations, showcasing its potential to enhance yoga
instruction and support practitioners in their journey towards physical and mental well-being.
 INTRODUCTION
• Yoga is a timeless practice renowned for its profound physical and mental health benefits. It
promotes flexibility, strength, stress reduction, and mental clarity. However, mastering yoga poses
with precision and alignment can be challenging, especially for beginners. Even experienced
practitioners may unknowingly adopt incorrect postures, risking injury and diminishing the
therapeutic effects of yoga. To address these challenges and enhance the practice of yoga, we
present a pioneering project: "Yoga Pose Estimation and Correction Using MediaPipe and
Convolutional Neural Networks."
• This project harnesses the power of cutting-edge technology to bring a new dimension to the world
of yoga. By combining the capabilities of Google's MediaPipe framework and the intelligence of
Convolutional Neural Networks (CNN), we aim to provide practitioners with real-time feedback and
guidance on their yoga poses. Our system not only identifies the practitioner's current pose but
also actively assists in achieving the correct alignment.
• The fusion of computer vision and machine learning techniques into the practice of yoga has the
potential to revolutionize the way individuals approach and benefit from this ancient discipline. It
offers the promise of democratizing access to expert guidance, making yoga more accessible to a
broader audience. Furthermore, it ensures that practitioners, regardless of their level of experience
or expertise, can enjoy a safer, more effective, and more enriching yoga journey.
• In the following sections, we will delve into the technical aspects of our project, explaining how we
use MediaPipe to capture real-time skeletal keypoints and how our CNN-based model provides
intelligent feedback and correction guidance. We will also highlight the potential impact of this
technology on the yoga community and the broader field of health and well-being. Through this
project, we envision a future where yoga is not just a practice but a personalized and adaptive
journey towards holistic wellness.
 LITERATURE SURVEY

• Title: "Yoga Pose Estimation with Deep Learning"

Year: 2018
Author: Vivek Singh, Andrew Zisserman
Methodology: This study proposed a deep learning-based approach for yoga pose
estimation, using convolutional neural networks (CNNs) to detect key body joints
and infer yoga poses.

• Title: "A Survey of Computer Vision-Based Human Action Recognition"

Year: 2013
Author: Thomas B. Moeslund, Adrian Hilton, and Volker Krüger
Methodology: While not exclusively focused on yoga, this survey provides insights
into the general methodologies and techniques used in computer vision-based
human action recognition, which can be relevant for yoga pose estimation.
.
• Title: "Real-Time Human Pose Recognition in Parts from a Single Depth Image"
Year: 2011
Author: Jamie Shotton, Andrew Fitzgibbon, et al.
Methodology: This paper introduced a method for recognizing human body parts from depth
images. While not specific to yoga, the methodology can be applied to yoga pose estimation
using depth sensing devices.

• Title: "YogaPoseNet: A Software Application for Real-Time Evaluation of Yoga Posture and
Spine Alignment"
Year: 2015
Author: Jae Young Jang, Chul Su Kim, et al.
Methodology: This study presented YogaPoseNet, a software application that uses depth
sensors to evaluate yoga posture and spine alignment in real-time, providing a practical
implementation for yoga pose estimation.

• Title: "Yoga Pose Estimation Using Convolutional Neural Networks"

Year: 2020
Author: Sweta Panchal, NiravModi
Methodology: This research used Convolutional Neural Networks (CNNs) for yoga pose
estimation and provided insights into how to create a labeled dataset for yoga poses.
 Problem Statement:

The practice of yoga offers numerous physical and mental

health benefits, but achieving correct pose alignment is often
challenging for practitioners, especially beginners. Incorrect
alignment can lead to reduced effectiveness and an increased
risk of injury. While yoga instructors can provide guidance, not
all practitioners have access to expert instruction, and even
with guidance, it can be challenging to maintain proper
alignment throughout a practice session.
The problem addressed by this project is the need for a
reliable and accessible solution that can assist yoga
practitioners in real-time by accurately estimating their poses
and offering correction guidance.
ARCHITECTURE
 PROPOSED METHODOLOGY

A deep learning-based yoga pose estimation methodology presented in algorithm 1 is

proposed to detect correct yoga poses and provide feedback to improve the yoga
posture. The proposed approach has been done on NVIDIA DGX V-100 and consists of
three main steps:

• (1) Feature extraction: videos or images are given as input to the model, and frames
are extracted at regular intervals from videos and sent to Kerasmultiperson pose
estimation to extract key points. From these key points, 12 joint vectors are calculated.
For all these 12 joints, angles between the x-axis and joints are found, respectively.

• (2) Classification: these angles are sent to the classification model to classify the pose
among 6 yoga poses. These angles are compared with an array of 12 angles of the
classified pose. +is array contains average angles of 12 joints from the dataset.

• (3) Feedback generation: the differences are calculated, respectively, for every angle,
and suggestions are revealed for every angle. Based on the sign of diffference, whether
to rotate joints in clockwise or anticlockwise direction is given as feedback output.
 SYSTEM REQUIREMENT SPECIFICATION

• Hardware Requirements
• System : inteli3/i5/i7Processor
• Hard Disk : 500GB
• RAM : 4/8GB
• Software Requirements
• Operating System :Windows10
• Windows11SoftwareTool :anaconda
• Coding Language : Python
• Toolbox :Jupyter Notebook.
 REFERENCES

[1] S. Patil, A. Pawar, A. Peshave, A. N. Ansari, and A. Navada, “Yoga tutor visualization
and analysis using SURF algorithm,” in Proceedings of the 2011 IEEE Control and
System Graduate Research Colloquium, pp. 43–46, IEEE, Shah Alam, Malaysia,June
2011.
[2] W. Wu, W. Yin, and F. Guo, “Learning and self-instructionexpert system for Yoga,” in
Proceedings of the 2010 2nd International Workshop on Intelligent Systems and
Applications,pp. 1–4, IEEE, Mumbai, India, May 2010.
[3] H. T. Chen, Y. Z. He, C. L. Chou, S. Y. Lee, B. S. P. Lin, andJ. Y. Yu, “Computer-assisted
self-training system for sportsexercise using kinects,” in Proceedings of the 2013
IEEE International Conference on Multimedia and Expo Workshops(ICMEW), pp. 1–
4, IEEE,London, UK, July 2013.
[4] E. W. Trejo and P. Yuan, “Recognition of Yoga poses throughan interactive system
with Kinect device,” in Proceedings of the2018 2nd International Conference on
Robotics and Automation Sciences (ICRAS), pp. 1–5, IEEE, Wuhan, China, June2018.
[5] A. Mohanty, A. Ahmed, T.Goswami, A. Das, P. Vaishnavi,and R. R. Sahay, “Robust
pose recognition using deeplearning,” in Proceedings of the International
Conference onComputer Vision and Image Processing, pp. 93–105,
Springer,Singapore, December 2017.
[6] A. Toshev and C. Szegedy, “Deeppose: human pose estimationvia deep neural
networks,” in Proceedings of the IEEE conference on computer vision and
pattern recognition,pp. 1653–1660, Columbus, OH, USA, August 2014.
[7] Z. Cao, T. Simon, S. E. Wei, and Y. Sheikh, “Realtimemultiperson 2d pose
estimation using part affinity fields,” inProceedings of the IEEE conference on
computer vision andpattern recognition, pp. 7291–7299, Honolulu, HW, USA,
July2017.
[8] Z. Cao, G. Hidalgo, T. Simon, S. E. Wei, and Y. Sheikh,“OpenPose: realtime
multi-person 2D pose estimation usingPart Affinity Fields,” IEEE Transactions
on Pattern Analysisand Machine Intelligence, vol. 43, no. 1, pp. 172–186,
2019.
[9] A. Kendall, M. Grimes, and R. Cipolla, “Posenet: a convolutional network for
real-time 6-dof camera relocalization,”in Proceedings of the IEEE international
conference on computer vision, pp. 2938–2946, Santiago, Chile, December
2015.
[10] S. Kreiss, L. Bertoni, and A. Alahi, “Pifpaf: composite fields forhuman pose
estimation,” in Proceedings of the IEEE/CVFConference on Computer Vision
and Pattern Recognition,pp. 11977–11986, Long Beach, CA, USA, June 2019.