ES810-3 Research Methods and Communications 1-2024.pdf

Proposal Presentation for ES810
Conference Presented in iSAI2024 Bangkok, Thailand

2. Exercise Pose Recognition and
Counting System using Robust
Topological Landmarks
Monday, 23 SEP, 2024
This research is supported by SIIT,
TU, NSTDA and TAIST-Tokyo Tech.
by
Nyan LINMYA ¹
¹ Sirindhorn International Institute of Technology (SIIT), Thammasat University, Pathum Thani, Thailand.

3. Overview
Background & Motivation
Research Problem
Scope
Significance
01
02
03
04
Literature Review05

4. Background & MotivationI
Studies on important of exercises for
a better quality of life
Focus on breathing, posture and
counting at the same time is hard
Wearable: possible non-intrusive
but not accurate, uncomfortable
3

5. Background & MotivationI
4
Another Method?
Vision based methods using
camera would be helpful.

6. Research Problem (Vision-Based Methods)II
Action Recognition
Deep Learning Approaches?
Long Time
Large Datasets
Lighting, Perspective, Body Size &
Uncontrollable Conditions on media
Shaped Based Methods
Joints & Body Parts (2D or 3D space)
Use those extracted features
Pose Estimation
5
Figure 1: MediaPipe vs YOLOv7

7. Scope - Proposed MethodsIII
A novel method for exercise recognition
and counting
Pose Estimation (Detect position &
orientation of human body)
Center averaging (Accurate & Robust)
Principle Component Analysis - PCA
(Reduced computational complexity)
6
Figure 2: BlazePose 33 keypoints topology as COCO (colored with green) superset

8. A. Image Dataset
For example, for the push-up there is
push-up up for the default position and
push-up down for the second or the final
position.
3 exercises: barbell biceps curl, push-up &
lateral raise
6 classes
7
Scope - Proposed MethodsIII
Figure 3: Examples of subcategories of poses from Workout/Exercise Images

9. B. Pose Estimation
Alpha Pose, MediaPipe, OpenPose & YOLO
MediaPipe: 30 FPS
Works on Andriod Devices with TFLiteGPU: 25ms
8
Scope - Proposed MethodsIII
Figure 4: MediaPipe vs YOLOv7
Figure 5: MediaPipe Hand Gesture Recognition with TFLite on mobile

10. 9
Scope - Proposed MethodsIII
C. Images to CSV Dataset
MediaPipe has 33 Landmarks (See Figure 2)
22 Landmarks without Facial Landmarks (See Figure 2)
X, Y, Z - axis, and Visibility for 33 key points, 99 features
(Visibility is not considered for this research)
Figure 6: Example CSV data file of first 3 Landmarks (X, Y, Z - axis) of MediaPipe

11. 10
Scope - Proposed MethodsIII
D. Libraries & Frameworks
Scikit-Learn
Free software machine learning library (Python)
Simple and efficient tools for predictive data analysis
Features various classification, regression and clustering algorithms
support-vector machines
random forests
Built on NumPy, SciPy, and matplotlib

12. 11
Scope - Proposed MethodsIII
E. Machine Learning Models
Logistic Regression, Support Vector Classification,
Decision Tree Classification & Random Forest Classification
Figure 6: Example of model training

13. F. Center Averaging & Scaling
12
Scope - Proposed MethodsIII

14. 13
Scope - Proposed MethodsIII
G. Model Evaluations
Accuracy
Precision
Recall
F1-Score
Figure 7: Example of model evaluations

15. PCA
14
Scope - Proposed MethodsIII
Step 1: Standardize the data
Step 2: Apply PCA
Step 3: Get the explained variance ratios
of the principal components
Step 4: Determine the number of
components to retain based on the plot
Step 5: Get the loadings (eigenvectors),
Calculate the importance scores for each
feature (column), and Sort the features
based on importance scores, and get the
most important features (columns)

16. For Research Contribution
Compares different machine
learning algorithms
Various pre-processing techniques
Improve accuracy and robustness
Improve efficiency
For General Public
Posture Correction
Implement HID
15
SignificanceIV

17. 16
Literature ReviewV
Studies such as [1] and [2] use MediaPipe Pose for exercise repetition counting by using
joint-angle calculation between feature points. Without the use of machine learning, it
would be sometimes incorrect as the positions of the angle can be change depending on
the camera angle.
There are several types of research done using MediaPipe and machine learning
algorithms, such as [3], [4], and [5]. However, they can only recognize stationary poses
and did not consider counting the repetitions.

18. 17
BibliographyVI
[1] A. Anilkumar, A. KT, S. Sajan, and S. KA, “Pose estimated yoga monitoring system,” 2021.
[2] Y. Kwon and D. Kim, “Real-Time Workout Posture Correction using OpenCV and MediaPipe,”
한국정보기술학회논문지, vol. 20, no. 1, pp. 199–208, 2022.
[3] U. Bahukhandi and S. Gupta, “Yoga pose detection and classification using machine
learning techniques,” Int Res J Mod Eng Technol Sci, vol. 3, no. 12, pp. 13–15, 2021.
[4] A. K. Singh, V. A. Kumbhare, and K. Arthi, “Real-time human pose detection and recognition
using mediapipe,” in International Conference on Soft Computing and Signal Processing, 2021,
pp. 145–154.
[5] W. Supanich, S. Kulkarineetham, P. Sukphokha and P. Wisarnsart, “Machine Learning-Based
Exercise Posture Recognition System Using MediaPipe Pose Estimation Framework,” 2023 9th
International Conference on Advanced Computing and Communication Systems (ICACCS),
Coimbatore, India, 2023, pp. 2003-2007, doi: 10.1109/ICACCS57279.2023.10112726.

19. This research is supported by SIIT,
TU, NSTDA and TAIST-Tokyo Tech.
Thank you.
Monday, 23 SEP, 2024

20. This research is supported by SIIT,
TU, NSTDA and TAIST-Tokyo Tech.
Questions &
Answers
Monday, 23 SEP, 2024