CradlePosture: Camera-Based Approach for Estimating Neonate’s Posture Based on Caregiver’s Holding Behaviors

2025 IEEE/SICE International Symposium on System Integration (SII)

1. CradlePosture: Camera-Based Approach for Estimating
Neonate’s Posture Based on Caregiver’s Holding Behaviors
Hiyori Tsuji
1)
, Takumi Yamamoto
1)
, Maiko Kobayashi
2)
,
KyoshiroSasaki
3)
,Noriko Aso
4)
, Yuta Sugiura
1)
1)
Keio University,
2)
WasedaUniversity,
3)
Kansai University,
4)
Kanagawa University
The 2025 17
th
IEEE/SICE International Symposium on System Integration

2. 2
[1]BystrovaK et al., Early contact versus separation: effects on mother-infant interaction one year later.Birth. 2009;36(2):97-109.
[2] Yao X et al., Automated Detection of Infant Holding Using Wearable Sensing: Implications for Developmental Science and Intervention. Proc. ACM IMWUT, 3(2), Article 64.
Background: Holding Behavior
•Impact on caregivers and infants
[1][2]
•Sensitivity of caregivers / self-regulation ability of infants
•Attachment between caregivers and infants
•Proper holding techniques
•Impact of neonate’s body posture on safety
[3]
•Neonate: an infant who is up to one-month years old

3. 3
[3] Mary E. Case., Abusive head injuries in infants and young children. Legal Medicine, 9(2):83–87, 2007. Special section: Forensic Pathology: New Concepts and Technologies.
[4] Liu J et al., Pababy: An Interactive System for First-time Parents to Learn Neonatal Nursing. CHI EA '22. Association for Computing Machinery, Article 320, 1–7.
[5] Tsuji H et al., Smartphone-Based Teaching System for Neonate Soothing Motions. SII 2024, pp. 178-183
Background: Holding Behavior
•Need for caregivers to learn proper holding techniques
•Limited opportunities to learn from experts
→ Easy training of holding techniques in non-professional settings ◎
Prenatal Training
Liuet al.
[4]
Tsuji et al.
[5]
Postnatal Training
Measurement during
actual neonate holding
Feedback to caregivers

4. Estimateneonate’spostureanglesduringholding
•Innon-professionalsettings
•Noneedtoattachsensorsdirectlyonneonates
•Caregiverconcerns:reluctancetoattachsensorstoneonates
•Reducedburden:minimizephysicalimpactonneonates
4
Purpose
[6] Yao X et al., Automated Detection of Infant Holding Using Wearable Sensing: Implications for Developmental Science and Intervention. Proc. ACM IMWUT. 3, 2, Article 64 (June 2019), 17 pages.
Approach using
wearable sensors
[6]

5. 5
Related Work: Camera-Based Posture Estimation
Research Approach
Wang et al.
[7]
Feature extraction using CNN
Pose estimation from spatiotemporal relations
Bhamidipatiet al.
[8]
Pose estimation using MediaPipe
Pose analysis with OpenCV
Leddy et al.
[9]
Comparison of pose estimation with MediaPipe
and IMU measurements
•Posture estimation model using adult posture data
•Accuracy limitations in neonatal posture measurement
Wang et al.
[7]
Leddy et al.
[9]
[7] Wang J et al., Ai coach: Deep human pose estimation and analysis for personalized athletic training assistance. Proceedings of the 27th ACM International Conference on Multimedia, MM ’19, page 374–382, 2019.
[8] BhamidipatiVS et al., Robust intelligent posture estimation for an ai gym trainer using mediapipeand opencv. ICNWC2023, pages 1–7, 2023.
[9]Leddy C et al., Concurrent validity of the human pose estimation model “mediapipepose” and the xsensinertial measuring system for knee flexion and extension analysis during hurling sport motion. In 2023 IEEE International Workshop on Sport, Technology and
Research (STAR), pages 49–52, 2023.

6. 6
Related Work: Camera-Based Infant’s Movement and Posture Analysis
[10] Hesse N et al., Learning an infant body model from rgb-d data for accurate full body motion analysis. MICCAI 2018, 21st Int. Conf., Granada, Spain. Springer, 792–800.
[11 Stahl A et al., An optical flow-based method to predict infantile cerebral palsy. IEEE Transactions on Neural Systems and Rehabilitation Engineering, 20(4):605–614, 2012.
[12] Chambers C et al., Computer vision to automatically assess infant neuromotor risk. IEEE Transactions on Neural Systems and Rehabilitation Engineering, 28(11):2431–2442, 2020.
Stahlet al.
[11]
Hesse et al.
[10]
Chambers et al.
[12]
•Restrictions on subjects and postures
•Capture only an infant’s entire body
•Lying down posture

7. 7
Proposed Method
Camera-Based Approach for Estimating Neonate’s Posture Based on Caregiver’s
Holding Behaviors
•Cradle hold
•Basic holding posture of a neonate
•MediaPipe Pose + XsensDOT
Neonate’s posture
angle
Caregiver’s holding
behavior

8. 8
System
Estimation of
Posture Angles
Detect Landmark
of the caregiver
Regression
Model
Record Video
Measure Posture Angles
MediaPipe Pose
Posture angles
•Inclination angle
•Adduction angle
[13]
[13]Hervey Karp, M.D. . The happiest baby on the block. BANTAM BOOKS.
Inclination
angle
Adduction
angle

9. 9
Method: Dataset Acquisition
•Participants: 3 males, 2 females (Age: Average = 22.6, SD = 0.5)
•Video: 60fps, XsensDOT: 60Hz
•Manual synchronization
•30sec ×2times
1.Inclination angle
2.Adduction angle
(Participants were instructed to gradually vary the angle
during each trial)
Gradually change the inclination angle
Definition of angle

10. Explanatory
variable
[14] Pose landmark detection guide —mediapipe—google for developers. https://developers.google.com/mediapipe/solutions/vision/pose_landmarker. (Accessed on 01/16/2024).
10
Method: Preprocessing and Regression Models Construction
Landmarks of
MediaPipe Pose
[14]
010203040
Randomly selected
from 0 ̊to 40 ̊
in 10 ̊intervals
Response
variable
Data Cleaning Preprocessing
Posture angle
Video
data
XsensDot
data
Relative
coodinates
•Data cleaning: 140 data randomly selected from 0 ̊to 40 ̊in 10 ̊ intervals
•MediaPipePose
[14]
: Absolute coordinates of 25 upper-body landmarks
→ relative coordinates (origin: midpoint of shoulders)
Random Forest
Regressor
Validation
1. Within-
participant
validation
2. Between-
participant
validation
010203040
140
[°]
[°]

11. 11
Result: Within-participant Validation
Inclination angle estimation
Adduction angle estimation
Evaluation of the posture angles estimation model
•High accuracy
→ Capture individual patterns of motion
P1 P2 P3 P4 P5 Mean
Inclination
angle
estimation
Model
R
2
0.9570.9690.9690.9440.965 0.961
±0.009
MAE 0.97 0.76 0.71 0.96 0.79 0.839
±0.106
Adduction
angle
estimation
model
R
2
0.6880.9730.9260.9300.972 0.898
±0.107
MAE 2.58 0.72 1.04 1.220.683 1.250
±0.696

12. 12
Result: Between-participant Validation
Inclination angle estimation Adduction angle estimation
Evaluation of the general posture angles estimation model
•Decreased accuracy
→ Challenges in universal applicability across participants
Inclination angle Adduction angle
R
2
0.625±0.003 -0.206±0.513
MAE 5.12±0.29 8.69±2.96

13. 13
Limitations and Future Work
•Experimental design
•Wide range of angles
•Use of multiple cameras
•Adaptability to different holding
•Toward practical application
•Applicability to actual neonates
•Improvement of estimation accuracy in between-participant validation
•Development of a feedback-based application

14. 14
Summary
Background
Impact on caregivers and children of holding behavior
Need for caregivers to learn proper holding techniques
Related Work Camera-based infant’s movement and posture analysis
Proposed Method
Camera-based approach for estimating neonate’s posture
based on caregiver’s holding behaviors
Implementation Dataset acquisition, Regression model construction
Result
For within-participant validation, angles could be estimated with high accuracy.
For between-participant validation, only inclination angles could be estimated.
Future Work Experimetaldesign, Toward practical application
CradlePosture: Camera-Based Approach for Estimating Neonate’s Posture Based on Caregiver’s Holding Behaviors
Hiyori Tsuji, Takumi Yamamoto, Maiko Kobayashi, KyoshiroSasaki, Noriko Aso, Yuta Sugiura

15. Appendix
15

16. 16
MediaPipePose
[14]
Open-source tool that can perform pose estimation
•Process
•Landmark: upper-body landmarks(No. 0~25)
•Absolute coordinates → relative coordinates
[14] Pose landmark detection guide —mediapipe—google for developers. https://developers.google.com/mediapipe/solutions/vision/pose_landmarker. (Accessed on 01/16/2024).
Correspondence between landmark
numbers and positions
origin
Converting coordinates

17. Applied to a baby
17
Estimation Accuracy of Neonate’s Posture with MediaPipePose
Applied to a baby doll, a caregiver
Estimation is possible when the entire body is captured without overlap with other
objects.
Point: camera position / how to hold a neonate
〇
△ ×
〇
△ ×

18. 18
Reason of Low Accuracy
•P1 Adduction angle
•Subtle shaking hinders the camera or sensors from accurately capturing the movements