Smart Ring Computing: Emerging Challenges and Opportunities in Sensing

This talk will present how a simple redesign of the light sensor’s location in a smart ring can repurpose it for personal safety, such as detecting drink spiking; and health monitoring of hand-grip strength indicators

1. Smart Ring Computing: Emerging
Challenges and Opportunities in Sensing
Huber Flores
ASSOCIATE PROFESSOR
1
October 10, 2024, NTNU, Gjøvik, Norway
Contact e-mail: [email protected]
Homepage: https://huberflores.com
Research group: https://dps.cs.ut.ee

2. Background
•Wearable computing devices are becoming increasingly
transparent and unobtrusive, fulfilling the vision of Mark Weiser.
2
Wearable computing evolution
[source] Malmivaara, Mikko. "The emergence of wearable
computing."Smart clothes and wearable technology. Woodhead
Publishing, 2009. 3-24.

3. Background
•Wearable computing devices are becoming increasingly
transparent and unobtrusive, fulfilling the vision of Mark Weiser.
3
Wearable computing evolution
[source] Malmivaara, Mikko. "The emergence of wearable
computing."Smart clothes and wearable technology. Woodhead
Publishing, 2009. 3-24.
[source] https://www.digitaltrends.com/mobile/oura-ring-vs-ultrahuman-ring-air-smart-rings-
comparison-which-is-better/

4. Smart ring computing
•Miniaturization of sensors and computing units
•Emerging applications in healthcare, (sleeping patterns)
•Innovative new designs are emerging to capture user input
through gestures.
4
[source]
https://www.self.com/re
view/oura-ring-gen-3
[source] Zhang, Tengxiang, et al.
"Thermalring: Gesture and tag
inputs enabled by a thermal
imaging smart ring."
ACM CHI 2020.

5. Smart ring computing
•Miniaturization of sensors and computing units
•Emerging applications in healthcare, (sleeping patterns)
•Innovative new designs are emerging to capture user input
through gestures.
5
Can smart rings be harnessed for
innovative new applications?
[source]
https://www.self.com/re
view/oura-ring-gen-3
[source] Zhang, Tengxiang, et al.
"Thermalring: Gesture and tag
inputs enabled by a thermal
imaging smart ring."
ACM CHI 2020.

6. Smart ring computing
•Yin, Z., Liyanage, M., Ottun, A. R., Paul, S., Zuniga, A., Nurmi, P., & Flores, H. (2023).
HIPPO: Pervasive Hand-Grip Estimation from Everyday Interactions.Proceedings of
the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies (ACM UbiComp
2023),6(4), 1-30.
•Yin, Z., Liyanage, M., Zuniga, A., Nurmi, P., & Flores, H. (2023, February). Hedgehog:
Detecting drink spiking on wearables. InProceedings of the 24th International Workshop
on Mobile Computing Systems and Applications(ACM HotMobile 2023).
6

7. HIPPO
7

8. Importance: hand-grip strength
•Hand-grip strength is extremely important. It can be an early
indicator of many health conditions.
8
Muscle
loss:
Cognitive impairment:
Diabetes:

9. Current solutions
9
Dynamometer
Pressure
based
Drawback
•unable for everyday
interactions
•requiring interactions
with delicated objects
•Mainly for medical use
•Information needed:
gender, age
•Hand-grip strength (kg)
•Hand-grip category
(weak, normal, strong)
Pervasive hand-grip estimation is
required

10. HIPPO solution
10
Pervasive and Ubiquitous
interactions with daily objects
for hand-grip strength estimation
•Portable and non-invasive
•Continuous health monitoring

11. HIPPO: Overview
11
(a) Light sensing deployment
and available fingers
(b) Hand-grip
relax state
(c) Hand-grip applied
strength state
Light changes from (b) to (c)
to capture hand-grip strength
Position-1: little
Position-2: middle
Position-3: index

12. HIPPO: Pipeline
12
Weak
Normal
Strong

13. HIPPO: Experiment setup
13
HIPPO
prototype
Dynamometer
(DYNA) -
gold standard
Pressure
Wooden ball
(W-BALL)
Pressure
plate
(PLATES)
Baselines
•24 participants (equal
genders, age)
•2 random trials for each
•one type of object
•ruler to measure hand
dimensions
•post interview

14. HIPPO: Gender Effect
Result: Gender characteristics can affect the physical strength
14
Hand-grip analysis divided by genders
Little finger Index finger

15. HIPPO: Hand Size Effect
Result: The little finger can achieve the best hand-grip strength
estimation according to gender and hand-size
15
Estimated pressure on the object captured by
the light senor in little finger

16. HIPPO: Hand-grip Strength Estimation
Result: hand-grip strength can be accurately estimated using HIPPO
and especially when the sensor is worn closely to the little finger
16
Features: light changes (L), gender (G)
and hand size (S), age (A).
Categories classification
by L-G-S-A:
Best performance of hand-grip strength by L-G-S
Little finger: 86%
Middle finger: 79%
Index finger: 84%
Little finger Middle finger Index finger

17. HIPPO: Comfort in Capturing Hand-grip Strength
Result: HIPPO provides the least intrusive and easiest-to-use solution
and is preferred by the user.
17
Ranking of the device (the most comfortable–4, the least comfortable–1)
HIPPO

18. Hedgehog
18

19. Importance
•Drink spiking is a critical social issue to overcome as it can cause
victims severe danger, anxiety, and physical or mental harm
➢memory loss
➢brain activity slowness
➢concentration difficulty
➢light-headedness
➢blurred vision
➢loss of balance
➢nausea and vomiting
➢speech difficulty
19
Preventing drink spiking with drugs is extremely important

20. Hedgehog: Overview
20
Characterization of drinks with light. (a) Hedgehog design, (b) light reflectivity pattern,
(c) Location(s) for sampling, (d) Sampling through a transparent surface,
(e) Sensing pipeline for analysis of collected samples.

21. Hedgehog: Testbed
Figure 2: (a) Smart ring prototype from different angles; (b)(1)Drink characterization;
(b)(2) Drink spiking with common pills; (c) Milk drink with aggregated compound
mixture; (d) drink spiking with soluble pills.

22. Hedgehog: Evaluation
Result: Hedgehog can characterize different drinks in different
positions.
22
(a) Little finger (b) Middle finger (c) Index finger

23. Hedgehog: Evaluation
Result: Hedgehog is capable of detecting changes in drink
characteristics
23
(a) Little finger (b) Middle finger (c) Index finger
Milk mixed with powder.

24. Hedgehog: Evaluation
Result: Hedgehog can detect drink spiking accurately
24
Water spiked by different pills
Classification performance:
Little finger (position-1): 48.64% ,
Middle finger (position-2): 89.71%,
Index finger (position-3): 82.24%
Middle finger achieves the best

25. Summary and conclusions
•HIPPO and Hedgehog illustrate how smart ring design can be
leveraged to create innovative applications.
•Repurposing sensor data and using advanced pipelines are essential
for obtaining high-quality data representations for classification.
•The results demonstrate practical applications for smart rings.
•Other applications?
25

26. Summary and conclusions
•HIPPO and Hedgehog illustrate how smart ring design can be
leveraged to create innovative applications.
•Repurposing sensor data and using advanced pipelines are essential
for obtaining high-quality data representations for classification.
•The results demonstrate practical applications for smart rings.
•Other applications?
26

27. Questions? Thank you!
Huber Flores([email protected])
Homepage: https://huberflores.com

27