WebRTC - an analytics perspective by callstats.io

Varun Singh, callstats.io CEO, gave a presentation at Enterprise Connect 2016. He talked about how WebRTC is being used and what are the analytics behind it.

1. An analytics
perspective
Varun Singh, CEO
Orlando, 07.03.2016
@callstatsio @vr000m

2. http://geek-and-poke.com/
2

3. callstats.io
First cloud-based monitoring service for WebRTC
•40 paying customers
•+80 developers accounts
Jan FebMarAprMayJunJulAugSepOctNovDecJanFeb 3

4. Very Simple API
•3 lines of code —> 5 minutes.
1.include <script>
2.initialize() —> needs registration keys
3.addNewFabric(pc, …) —> call started
•optionally: send user events: audio muted/unmuted, video paused/
resumed, call terminated, call held.
•optionally: collect and send user feedback
http://www.callstats.io/api/
4

5. It is not all about throughput*
•… it is about latency or delay!
*)capacity
bandwidth
speed
audio only call

6. Disruptions
Disruption: loss of connectivity when
network interfaces change, low
available capacity,or high delay
The light grey vertical lines show disruption, highlighted by the red bounding boxes.
6

7. Disruptions and user behaviour
User Behaviour: The user tries
to correct for the disruption by
turning on and off video
7

8. state of webrtc
(as we observe it)
8

9. multiparty?
0 %
10 %
20 %
30 %
40 %
50 %
60 %
70 %
80 %
90 %
100 %
2 3 4 >5
9
65%
31%
3% 1%

10. ~12% calls
fail to set up
10

11. Failure Reasons
14% 85%<1%
~20% of the setup
calls have issues
11

12. Types of Relays
0 %
10 %
20 %
30 %
40 %
50 %
60 %
70 %
80 %
90 %
100 %
No Relay TURN/UDP TURN/TCP TURN/TLS
78%
13%
7%
2%
12
22% of calls need
infrastructure

13. getUserMedia() errors?
0 %
25 %
50 %
75 %
100 %
Permission Denied
Device not found
Permission Dismissedconstraint not satisfied
device not supported1 % 3 % 5 % 35 % 56 % 13

14. 0 %
20 %
40 %
60 %
80 %
100 %
no rejoins once twice thrice3 % 4 % 17 % 76 %
Churn* *participant rejoins the
same call repeatedly
Average joins per
participant per conference
14

15. Dropped calls
0 %
20 %
40 %
60 %
80 %
100 %
connectivity lost
high delay
high loss
battery
crash16 % 3 % 14 % 23 % 44 %
15
one-way
audio/video

16. Time to Failure
0 %
20 %
40 %
60 %
80 %
100 %
<10s
10-30s 30-60s
>60s
16
People are
very patient!
*not mobile
50%
33%
7%
10%

17. Lessons Learnt
•ask for audio and video device permissions
separately
•deploy TURN servers with UDP and TCP support
•detect crashes and re-setup connections
•media pipelines sometimes crash
•screen sharing plugin crashes

18. Summary (1/2)
•Browsers: Chrome dominates WebRTC
•OSes: Windows and Mac are pretty even
•Participants: ~3 participants in a call on average
•Relays: ~20% sessions need a TURN server
18

19. Summary (2/2)
•Setup time: 80% sessions setup in <5s
•Call Setup Failures: ~10% of calls fail to setup
•Reason for failure: 85% due to NAT/FW
•Call Drop: 20% calls fail after setup
•Churn: 25% of sessions have a participant rejoining
19

20. Vision
Gather
Report
Analyze
FIX
Deploy?
webrtc-internal?
via email?
Has this been
reported before?
Did it solve the issue?
Until next report?
Collect
Diagnose
Fix
Deploy
A world where real-time
communication is both
frictionless and effortless to
set up, operate, and scale.
20

21. Free developer account
https://dashboard.callstats.io/register