Making WebKitGTK Accessible Again

Since the GTK4 port of WebKit was released, a major missing feature was the
accessibility of the web contents. Accessibility technologies like Orca
couldn't inspect what was the web content visible on screen. But as of GNOME
46, this is (mostly) fixed!

Let's deep dive into how WebKitGTK implements and exposes the accessible
information of its web pages, how that conflicted with GTK4's expectations, and
also discuss the current state, and potential future improvements.


(c) GUADEC 2024
The GNOME Conference
Denver, US
July 19-24, 2024
https://events.gnome.org/event/209/
https://www.youtube.com/watch?v=iAlxa5sZs-0

1. Ma king WebKitG TK Ac c essibleMa king WebKitG TK Ac c essible
Aga i nAga i n
Georges StavracasGeorges Stavracas
1 / 27

2. We b K i t G T KWe b K i t G T K
2 / 27

3. GTKGTK
3 / 27

4. GTK3 vs GTK4GTK3 vs GTK4
4 / 27

5. WPE WebKitWPE WebKit
https:/ / webkit.org/ wpe/
5 / 27

6. Current StatusCurrent Status
Flatpak Bubblewrap Unsandboxed
GTK3 ! " "
GTK4 ! " "
6 / 27

7. ChallengesChallenges
7 / 27

8. Challenge 1Challenge 1
Out-of-process AccessibilityOut-of-process Accessibility
Tr e e sTr e e s
8 / 27

9. Status: Fixed ✔✔
9 / 27

10. Challenge 2Challenge 2
D-Bus FilteringD-Bus Filtering
10 / 27

11. Status: Fixed ✔✔
11 / 27

12. Challenge 3Challenge 3
Flatpak Sub-SandboxingFlatpak Sub-Sandboxing
12 / 27

13. Pha se 1 : The BrowserPha se 1 : The Browser
1. GNOME Shell executes the org.gnome.Epiphany.desktop
fi le
2. It runs flatpak run org.gnome.Epiphany
3. Flatpak spawns Bubblewrap
4. Bubblewrap spawns xdg-dbus-proxy with permissions
5. Bubblewrap spawns the epiphany p ro c e s s
13 / 27

14. Pha se 1 : The BrowserPha se 1 : The Browser
1. GNOME Shell executes the org.gnome.Epiphany.desktop
fi le
2. It runs flatpak run org.gnome.Epiphany
3. Flatpak spawns Bubblewrap
4. Bubblewrap spawns xdg-dbus-proxy with permissions
5. Bubblewrap spawns the epiphany p ro c e s s
14 / 27

15. Pha se 2 : The Web ViewPha se 2 : The Web View
1. GNOME Web connects to D-Bus
2. GNOME Web acquires a unique bus name ( :1.300 )
3. GNOME Web creates its widgetry
4. GNOME Web a WebKitWeView
5. We b K i t We bV i e w u s e s flatpak-spawn fo r t h e We b p ro c e s s
6. flatpak-spawn c a l l s t h e F l a t p a k p o r t a l
15 / 27

16. Pha se 3 : The Fla tpa k Porta lPha se 3 : The Fla tpa k Porta l
1. Flatpak portal creates a new sandbox
2. Flatpak portal spawns Bubblewrap
3. Bubblewrap spawns xdg-dbus-proxy with sockets
4. Bubblewrap spawns the Web process
16 / 27

17. Pha se 4 : The Web Proc essPha se 4 : The Web Proc ess
1. The Web Process connects to D-Bus
2. The Web Process acquires a unique bus name ( :1.301 )
3. The Web Process starts doing web processy stuff
4. The Web Process notifies G NO ME Web it’s ready
17 / 27

18. Pha se 5 : It A ll Fa lls A pa rtPha se 5 : It A ll Fa lls A pa rt
1. GNOME Web learns the Web Process is ready at :1.301
2. GNOME Web calls org.a11y.atspi.Socket.Embedded
3. It fails
18 / 27

19. Why?Why?
19 / 27

20. Pha se 1 : The BrowserPha se 1 : The Browser
1. GNOME Shell executes the org.gnome.Epiphany.desktop
fi le
2. It runs flatpak run org.gnome.Epiphany
3. Flatpak spawns Bubblewrap
4. Bubblewrap spawns xdg-dbus-proxy with permissions
5. Bubblewrap spawns the epiphany p ro c e s s
20 / 27

21. Ephy’s xdg-dbus-proxy was not set to talkEphy’s xdg-dbus-proxy was not set to talk
to to :1.301:1.301 !!
21 / 27

22. Status: Not Fixed ⚠⚠
22 / 27

23. Challenge 4Challenge 4
Ac c e ssi ble Tre e C onte ntsAc c e ssi ble Tre e C onte nts
23 / 27

24. Status: Not Fixed ⚠⚠
24 / 27

25. Roa dma pRoa dma p
• Continue investigating the Flatpak situation
• Massage the a1 1 y tree of WebKi t i nto what ATs expect
25 / 27

26. Q&AQ&A
26 / 27

27. Tha nksTha nks
27 / 27