Using libei to support accessibility functionality on Wayland
Currently some accessibility related functionality is missing on Wayland. In particular with GTK4, which has dropped some legacy things that have a replacement that currently only works on X11. Which makes Orca's keybindings not work with GTK4 in Wayland sessions.
Libei has been brought up as a way to address these things in https://gitlab.gnome.org/GNOME/mutter/-/issues/9 and https://gitlab.gnome.org/GNOME/mutter/-/issues/2078. It looks like there are a couple related comments in https://www.freedesktop.org/wiki/Accessibility/Input too.
For things like Orca key bindings, https://gitlab.gnome.org/GNOME/at-spi2-core/-/blob/main/atspi/atspi-device.c (which has an X11 implementation) has APIs for grabbing keys. That might be out of scope for libei, and need to be handled elsewhere? It also has a atspi_device_add_key_watcher
method, which I think receives all key presses and releases? In which case libei would suitable.
But I don't see any API's in that class for synthesizing input events. https://gitlab.gnome.org/GNOME/at-spi2-core/-/blob/main/registryd/deviceeventcontroller.h seems to have key grabs, and also methods for emitting keyboard and mouse inputs? I'll need to see how Orca or anything else is using those, but libei seems appropriate for this use case too.
Key grabs may be out of scope for libei, but monitoring keypresses and synthesizing input is exactly the sort of thing libei can do, so it makes sense to use it for that. Either libei provides the necessary functionality for that already, or any smaller details that are necessary could be added.
- How should key grabs be handled? If not libei, some other mechanism will be needed.
- If an accessibility tool needs to monitor key presses and synthesize events, would it create two libei context, one as a "sender" and one as a "receiver"?
- Authentication: this is listed in the readme as an "open question". For a screen reader, it definitely needs to work without a (not fully accessible) prompt each time.
- How should this be integrated into accessibility software?
- I believe an
AtspiDevice
backend could be added for Wayland that depends onlibei
, and potentially using some other wayland/dbus mechanism for grabs. I'm less sure about input event synthesis.
- I believe an