Input Method Hub
Introduction
The purpose of this ticket is to serve as an entry page for input method support in Wayland. It shall give newcomers a first understanding of what input methods are and how they are implemented in Wayland.
The ticket shall also act as a hub to organize ongoing efforts for improving input method support in the Wayland Protocols project and point to as well as guide the individual downstream projects basing their implementations on us.
Table of Contents
Motivation
Keyboards were one of the first, and are still one of the most important devices for entering data into a computer. In Wayland we have the wl_keyboard interface to directly handle the most common form of keyboards: the well known physical hardware device with more or less 100 individual buttons. Here each button is identified by a separate code. Usually the buttons represent the letters of the alphabet, a few special characters and some additional function keys.
A standard keyboard, here with RGB for increased typing speed.
Basically these individual codes are then just forwarded via wl_keyboard to a Wayland client. Regularly though this basic form of data input is not enough and we rely on more sophisticated means. Reasons for that can be:
- User writes in a language that has more individual letters than can reasonably be put on a physical keyboard, for example several Asian languages (often coined CJK as of Chinese, Japanese, Korean).
- User does not use a physical keyboard but uses other means of inputting text, for example a virtual keyboard on a touch screen.
- Additional metadata about text should be communicated, for example to provide suggestions for word completion while the user types in individual letters.
CJK input method presenting word selection options.
Software components that – independent of the connected physical hardware – provide such functionality, are called input methods.
Challenges
At the moment Wayland Protocols provides several protocols at different versions to enable input methods support on Wayland. While these protocols already allow to get CJK input and virtual keyboards working to a certain degree, the situation is far from optimal:
- The two relevant protocols in Wayland Protocols are at different versions.
- Some downstream projects use these protocols, but already more advanced versions.
- Some projects use completely different protocols.
- There is a number of technical issues with the currently provided protocols and merge requests to solve some of them are already stalled for a long time.
Current State
Overview
An overview of how input method support is engineered in principle on Wayland can be found in this blog post.
One can think of it like a pipeline with both ends being connected to different clients and each end being defined by a separate protocol. On one end a special client acts as an input method (for example IBus) which sends text and on the other end a client with a focused text input field receives the text.
Diagram sourced from above article visualizing the pipeline model.
That is not a one-way street though. The client receiving text needs to often provide metadata to the input method client so that the input method can function correctly.
Upstream Protocols
In the following Wayland protocol extensions are listed that are relevant for input methods support and are still actively used by some clients.
text-input-unstable-v3
Documents
Details
Defines the receiving side of the pipeline. Influenced by the gtk-text-input protocol, which is now deprecated. In comparison to previous text-input versions v1 and v2 the third version has been slimmed down and uses atomic operations.
Notable requests:
-
set_surrounding_text: Provides metadata about the text, for example for auto completion of words. -
set_content_type: Provides metadata about the text field, for example so a phone keypad instead of a full keyboard is shown on a touchscreen.
input-method-unstable-v1
Documents
Details
Defines the dispatching side of the pipeline. Primarily provides methods to interact with the receiving end on the other side of the pipeline, but also requests to define an "input panel" surface.
Notable requests:
-
preedit_string: Sends a pre-edit string via pipeline. -
commit_string: Sends final text via pipeline. -
keysym: Forwards a key event. Seems to collide withkeyandmodifiersrequests. -
grab_keyboard: Grabs input from a wl_keyboard. Complemented by thekeyandmodifiersrequests. -
set_toplevel: Sets a referenced surface as the input panel at indicated position on a certain output.
Downstream Protocols
input-method-unstable-v2
Documents
Details
TODO
virtual-keyboard-unstable-v1
Documents
Details
TODO
qt-text-input-method-unstable-v1
Documents
Details
TODO
Implementation Matrix
The following table lists clients and compositors with the protocols they support at which version. Please post a comment if you can provide information which is missing (including links) or wrong. I will then update the table.
| text-input | input-method | virtual-keyboard | qt-text-input-method | |
|---|---|---|---|---|
| Chromium Ozone | v1 | - | - | - |
| fcitx5 | - | v1, v2 | - | - |
| GTK | v3 | - | - | - |
| IBus | - | v1 | - | - |
| Maliit | - | v1 | - | - |
| squeekboard | - | v2 | v1 | - |
| Qt | v2, v3, v4-MR | - | - | v1 |
| COMO | v2, v3 | v2 | v1 | - |
| KWin | v2, v3 | v1 | - | - |
| Mir | v1, v2, v3 | v1, v2 | v1 | - |
| Mutter | v3 | - | - | - |
| Phoc | v3 | v2 | v1 | - |
| Smithay | v3 | v2 | v1 | - |
| Weston | v1, v3-MR | v1 | - | - |
| wlroots | v3 | v2 | v1 | - |
Stalled Upstream Protocol Work
text-input-unstable-v3
- !10 (merged): Adds hint that no input panel should be shown by the input methods client. Stalled because no implementations exist making real use of it.
- !16: Mark invalid state changes with serial 0 so a client can filter them. Stalled because of integer overflow ambiguity.
- !19: Specify text deletion in the absence of set_surrounding_text support. Stalled as no reviewer showed up yet.
- !34: Adds reset request, a "leaner" alternative to disable/enable. Stalled because benefit not clear and technical alternative was proposed.
virtual-keyboard-unstable-v1
- !11: Adds the protocol. Stalled because of use case is not clear and technically a virtual seat might be preferred.
Next Steps
The goal is to consolidate our different protocols and their versions as well as fix remaining issues with them.
Requirements
The following is a rather unrevised list to give some clues:
- For receiving clients single protocol to implement.
- Allow changing of the preferred input method client at runtime.
- Besides text provide means to send actions like "copy" with Ctrl + C.
- ...
Specific requirements are aggregated in the following subtasks:
- #40: Requirements specifically from the viewpoint of the input method dispatching side.
- #43: Synchronize cursor rectangle with wl_surface.
Protocol Consolidation
Analysis
Q: Is it enough to have a single protocol for the sending side amending/replacing input-method-unstable-v2
and one for the receiving side amending/replacing text-input-unstable-v3? See #41 for a discussion on the general nature of these protocols.
A: ...
Q: Should we emulate physical keyboard keys for actions like "copy" or provide them via a separate interface? See #12 for details.
A: ...
Plan
TODO: Outline solution.
Project Management
Upstream
- Advance according to the plan above.
- This ticket is the main hub which will link to forked off subtasks.
- Work-in-progress branch is wip/text-input-next.
Implementations
TODO: Declare projects participating, link their respective overview tasks and name a person of contact for each.