Screen Keyboard

A touchscreen keyboard for GNOME 50 and 51, with a movable floating layout, predictive text, and swipe typing. Built from GNOME's on-screen keyboard and powered by FUTO models running on your computer.

• Float or dock. Move a compact keyboard around the screen or dock it along the bottom. Its position and mode are remembered across logins.
• Type or swipe. Tap letters, hold for numbers and accents, or slide a finger across a word. Swipe typing currently supports English US and GB layouts.
• Local predictions. FUTO ML4 supplies corrections and word suggestions. A frequency dictionary provides suggestions while the model starts up.
• Personal vocabulary. Learn from typing and accepted suggestions, with controls to disable learning or forget individual words.
• Private input. Recognition runs locally. Passwords and private fields disable prediction and swipe typing. No text or swipe traces are uploaded.

Easy install

One command downloads, builds, installs and enables the keyboard, including ML4 predictions and FUTO swipe typing. Run as your desktop user, without sudo:

curl -fsSL https://git.salix.host/salix/gnome-not-shit-screen-keyboard/raw/branch/master/install.sh | bash

The installer downloads the source, builds both native helpers, fetches the verified FUTO ML4 and swipe models, and installs the extension for your user. It prepares and validates the package before replacing an existing installation. GPU acceleration is optional and detected automatically. No GPU vendor or backend setting is needed for a normal installation. Missing Vulkan build tools or a failed automatic GPU build selects CPU predictions instead.

1. Log out and back in after installation.
2. Turn on Screen Keyboard in Quick Settings.
3. Keep GNOME's built-in On-Screen Keyboard off under Settings → Accessibility → Typing to avoid two keyboards.

If GNOME could not enable the extension until the new login, run:

gnome-extensions enable screen-keyboard@salix.host

To inspect the installer or build local changes:

git clone https://git.salix.host/salix/gnome-not-shit-screen-keyboard.git
cd gnome-not-shit-screen-keyboard
./install.sh

From a checkout, the installer uses that checkout, including local edits. Run git pull and ./install.sh again to update, then log out and back in. Settings and learned vocabulary are retained.

To install a different branch, use the same pattern as SearchLightNG:

curl -fsSL https://git.salix.host/salix/gnome-not-shit-screen-keyboard/raw/branch/master/install.sh | bash -s -- some-branch

Requirements

• GNOME Shell 50 on Linux. Earlier Shell versions are not supported.
• GCC and G++, CMake 3.22 or newer, Make, Git, Python 3 with venv support, and glib-compile-schemas.
• An internet connection for source, model and build-tool downloads.
• Several GB of free disk space for the first swipe build. Compilation can take several minutes. Build dependencies are cached for later updates.

Install these tools through your distribution's package manager. If Python has no venv support, an installed uv can supply the swipe build environment. The installer checks prerequisites and reports anything missing.

GPU predictions

The prediction helper uses Vulkan automatically when built with Vulkan support and a compatible GPU is available. This includes the Radeon 780M with Mesa's RADV driver. It falls back to CPU when no GPU is available or GPU model/context initialization fails. Swipe decoding uses its existing engine.

Install the optional build dependencies before running the installer:

# Fedora
sudo dnf install vulkan-loader-devel vulkan-headers glslc spirv-headers-devel
# Ubuntu / Debian
sudo apt install libvulkan-dev glslc spirv-headers

The target desktop also needs the Vulkan loader and its GPU's Vulkan driver. For AMD/Intel on Fedora or Ubuntu, install mesa-vulkan-drivers. XLM_VULKAN=ON ./install.sh requires a Vulkan build and reports missing build dependencies. The default, AUTO, builds CPU-only if they are missing or retries CPU-only if the automatic build fails; XLM_VULKAN=OFF ./install.sh explicitly builds CPU-only.

Native helpers are built for the installing computer's CPU. Build-only archives are local installation artifacts, not universal binaries for other computers.

The helper logs inference backend: Vulkan / <device> or inference backend: CPU to stderr. For direct helper tests, XLM_BACKEND=cpu forces CPU execution; XLM_BACKEND=auto is the default. GGML_VK_VISIBLE_DEVICES can restrict Vulkan device selection. Acceleration still needs latency and power measurements on your hardware, especially for this small model and its short prediction bursts.

Using the keyboard

Tap Float above the keys for the compact layout. Drag the handle below the keys to move it with one finger, or place two fingers on the handle and spread them apart to enlarge it or pinch inward to shrink it. The size is saved, and Preferences also offers a Floating keyboard size control from 75% to 175%. The keyboard stays within the available screen area and stays open when you click elsewhere. Use the hide button or Quick Settings toggle to dismiss it. Tap Dock to return to the full-width layout.

If Logia is installed, a microphone button appears on the keyboard. Tap it to start desktop dictation, then tap it again to stop and insert the transcript into the focused app. The button works with native and Flatpak Logia installations. Logia's GNOME Shell extension must be enabled for desktop dictation.

Choose your touchscreen under Default display in Preferences to keep both layouts on that screen, including when opening with an edge gesture. The choice is saved by display connector. If that display is disconnected, the keyboard uses the primary display until it returns. Automatic (focused window) keeps the existing display selection behavior.

Tap a suggestion to insert it. For swipe typing, start on a letter, slide across the word and lift your finger. The best candidate is inserted, and the suggestion bar offers alternatives. Press Backspace immediately afterwards to remove the whole word; swiping again offers the next candidate first. Long-press a letter for its accented alternatives or hinted number.

Spacing works like FUTO Keyboard. A swiped word is not followed by a space until you continue: the next swiped or typed word is separated automatically, while a comma, period or other punctuation attaches to the word and is followed by a space. Accepted suggestions and typed punctuation such as . , ? ! also add a space, and punctuation typed after a space moves before it. Tap Space twice for a period. Backspace immediately after a double-space period or a moved punctuation mark reverts it. Numbers, times and web addresses keep their punctuation together. Turn off Automatic spaces in Preferences to type spaces after punctuation yourself; swiped words and suggestions still separate the next word.

The emoji key opens a single scrolling list of emoji, like FUTO Keyboard. Drag or flick to scroll, tap to insert, or hold an emoji for its skin tones. The category buttons below jump to each category, and recently used emoji come first.

Hold Space briefly until it turns blue, then slide left or right to move the cursor through text, or up and down to move between lines. Release to finish. A quick tap still inserts a space. Cursor movement uses the application's arrow key behavior, so it also works without surrounding-text support.

Open preferences with:

gnome-extensions prefs screen-keyboard@salix.host

Preferences include floating mode, automatic opening, swipe typing, automatic spaces, the prediction model path, learning controls and forgetting a learned word.

Prediction and swipe typing require applications that expose input-method focus and surrounding text. Swipe typing starts outside an existing word and with no text selected. Terminals, numeric fields, email addresses and URL fields use ordinary typing. Moving focus or editing the text discards pending swipe results.

Install options

Option	Effect
JOBS=2	Limit compilation to two parallel jobs. Default is four.
WITH_SWIPE=0	Install without swipe typing or its build dependencies.
XLM_VULKAN=ON	Require Vulkan GPU support at build time. Default AUTO; OFF builds CPU-only.
WITH_ML4=0	Skip the ML4 download. Dictionary suggestions still work; an existing model remains usable.
ML4_FILE=/path/to/ml4_q6_k.gguf	Supply a local copy of the exact pinned ML4 model. Its checksum is verified.
BRANCH=name	Choose the source branch when downloading. Local checkouts use their own files.
SWIPE_BUILD_CACHE=/path	Store swipe build dependencies in another directory.

For example, from a checkout:

JOBS=2 ./install.sh

Or install only the keyboard and dictionary predictions:

WITH_SWIPE=0 WITH_ML4=0 ./install.sh

To compile and prepare a package without installing or enabling it:

./install.sh --build-only

This produces an extension archive in ~/.cache/screen-keyboard/ and prepares ML4 separately in the same cache. XDG_CACHE_HOME can change the cache root. Use ./install.sh --help for the available controls.

Models and storage

Ordinary predictions use FUTO ML4, specifically ml4_q6_k.gguf. The installer fetches it from FUTO's official resources repository, pinned to a revision and SHA-256 checksum. It also reuses a matching file already installed or in ~/Downloads. There is no need to find a model manually.

Powered by FUTO Swipe. Swipe typing uses the published FUTO Swipe models and swipe library. Like FUTO Keyboard, the helper maps QWERTY layouts close to FUTO's reference layout onto its specialized QWERTY decoder, including the docked and floating layouts. Other English key arrangements use the universal encoder with the actual key positions. Candidates keep dictionary capitalization and apostrophes, such as "I'm", "don't" and "Monday". Model files, metadata and licenses are included at install time.

Location	Contents
~/.local/share/gnome-shell/extensions/screen-keyboard@salix.host/	Extension, helpers, dictionary and swipe models. Respects XDG_DATA_HOME.
~/.local/share/screen-keyboard/ml4_q6_k.gguf	ML4 prediction model.
~/.local/share/screen-keyboard/learned.txt and learned.txt.pairs	Local vocabulary and word-pair history, with owner-only permissions.
~/.cache/screen-keyboard/	Download and build cache. Respects XDG_CACHE_HOME.

Troubleshooting

The extension is missing after installation. Log out and back in, then run gnome-extensions enable screen-keyboard@salix.host. Check that gnome-shell --version reports version 50.

Two keyboards appear. Disable GNOME's built-in On-Screen Keyboard in Accessibility settings.

Swiping does nothing. Check that swipe typing is enabled, the input source is English US or GB, and the cursor is outside a word. Allow the helper to start when first opening the keyboard. Some applications do not expose the text-input information required for recognition.

A build runs out of memory. Retry with JOBS=2, or JOBS=1 on smaller machines. The cached swipe dependencies are reused.

A model download fails verification. Retry the installer. If you supplied ML4_FILE, it must match the current pinned model. The older ml4_1_f16_meta_fixed.gguf is not a substitute for this installer.

Remove

gnome-extensions disable screen-keyboard@salix.host
gnome-extensions uninstall screen-keyboard@salix.host

Log out and back in. Models, learning history and build caches remain in the locations above, so uninstalling the extension does not erase your vocabulary.

Development

The keyboard starts from GNOME Shell 50.4's js/ui/keyboard.js and tracks upstream API changes for newer releases (GNOME 51 moves PopupMenu.open() and close() from an animation boolean to a {animate} params object), with editable JSON layouts and separate native inference processes. See development notes for integration details and prediction checks. The README describes implemented behavior; live GNOME touchscreen validation is still needed for the new swipe integration.

node --test tests/*.test.cjs
python3 tests/model-download.py
python3 tests/installer.py
./swipe-bridge/build.sh
python3 tests/swipe-bridge.py

The swipe regression check uses a small sample of donated FUTO test traces. It checks recognition, request validation, layout switching and vocabulary refresh; it is not a general accuracy benchmark.

License and credits

The extension is GPL-2.0-or-later, inherited from GNOME Shell. See LICENSE. The separate swipe helper and FUTO swipe library use GPL-3.0-or-later. Prediction libraries retain their MIT, Apache-2.0 and BSD licenses. FUTO model weights have their own terms; the swipe model license is included with downloaded assets. The donated swipe test fixtures include FUTO's MIT notice.

Thanks to GNOME for the keyboard foundation and FUTO for the on-device models and swipe implementation.