Ecosyste.ms: Awesome

An open API service indexing awesome lists of open source software.

Awesome Lists | Featured Topics | Projects

https://github.com/pl-semiotics/rM-vnc-server

Damage-tracking VNC server for the reMarkable tablet
https://github.com/pl-semiotics/rM-vnc-server

remarkable-tablet

Last synced: 25 days ago
JSON representation

Damage-tracking VNC server for the reMarkable tablet

Awesome Lists containing this project

README 

        
# Introduction



The [reMarkable tablets](https://remarkable.com) provide a very good

experience for digital writing/drawing. Unfortunately, sharing drawing

in real-time is difficult, especially on Linux, where the official

reMarkable cloud streaming does not work.



Since the reMarkable uses an e-paper display, it's possible to extract

high quality damage tracking information from the software responsible

for driving the display. This repository uses that information and

[libvncserver](https://libvnc.github.io) to implement a simple

well-performing VNC server. This allows streaming the display contents

off the device to any VNC/RFB client. Performance is quite good, at

least when using the ZRLE compression built into libvncserver; over

USB connections, drawing is quite smooth, with negligible latency.

Large framebuffer updates, such as opening or closing a notebook, are,

in fact, frequently displayed on the client before the reMarkable

screen has finished refreshing. The code has been less well tested

over WiFi connections; drawing over an SSH tunnel over a WiFi

connection seems to occasionally stutter due to unpredictable latency,

although unencrypted connections still seem to perform well.



Dependent projects provide the damage-tracking information that is

required. For the rM1, the required information is extracted from the

kernel, using

[mxc_epdc_fb_damage](https://github.com/pl-semiotics/mxc_epdc_fb_damage).

For the rM2, the epaper display is driven from userspace software,

rather than from the integrated e-paper display controller on the

processor (which the rM1 used), and so damage information is

unfortunately not available outside of the process driving the

display. Currently, all known deployed drivers for the display (most

noticeably `xochitl`, the official interface on the rM), use a

proprietary library `libqsgepaper.a` to draw to the display. The rM2

version of the server therefore uses

[libqsgepaper-snoop](https://github.com/pl-semiotics/libqsgepaper-snoop),

which injects a damage-reporting stub into processes that use

`libqsgepaper.a` to control the rM2 framebuffer.



## Input support



The VNC server also supports injecting input from the VNC clients into

the reMarkable's input devices, using

[rM-input-devices](https://github.com/pl-semiotics/rM-input-devices).

Cursor movements with button 3 active are sent to the tablet as touch

events. Cursor movements with button 1 active are sent as wacom stylus

contacts at full pressure (allowing rudimentary drawing). Cursor

movements with button 2 active are sent as wacom stylus

movement-within-proximity-but-without-contact events. Keyboard keys

left/up/page up are sent as a left key, which moves back a page in the

current notebook; right/down/page down similarly are sent as a right

key to move forward a page. Home and escape are sent as the home key,

which generally returns to the home page of xochitl.



## Cursor location reporting



If a VNC client supports the cursor position pseudo-encoding, the

position of the wacom stylus is sent to it. Unfortunately, sending pen

status such as in/out-of-proximity/contact events is not easy to do in

the current protocol; a protocol extension may be warranted if clients

would find such information useful.



# Building



The supported way to build this is via the

[Nix](https://nixos.org/nix) package manager, through the

[nix-remarkable](https://github.com/pl-semiotics/nix-remarkable)

expressions. To build just this project via `nix build` from this

repo, download it into the `pkgs/` directory of `nix-remarkable`.



There are four variants that can be built: for either rM1 or rM2, and

in a "standalone" configuration or not. The standalone configuration

will statically link against all libraries not installed on the

reMarkable by default, and will bundle necessary kernel modules and

tools; it is recommended unless you have a particular need for

sharing. By default, the Nix expressions will build all variants.



For other systems, the commands needed to compile and link are given

in the [Makefile](./Makefile). Build dependencies depend on the

configuraiton being built, but all versions require libvncserver

(built for the reMarkable) and the reMarkable kernel headers as well

as the `rM-input-devices.h` from

[rM-input-devices](https://github.com/pl-semiotics/rM-input-devices).



For the reMarkable 1, the `mxc_epdc_fb_damage.h` file from

[mxc_epdc_fb_damage](https://github.com/pl-semiotics/mxc_epdc_fb_damage)

is also required; for the reMarkable 2, the `libqsgepaper-snoop.h`

file from

[libqsgepaper-snoop](https://github.com/pl-semiotics/libqsgepaper-snoop)

is needed instead.



For all versions, the -standalone versions require the standalone

static library versions of the relevant

dependencies---[rM-input-devices](https://github.com/pl-semiotics/rM-input-devices)

for both,

[mxc_epdc_fb_damage](https://github.com/pl-semiotics/mxc_epdc_fb_damage)

for the rM1, and

[libqsgepaper-snoop](https://github.com/pl-semiotics/libqsgepaper-snoop)

for the rM2.



Prebuilt binaries are available in the [Releases

tab](https://github.com/pl-semiotics/rM-vnc-server/releases).



# Usage



Copy the relevant executable to the device and run it; this will start

a vnc server listening on port 5900. Any VNC client should be able to

provide a (view-only) view of the tablet's screen when pointed at the

reMarkable's IP address and standard VNC port (5900).

[Vinagre](https://gitlab.gnome.org/GNOME/vinagre),

[Remmina](remmina.org), and

[gst-libvncclient-rfbsrc](https://github.com/pl-semiotics/gst-libvncclient-rfbsrc)

have been tested.



Note that the server will bind to all available network interfaces by

default, so only run this if the tablet's WiFi is disconnected or

connected to a trusted network. iptables rules could be used to

restrict traffic.