Tailscale on reMarkable 2

I spent a bit of time this morning wiring up a reMarkable 2 into my Tailscale network. Results here: it works in userspace networking mode.

The target

In the last couple years there seems to have been a run of “electronic notebooks”: tablet-sized devices with e-paper displays, writeable via stylus. Here’s an Engadget review of several; opinions are theirs, not mine.

I got a reMarkable 2 a while back, which has some nice properties and some rough edges. The steady stream of software updates has sanded off some of the latter.

Unlike the PineNote, the reMarkable isn’t open-source. However, they do allow root access to the device and its filesystem: when the device is plugged in, you can SSH in and overwrite / tweak / brick the device to your heart’s content. This is a great feature to have, and there seems to be a healthy little community that reverse-engineers pieces of the stack, documents what they find, and shares what they get up to.

Moving data around

I’m getting more serious about data backups, and I want more granular backups of my notes than reMarkable-the-company provides.

reMarkable-the-company offers a subscription-based cloud syncing service. I got my tablet before the “you have to pay for it” period, but it’s still “syncing”, not “backups”. And, of course, that data is wholly in their hands.

I have a NAS on my Tailscale network with a decent backup regime. I figured if I can get the reMarkable on the network, and add a cron on the reMarkable, I can use some of the awesome tooling folks have built to build a decent backup/sync pipeline outside of reMarkable-the-company’s regime.

Running on reMarkable

The “gotchas” I hit:

The reMarkable’s root partition is tiny and almost full. I built a combined binary and linked it from /home rather than installing to the root partion.
The Tailscale systemd unit isn’t sufficient- it needs an environment with some variables defined. As it turns out, that’s fine, because…
The reMarkable’s kernel doesn’t support tunnel devices (/dev/net/tun), so we have to run Tailscale in userspace networking mode by tweaking some flags.

You can skip to the end if you aren’t interested in the “why"s.

First build

Getting working binaries on the reMarkable was pretty easy. Once I had set up ssh, I found someone else who had built Go code for the reMarkable, and combined with the build instructions from Tailscale’s source.

This doesn’t work:*

CONTENT="$(mktemp -d)"
git clone --depth 1 https://github.com/tailscale/tailscale.git
cd tailscale
GOOS=linux GOARCH=arm GOARM=7 \
  go build \
  -o "$CONTENT" \
  ./cmd/tailscale{,d}

scped these binaries over and- great! They run; but…

Building, but better

I didn’t want to fiddle with PATH settings, so my plan was to install these in /usr/bin/tailscale and /usr/sbin/tailscaled (the default locations). Alas, the binaries filled up the remains of the root filesystem:

∵ ls -lh $CONTENT/
total 40M
-rwxr-xr-x 1 cceckman users 15M May 20 11:32 tailscale
-rwxr-xr-x 1 cceckman users 25M May 20 11:32 tailscaled

reMarkable: ~/ df -h /
Filesystem                Size      Used Available Use% Mounted on
/dev/root               257.7M    233.4M      6.8M  97% /

Tailscale’s wonderful knowledge base to the rescue! This article describes how to build a single binary that acts as tailscale or tailscaled based on how it’s invoked; and points to the release script that can discard some features.

A little copy-and-paste later, and we have a new build command that works:

GOOS=linux GOARCH=arm GOARM=7 \
  go build \
  -o "$CONTENT"/tailscale.combined \
  -tags ts_include_cli,ts_omit_aws,ts_omit_tap,ts_omit_kube \
  -ldflags "-w -s" \
  ./cmd/tailscaled

That gives us:

-rwxr-xr-x 1 cceckman users 15M May 20 11:32 tailscale
-rwxr-xr-x 1 cceckman users 20M May 20 11:43 tailscale.combined
-rwxr-xr-x 1 cceckman users 25M May 20 11:32 tailscaled

20MiB instead of a total of 40MiB - I’ll take it!

Golang’s “single static binary” is both a bane and a boon for this project.

On the one hand, it means that I can build the Tailscale binary on my laptop and copy it over, without worrying about headers or linkage or the particular Linux flavor that reMarkable uses.

On the other hand… it’s still a 20 MiB binary; and without fiddling with what binary I use, we’re apparently duplicating 20MiB of text/data. Eugh.

That’s still too much data for the root partition, though. Luckily, the reMarkable’s /home partition is bigger - several GiB, shared with user data:

reMarkable: ~/ df -h /home
Filesystem                Size      Used Available Use% Mounted on
/dev/mmcblk2p4            6.4G      1.3G      4.8G  22% /home

In my install script, I changed to link rather than copy:

TSINSTALLPATH="~/tailscale"
ln -sf $TSINSTALLPATH/tailscale.combined /usr/bin/tailscale
ln -sf $TSINSTALLPATH/tailscale.combined /usr/sbin/tailscaled

A little less space for papers, but it leaves space for software updates.

Running the service

I started by using the tailscaled.service unit without modification, but hit some surprises when I went to enable --now:

EnvironmentFile=/etc/default/tailscaled
ExecStart=/usr/sbin/tailscaled --state=/var/lib/tailscale/tailscaled.state --socket=/run/tailscale/tailscaled.sock --port=${PORT} $FLAGS

Systemd noticed the EnvironmentFile was missing:

May 20 15:55:56 reMarkable systemd[1]: tailscaled.service: Failed to load environment files: No such file or directory

The service expects /etc/default/tailscaled to be populated with a port number (for the control server) and some flags. I copied what looked like the relevant bits from a nix-based install.

This doesn’t work:*

cat <<EOS >/etc/default/tailscaled
PORT=41641
FLAGS="--tun tailscale0"
EOS

Userspace networking

tailscaled was, at this point, repeatedly restarting:

May 20 15:58:36 reMarkable tailscaled[5653]: is CONFIG_TUN enabled in your kernel? `modprobe tun` failed with: modprobe: FATAL: Module tun not found in directory /lib/modules/5.4.70-v1.3.4-rm11x
May 20 15:58:36 reMarkable tailscaled[5653]: wgengine.NewUserspaceEngine(tun "tailscale0") error: tstun.New("tailscale0"): CreateTUN("tailscale0") failed; /dev/net/tun does not exist

Sure enough, there doesn’t appear to be TUN support on the tablet:

reMarkable: ~/ ls /dev/net/
ls: /dev/net/: No such file or directory

Again, KB to the rescue! The article on userspace networking provides some flags to make tailscaled act as a SOCKS5 and HTTP(S) proxy- and we’ve just set up the environment file that lets us provide those flags:

cat <<EOS >/etc/default/tailscaled
PORT=41641
FLAGS="--tun userspace-networking --socks5-server=localhost:1055 --outbound-http-proxy-listen=localhost:1055"
EOS

With this configuration, tailscale up provided a friendly login link, I was able to get to the device with SSH, and I was able to curl to another device in my tailnet.

Results

Here’s a gist of the full script; ./rm2-tailscale.sh <IP or hostname of remarkable> should do it, if you have SSH to the device set up. I release that script into the public domain, so go wild with updates / tweaks.

Any updates I make are likely to land in my homelab repo, here.

Next steps

The reMarkable install doesn’t appear to be doing the MagicDNS thing; I haven’t looked at why.

And I still need to set up a path for backups something to bring the data back to storage. Next time!