I tried out Tailscale’s experimental in-process networking support. It did
what I wanted: I was able to serve HTTPS to my Tailscale network from an
unprivileged process. I also wrote a small program, tsproxy
, to provide the
same for any server; this let me “install” code-server on a Chromebook, but
with the compute power of another host.
This post walks through what I was trying to do, and what I learned along the way.
TLS over Tailscale
I’ve been using Tailscale for a while now, mostly to SSH between my laptop and my development machine. I wanted to also use it to run a reading list server, so the admin interface (add/edit/publish entries) is never visible outside “my devices”.
A key portion of flow I want for a reading list is “share-to” support: open a link and send it to “read later”. This has a bunch of requirements stemming from the notion of progressive web apps; most importantly for connectivity purposes, it requires that the site is served via HTTPS. 1
Luckily, Tailscale has recently (fall 2021)
launched a feature that lets servers in
your Tailscale network get valid TLS certificates from Let’s Encrypt, at
hostnames like <host>.<network>.ts.net
. I was able to pretty quickly walk
through the example, reach my server in the browser (via HTTPS!), and
“install” it.
Great success!
In which cceckman is picky
…but I wasn’t satisfied.
My main concern was around automation and permissions. At first glance, the
servetls
example expects to have access to the tailscaled
daemon in order
to acquire a TLS certificate; that’s not easily possible on my system.2
There are workarounds - regenerating certificates with a cron, chmod
,
something something user namespace - but I didn’t want to maintain even more
mechanism.
The other problem is identity.3 In the above configuration, the installed app was found at
https://<hostname>.<tailnet>.ts.net:<five-digit port>
The name doesn’t show up often, so it’s not the worst name. But that presentation doesn’t match my plans for this app: I’m not going to be running this service on my desktop in the longer term, and I’m not huge on saving / hard-coding a port.
On a more abstract level, I’m used to thinking of DNS as naming “what you want” rather than “how you get there”. If “what you want” is a particular host, sure, use the hostname. But my mental model for this server is that it’s a service, not a daemon- it’s not logically tied to the host.
What I’d really like to do is put just this server on my Tailscale network- without having the client know the particular host / port incantation, and without the server having host-level credentials.
Experiments with tsnet
Of course, that functionality is already on the roadmap; Tailscale provides
experimental (!) code for offering a service directly to the network. The
tsnet package provides a net.Listener
with a configurable hostname, and
tailscale.GetCredentials
knows how to do-the-right-thing for such a
connection.
The net result is that the server can be reached at the much-prettier address:
https://<servername>.<tailnet>.ts.net
without any additional host-level plumbing: no installing Tailscale, tweaking permissions or ownership, certificate crons, etc.
I started by working through the example. As expected for experimental code, I had to do a little finagling to get it to work:
-
The
tsnet
package won’t do anything unless the user acknowledges that it’s experimental code (by setting an environment variable). This is A Good Thing- it forces the user to set their expectations to “ready for breakage”.I’m intentionally going to leave that environment variable out of this post. Copy-pasting won’t work- if you want to try out this experimental feature, you’ll also need to accept the risk.
-
The first time I ran the program, I had to do some tweaks to authenticate.
A server or host connecting to Tailscale needs to authenticate to the control server; once it’s known to the control server it can use locally-stored credentials to authenticate. The human-facing
tailscale
CLI or gui will automatically prompt for login; but by default, it seems that atsnet
-using program will just stall out waiting for login.It seems that
TS_LOGIN=1
will causetsnet
to approximate the CLI’s “go to a link” registration flow; with that environment variable, within the slew of logs, I found a link tohttps://login.tailscale.com
that completed the authentication flow for the program. But this wasn’t super friendly- I had to watch through a somewhat verbose log stream to get the URL.I had better success registering an auth key for Tailscale and providing it to the process via the
TS_AUTHKEY
environment variable (e.g.TS_AUTHKEY=tskey-asdf-jlksemicolon
). For testing, I used the “ephemeral” key typ, so the server automatically disappeared from my network when the processe exited. For “real” use, a one-off key seems like the way to go; after the initial enrollment, a persistent saved directory will keep long-term credentials.For the future of this feature (hi Tailscale team!), it would be nice to have more of this accessible programmatically. At least, I think I’d prefer an error rather than stall if there’s no authentication path available; a more complicated flow where
tsnet
returns a login URL might facilitate friendlier flows.
After working through the example, I ported my own server to tsnet
; you can
see the results in commit faa374.
Expanding the network: tsproxy
I did most of this development remotely: my eyes and hands at my laptop, but files and servers on another machine. Over the last couple years I’ve experimented with various ways to accomplish this, with varied results;4 so far, I’ve settled on using code-server as an “installed” web app. (“Installed” rather than “in-browser” allows keybindings to behave like a local program - very useful for an IDE.)
I’d had a script to make this work with Tailscale’s creds
feature, but it was
rather clunky, using sudo tailscale creds
and moves/chown
s to get the keys
to code-server
.5 Again, my problem is getting just this one server
onto my network - is there a way to do that without diving into code-server
’s
source?
“Yes”, it turns out. I didn’t golf, but a simple proxy took me a little more than 100 lines of Golang code. An example invocation, after the initial authentication:
nohup code-server 2>&1 --socket ~/.code-server.sock &
tsproxy --from code-server --to ~/.code-server.sock
That gets me https://code-server.<my-network>.ts.net
as an install-friendly
URL.
Please be wary of using this program; it was hacked, not engineered. (No one has reviewed the code; it has no tests or continuous builds; etc etc.) But if you’re interested, the code is at https://github.com/cceckman/tsproxy; and it does seem to work for this use case.
Ur doin it wrong (?)
I’m sure there are other ways to do what tsproxy
does. As I was writing this
post, Tailscale’s December newsletter came out, pointing to
this tool for running Tailscale
in Docker. More generally, if I understand correctly, it’s possible to run
multiple tailscaled
s in different network namespaces - providing even stronger
isolation around the proxied server’s connectivity.
But…well, I understand net.Listener
and a two-flag program much better than
I understand network namespaces. tsproxy
gets me a useful result without much
work; and for now, that’s good enough for me. If anyone has other suggestions,
feel free to reach out - I am always interested in learning new things!
Comments? Ideas?s See cceckman.com for my contact info.
-
“share-to” support doesn’t appear to be a fully standardized feature. MDN doesn’t seem to cover it, but web.dev does- so “supported in the Chrome/Android suite”, at least, which is enough for me for now. ↩︎
-
As noted elsewhere, WSL2 isn’t a super friendly environment for Tailscale (no systemd), though it’s workable. As I was writing this post I learned of this project to allow processes in WSL2 to interact with the host (Windows)
tailscaled
; I might give it a go! ↩︎ -
Or naming, per the oft-repeated saying. ↩︎
-
The issues I’ve hit:
At some point, the Chrome OS shell seems to have acquired an off-by-one error in its cursor or text rendering. As the cursor progresses to the right, the cursor becomes steadily more out of sync with the text. This put me off from using SSH / shell directly for most editing.
I used VS Code in a Crostini VM for a while. However, this seemed to have pretty high input / response latency- enough to be a bit of a distraction. My unfounded suspicion is that this leads to running two copies of the Chrome rendering stack, one for the OS-managed Chrome instance, and another (Electron) in the VM (proxied via various layers). As a web app,
code-server
uses the host system’s rendering engine, shared libraries, etc.This is just my impression / guesses, not an empircal measurement - though I intend to do some experiments in that (or this) vein in the future. ↩︎
-
This would’t be too bad on its own, but the solution I had found for reaching into WSL2 wound up running two
tailscaled
s, with lots of manual startup steps. I’m sure there’s many better ways! ↩︎