Wayland RDP Server with audio using Microsoft's WSLg stack
I spent some time playing with the Microsoft's modified Weston/PulseAudio/FreeRDP stack. This is what Windows is using to display GUI Linux apps under WSL. I primarily use Linux and I use Linux remote desktops quite a bit. I use xrdp, freerdp, x11vnc, vinagre, guacamole for different things. I've tried GNOME's built in RDP server and freerdp2-shadow-x11. They all work well enough. xrdp is best so far, but it's a PITA to set up and the stack is kind of heavy. I know it's possible to get audio and H264 video working with xrdp, but I have not tried it yet. I am curious about a Wayland RDP server. I installed Weston (the reference Wayland compositor) on Debian and was shocked at how easy it was to get an RDP server up and running. Weston pulls in few dependencies, it's quite small, and has an RDP backend baked in that works quite well out of the box. Just run "weston --backend=rdp-backend.so". Depending on the version, it may be necessary to generate a key file, which is easy. There is no authentication, it just fires up an RDP server logged in as the current user. This is perfect for me since I'm using wireguard for network level authentication. It supports the RemoteFX codec, but not the progressive codec, and no H264 codec, no sound. Not bad.
It annoys me to say so, but Windows has the best remote desktop, MS designed RDP after all. It's awesome that MS is integrating Linux these days, and I was curious how WSL works. It turns out they are using RDP. They designed a new extension called rail that send each window individually, to make it seamless. They have forks of Weston, PulseAudio, and FreeRDP up on github, with all of their modifications to the Linux side of the RDP stack. Interesting...
After poking around in my free time last week, I got them up and running on a minimal Debian bullseye install. Microsoft compiles minimal binaries, without some of the goodies that I want, so after trying those I decided to compile from source. Their stack is designed to be run on the same host, so they aren't worried about network latency at all. I had to make some small tweaks to get it working well with FreeRDP. Audio is uncompressed PCM (no easy fix, afaik). Audio didn't play well with the FreeRDP pulse client plugin (easy fix). RemoteFX is hard-coded to disabled (easy fix).
Firefox works great. I can stream youtube videos in 1080p with audio on the same network and can't even tell it's remote.
Here is my shell script that sets it up:
https://jacobstoner.com/msweston.sh
Run it as root on Debian bullseye. It downloads, tweaks, compiles, and installs Microsoft's forks of weston, pulseaudio, and FreeRDP and sets them up to run as root on boot. You could also just use it as a guide, line-by-line, that's probably a good idea anyway.
MS uses a supervisor process that sets everything up, keeps it connected, and restarts things as needed. That won't work for us since we aren't connecting to Windows. During testing Weston stopped accepting clients a couple times when trying unsupported client options and it was necessary to killall -9 weston. This is just for testing, it is not meant as a production environtment, espeically because it's running as root; but you could totally build a production worthy terminal server around these tools.
To connect using freerdp, I've had the best results with this command:
xfreerdp /v:10.0.1.135 /cert-ignore /size:1024x768 /f /floatbar:sticky:off,show:always /sound:latency:200 -gfx /rfx
The thing that bothers me the most is the inability to resize the client desktop size after connection. Of everything that I have tested, that only works with xrdp and Windows. It would also be nice if the audio stream was compressed and if progressive or H264 video encoding was supported. It would be cool to get rail support in the freerdp client, that would open up some interesting options for virtualizing apps. Hopefully the MS patches get merged upstream and more community work can be done on this.