About wayland / weston
This chapter does not pretend to cover all the differences between X11 and wayland, it just tries to give the big details.
The goal of wayland is to replace X11, it is like the core of X11: an IPC (Inter Process Communication) mecanism between graphical applications.
Taken from the wayland documentation, here's the workflow when an X11 application wants to notify a graphical change:
You can see that you have plenty of communications and roundtrips until the modified bits reach the hardware.
With wayland, it goes much faster:
The philosophy of wayland is "every frame is perfect", the compositor will only send a finalized frame to the hardware. Forgot about tearing, flashes and graphical artifacts you can see under X11 when fastly moving a window.
The RDP backend
The weston project aims to be a reference compositor for wayland. The code should be clear and functional, and give enough material for people that want to implement their own compositor. The project implement a reference shell for a desktop (desktop-shell) and already have some backends for:
- DRM / KMS: to drive these API for video;
- udev: to access to input devices (mouse and keyboard);
- fbdev: the good old framebuffer;
- rpi: a backend to use that particular hardware.
In weston a backend implements access to seats (mouse, keyboard, ...) or output devices (GPU most of the time). The RDP compositor is based on the FreeRDP project to provide the RDP transport. This backend is headless, it uses a memory buffer as a video card and wires a seat per connected RDP peer. It uses the pixman renderer that will use the surfaces sent by wayland clients and will compose the desktop's picture. That means that even before we have any connected RDP client, the full picture is already computed.
The backend listens for incoming RDP connections, and instantiates a seat (mouse and keyboard) for each incoming peer. Each connected peer has its own pointer and focus (having 2 pointers is really funny).
When a wayland client as some activity, it notifies the compositor that will rebuild the desktop picture. It takes all the list of surfaces in z-index order and compose until the full screen has been painted (or when there's no more surfaces). The touched part of the screen are sent to the RDP peers, they're encoded either in raw format or using the NS or remoteFx codecs depending of the client capabilities. The compositor also handle the SUPPRESS_OUTPUT message sent by RDP clients when they don't want screen updates (this is the case when the window of a RDP client is iconified).
Limitations and future
Theoretically with RDP you can have client-side pointer support, but adding in weston is not that easy:
- for weston the mouse pointer surface is anonymous, so when rendering the full scene we should draw everything but this surface. Moreover with the multi-seat support we should maintain an image per seat, and draw everything but the surface of the current seat.
- RDP cursors can only be bitmaps or bitmasks so to support client-side pointer, we should scan the surface to check that it matches that requirement.
For now we disable the client-side pointer completely, so when moving the mouse, weston does refreshes by sending pixmap updates.
Some ideas for the RDP compositors:
- map the RDP clipboard channel in wayland, to have copy'n paste between a RDP peer and the wayland applications;
- RDP8 has introduced mode switches initiated by the RDP client, everything is ready for that in weston and in the development branch of FreeRDP;
- implement a frame synchronization mecanism, as in the current code, weston sends as many frames as it can even if the network or the destination peer can't handle them;
- create a wayland extension of the protocol to access to RDP channels. I'm dreaming of a modified VLC that would use the TSMF channel (multimedia extension) and would play videos with the video decoding being done on the RDP client side.
Here's some good links on the subject: