digitalmars.D - Why I love D: interfacing with XCB
- H. S. Teoh (209/209) Apr 06 2022 Background: XCB is a library for interfacing with the X11 windowing
- Adam D Ruppe (5/11) Apr 06 2022 Those alleged issues are pretty overblown. My biggest sadness
- H. S. Teoh (49/62) Apr 06 2022 I suspect some of the issues have since been solved, since Xlib nowadays
- Stanislav Blinov (6/22) Apr 06 2022 6) If you care about key repeat events (which most applications
Background: XCB is a library for interfacing with the X11 windowing
system on *nix systems. The original core library is Xlib, which is
still in widespread use but is dated and suffers from a variety of
issues. XCB is an alternative intended to provide a better, albeit
lower-level API. Recent releases of Xlib are actually implemented using
XCB.
XCB is an asynchronous (non-blocking) API that maps almost 1-to-1 to the
X11 protocol, so its API functions come in pairs: one to send the
request, another to check the response. Almost all of XCB's API is
auto-generated from XML files describing the X protocol, so the API has
quite a lot of recurring elements that lends itself very well to
automation with D's metaprogramming capabilities. In particular, there
are two main categories of API functions:
1) Functions corresponding with X11 protocol requests that don't expect
a reply. These are of the form:
xcb_void_cookie_t xcb_{requestName}(
xcb_connection_t* conn, ... /* params */);
xcb_void_cookie_t xcb_{requestName}_checked(
xcb_connection_t* conn, ... /* params */);
The first variant is for requests whose response you don't care about.
The second variant lets you check for the server's response using
xcb_request_check().
2) Functions corresponding with X11 protocol requests that expect a
server reply (usually returning data from the server). These come in
pairs:
xcb_{requestName}_cookie_t xcb_{requestName}(
xcb_connection_t* conn, ... /* params */);
xcb_{requestName}_reply_t xcb_{requestName}_reply(
xcb_connection_t* conn,
xcb_{requestName}_cookie_t cookie,
xcb_generic_error_t* err);
(Actually it's not just pairs, there's also the corresponding
xxx_unchecked() functions, but let's keep it simple here.)
As you can imagine, this API, while very flexible in letting you
parallelize multiple server roundtrips (thus reducing pause times
waiting for the server to respond before you send the next request),
leads to rather verbose code, for instance:
auto cookie1 = xcb_query_tree(conn, winId);
auto cookie2 = xcb_get_property(conn, winId, ...);
auto cookie3 = xcb_map_window_checked(conn, winId);
... // send other requests
// Now process responses
xcb_generic_error_t* err;
auto resp1 = xcb_query_tree_reply(conn, cookie1, &err);
if (err) { ... /* handle error */ }
... // process resp1 here
free(resp1);
auto resp2 = xcb_get_property_reply(conn, cookie2, &err);
if (err) { ... /* handle error */ }
... // process resp2 here
free(resp2);
err = xcb_request_check(conn, cookie3);
if (err) { ... /* handle error */ }
writeln("window mapped");
The separation of request from response handling also makes such code
hard to read (you have to look in two different places to figure out
which request pairs with which response).
But thanks to D's metaprogramming capabilities, we can automate away
most of this boilerplate, into something like this:
void delegate()[] futures;
futures ~= XCB.query_tree(conn, winId, (resp1) {
... // process resp1 here
});
futures ~= XCB.get_property(conn, winId, ..., (resp2) {
... // process resp2 here
});
futures ~= XCB.map_window(conn, winId, {
writeln("window mapped");
});
// Actually run the response handlers.
foreach (f; futures) { f(); }
The idea is to use .opDispatch to auto-generate the calls to the
underlying XCB functions, pairing the response code with the request
code so that boilerplate like error handling and free()ing the response
packet can be automated away.
Of course, in order to reap the benefits of parallelizing server
requests with XCB, we don't actually run the response handlers
immediately; instead, we wrap the response code in a delegate that gets
returned to the caller, a kind of "future" that runs the response
handler later. These we collect into an array that gets run at the end
of the block of outgoing requests. `void delegate()` was chosen in order
to have a uniform future type for any kind of request/response pair, so
that we can schedule blocks of requests/responses however we like
without having to worry about matching up cookie and response types.
(Conceivably, this API could be improved even further by keeping the
futures array in the `XCB` wrapper itself, with a .flush method for
flushing the queued response handlers. Or insert them into the event
loop.)
The definition of the `XCB` wrapper is:
----------------------------
struct XCB
{
enum OnError { warn, exception, ignore }
private static void handleError(OnError onError, lazy string errMsg)
{
final switch (onError)
{
case OnError.exception:
throw new Exception(errMsg);
case OnError.warn:
stderr.writeln(errMsg);
break;
case OnError.ignore:
break;
}
}
/**
* Syntactic sugar for calling XCB functions.
*
* For every pair of XCB functions of the form "xcb_funcname" taking
* arguments (Args...) and "xcb_funcname_reply" returning a value of type
* Reply, this object provides a corresponding method of the form:
*
* ------
* void delegate() XCB.funcname(Args, void delegate(Reply) cb,
* OnError onError)
* ------
*
* The returned delegate is a deferred computation object that, when
* invoked, retrieves the reply from the X server and invokes `cb` with the
* reply object if the reply is valid, or else takes the action specified
* by onError, the default of which is the throw an exception.
*
* For every XCB function of the form "xcb_funcname_checked" that do not
* generate a server reply, this object provides a corresponding method of
* the form:
*
* ------
* void delegate() XCB.funcname(Args, void delegate() cb, OnError onError)
* ------
*
* The returned delegate is a deferred computation object that, when
* invoked, retrieves the reply from the X server and invokes `cb` if the
* request was successful, or else takes the action specified by onError,
* the default of which is the throw an exception.
*/
template opDispatch(string func)
{
enum reqFunc = "xcb_" ~ func;
alias Args = Parameters!(mixin(reqFunc));
static assert(Args.length > 0 && is(Args[0] == xcb_connection_t*));
enum replyFunc = "xcb_" ~ func ~ "_reply";
static if (__traits(hasMember, xcb.xcb, replyFunc))
{
alias Reply = ReturnType!(mixin(replyFunc));
static void delegate() opDispatch(Args args,
void delegate(Reply) cb,
OnError onError = OnError.warn)
{
auto cookie = mixin(reqFunc ~ "(args)");
return {
import core.stdc.stdlib : free;
xcb_generic_error_t* err;
Reply reply = mixin(replyFunc ~ "(args[0], cookie, &err)");
if (reply is null)
handleError(onError, "%s failed: %s".format(
reqFunc, err.toString));
else
{
scope(exit) free(reply);
cb(reply);
}
};
}
}
else // No reply function, use generic check instead.
{
static void delegate() opDispatch(Args args,
void delegate() cb = null,
OnError onError = OnError.warn)
{
auto cookie = mixin(reqFunc ~ "_checked(args)");
return {
xcb_generic_error_t* e = xcb_request_check(args[0],
cookie);
if (e !is null)
handleError(onError, "%s failed: %s".format(
reqFunc, e.toString));
if (cb) cb();
};
}
}
}
}
----------------------------
As you can see, the XCB struct uses introspection to figure out which
category of functions are needed, figures out the required parameter
types, inserts extra parameters for the response handler callbacks,
etc.. I also threw in some rudimentary error-handling configurability so
that the user can decide which server errors are fatal, which should
issue a warning, and which can be safely ignored.
Armed with the above wrapper, writing an X11 application with XCB
becomes almost as easy as using Xlib (albeit with some wrinkles -- see
below [*]), but reaping all the perks of higher parallelism.
And all this is thanks to D's awesome metaprogramming capabilities, and
built-in support for delegates. (Without delegates and GC, it would be
impossible to achieve the conciseness shown above.)
D is awesome.
//
[*] In spite of its flaws, Xlib does provide some really nice high-level
functions that hide away a lot of protocol-level complexity; XCB has no
such facility so you either have to "cheat" by calling Xlib
occasionally, or reimplement the needed functions yourself. One such
pain area is keyboard handling. Won't get into the dirty details here,
but if anybody's interested, just ask. ;-) (I *did* manage to get
rudimentary keyboard handling without touching Xlib at all.)
T
--
"Hi." "'Lo."
Apr 06 2022
On Wednesday, 6 April 2022 at 18:42:42 UTC, H. S. Teoh wrote:The original core library is Xlib, which is still in widespread use but is dated and suffers from a variety of issues.Those alleged issues are pretty overblown. My biggest sadness with it is it assumes i/o failures are fatal, but you can kinda hack around that by throwing a D exception from the handler lol.One such pain area is keyboard handling. Won't get into the dirty details here, but if anybody's interested, just ask. ;-) (I *did* manage to get rudimentary keyboard handling without touching Xlib at all.)keyboard handling is kinda inherently messy
Apr 06 2022
On Wed, Apr 06, 2022 at 07:10:36PM +0000, Adam D Ruppe via Digitalmars-d wrote:On Wednesday, 6 April 2022 at 18:42:42 UTC, H. S. Teoh wrote:I suspect some of the issues have since been solved, since Xlib nowadays is implemented on top of XCB. :-PThe original core library is Xlib, which is still in widespread use but is dated and suffers from a variety of issues.Those alleged issues are pretty overblown. My biggest sadness with it is it assumes i/o failures are fatal, but you can kinda hack around that by throwing a D exception from the handler lol.Yeah, but with X (like *core* X protocol, without Xlib) it's a whole new level of messy that I hadn't expected before. :-P Basically, the X server *only* trades in keycodes, which are arbitrary numbers associated with physical keys on your keyboard. To make any sense of these keycodes, you need to map them to keysyms, i.e., standard numbers representing named keys. In Xlib, mapping a keycode to a keysym is just 1 or 2 function calls away. In XCB, however, you have to reinvent what Xlib does under the hood: 1) First, you have to retrieve the keyboard mapping from the X server, which is a table of keysyms that a particular keycode may map to. 2) But each keycode may map to ≥4 keysyms; to decide which one is in effect at the time of a keypress, you need to look at the current modifiers in effect (shift, capsLock, etc), and based on the algorithm described in the X protocol, decide which of 4 keysyms it will be. This would've been a trivial task, except that the meaning of the modifier bitmask may change depending on server configuration; in order to figure out which bit corresponds with the "mode switch" modifier, you need to ask the X server for the modifier map, and scan the table for the "mode switch" keysym and remember which bit it corresponds to. 3) You'd think that solves the problem, but no, there's more. Both the modifier map and the keyboard map may change over time, so you also have to process MappingNotify events, and reload the relevant portions of the keyboard map or the modifier map (and potentially refresh your current understanding of modifier bits). One potential gotcha here is that all subsequent KeyPress events after the MappingNotify will use the new mapping, so you have to make sure you refresh the keymap and modmap *before* processing any subsequent KeyPress events. 4) But even after this, you're not quite done yet. Most applications don't know nor care what keycodes or keysyms are; they usually want to translate keysyms into actual characters (ASCII, Unicode, what-have-you). To do this, you need to use a table of keysym -> ISO 10646 (i.e., Unicode) values: https://www.cl.cam.ac.uk/~mgk25/ucs/keysym2ucs.c 5) This isn't all there is to it, though. There's also XKB and the Input extension, which you will need if you want to support things like combining characters, dead keys, and input methods. (I haven't gotten that far yet, 'cos with (1)-(4) I already have a pretty workable system. XKB, if the server has it (pretty much all modern servers do), simulates most of its functionality via X11 core protocol to clients who didn't enable the XKB extension, so keyboard input will "mostly work" at this point, as long as you handle (1)-(4) correctly.) So there you have it, X11 keyboard handling without Xlib in a nutshell. ;-) T -- Ph.D. = Permanent head DamageOne such pain area is keyboard handling. Won't get into the dirty details here, but if anybody's interested, just ask. ;-) (I *did* manage to get rudimentary keyboard handling without touching Xlib at all.)keyboard handling is kinda inherently messy
Apr 06 2022
Stanislav Blinov <stanislav.blinov gmail.com> On Wednesday, 6 April 2022 at 20:20:26 UTC, H. S. Teoh wrote:1) ... 2) ... 3) You'd think that solves the problem, but no, there's more. Both the modifier map and the keyboard map may change over time, so you also have to process MappingNotify events, and reload the relevant portions of the keyboard map or the modifier map (and potentially refresh your current understanding of modifier bits). One potential gotcha here is that all subsequent KeyPress events after the MappingNotify will use the new mapping, so you have to make sure you refresh the keymap and modmap *before* processing any subsequent KeyPress events. 4) ... 5) ... So there you have it, X11 keyboard handling without Xlib in a nutshell. ;-)6) If you care about key repeat events (which most applications that deal with text input should, outside of, perhaps, some games) - there are none. You have to look ahead at the next event every time you receive a KeyRelease. Which makes an event loop that much more "interesting".
Apr 06 2022