Empty array encoding being problematic for implementers
For those who don't know me, I'm the main author of zbus, a Rust library for D-Bus and currently also writing a broker impl on top of that.
In the Rust world, serde is the de facto standard for any encoding/decoding API so zbus provides that. However, the way serde works, I do not get any introspection information about the child type of an array if it has no elements in it. While this is likely a non-problem for other formats, it was a big issue for zbus (actually zvariant, the underlying sub-project responsible for the encoding/decoding) since D-Bus requires padding for the first element in an array even when there is no first element. Since the padding is dependent on the element type and I don't have this information at hand, this seemed like a show stopper at first.
I ended up working around this by requiring developers to implement a trait, Type
, that gives me the signature of the type in question and having it available to the encoder/decoder machinery, which parses the signature while encoding and decoding. While this worked, it meant that I had to provide implementation of Type
trait for a lot of common types in the ecosystem and generic implementations for container types in the standard library (this list keeps growing, albeit slowly). Another issue is that having to parse signatures adds a significant enough performance penalty. To make things as easy as possible for developers, we also provide a derive macro that makes adding implementation of Type
for custom types super easy. However, this macro has serious limitations and has no way to interact with the usual serde macros that Rust developers are used to.
I had been thinking about creating an issue for this on gitlab.fdo for years now but given that I did not see any possible non-breaking solution for this, I had been reluctant and recently just forgot all about it. While discussing about encoding of optional/nullable types as array yesterday with Simon (#25), I got reminded of this thorn again and thought maybe someone has some idea about solving this somehow in a backwards compatible way. Perhaps a peer can ask for this capability during handshake or through some new API and the bus can then expect no padding for elements in an empty array?
If we could find a solution, this would help me immensely and let me drop a huge amount of code and improve performance as well. So I thought it's worth asking for.