I thought about this for a bit. I think largely to do things with messages you don't know about is probably a bad idea in general; writing code that works this way is bound to create a lot of trouble in the future, and it's hard to always reason about from every PoV. However, there are some use cases where dealing with types not known at compile-time is useful, obviously debugging tools. In that case I think the right thing to do is just have a way to look up schemas based on some sort of identity. Cap'n'proto is not necessarily the greatest here: It relies on a randomly-generated 64-bit file identifier. I would prefer a URL or perhaps a UUID instead. Either way, carrying a tiny bit of identity information means that the relatively-niche users who need to introspect an unknown message don't cause everyone else to need to pay up-front for describability, and those users that do need introspection can get the entire schema rather than just whatever is described in the serialized form.
It's better to design APIs to be extensible in ways that doesn't require dynamic introspection. It's always possible to have a "generic" header message that contains a more specific message inside of it, so that some consumers of an API can operate on messages even when they contain some data that they don't understand, but I think this still warrants some care to make sure it's definitely the right API design. Maybe in the future you'll come to the conclusion it would actually be better if consumers don't even try to process things they're not aware of as the semantics they implement may some day be wrong for a new type of message.
> I think largely to do things with messages you don't know about is probably a bad idea in general
Versioning, at the least, is extremely difficult without this.
Look at the Vulkan API for an example of what they have to do in C to manage this. They have both an sType tag and a pNext extension pointer in order for past APIs to be able to consume future versions.
It's better to design APIs to be extensible in ways that doesn't require dynamic introspection. It's always possible to have a "generic" header message that contains a more specific message inside of it, so that some consumers of an API can operate on messages even when they contain some data that they don't understand, but I think this still warrants some care to make sure it's definitely the right API design. Maybe in the future you'll come to the conclusion it would actually be better if consumers don't even try to process things they're not aware of as the semantics they implement may some day be wrong for a new type of message.