Although this is a work in progress/not ready for merge, I think some of the ideas are interesting, for example allocating memory for handles, using libloading to load the plugin, and the iterator interface to enumerating factories. Please look through it and let me know what you think. Also please ask questions if any part of the code is unclear or confusing!
The part I'm not sure of yet is wrapping the untyped handle in something strongly typed. This will always be unsafe (if a plugin provides certain interfaces but deviates from the expected method signatures) but I think this just has to be accepted. A requirement of a good wrapper is that it should be possible to wrap an interface so that the user doesn't need to write unsafe code.
Anyway I'm keen for this WIP PR to start a conversation.