[th/cleanup] nmci: rework handling of cleanups and add a generic list of cleanup items

We have implemented specific cleanup actions like

  context.cleanup["connections"].add(name)

but it should be possible to register ad-hoc, special cleanup actions. This branch allows that with

  context.cext.cleanup_add(lambda cext: my_cleanup())

While adding that, rework the existing cleanup actions to use the same mechanism. Now cleanup objects can be queued and there is a base class "Cleanup" for tracking these objects.

This makes the cleanup actions generic, and when adding a new kind of cleanup you can either:

• implement (and queue) a new Cleanup subclass
• queue the Cleanup base class with a callback parameter

Also, the cleanup now have logic about the order in which they are run.

• each cleanup action has a (numeric) priority. The actions with lower priority are executed first.
• in case of equal priority, the cleanup action are run in reversed order. The idea is that as you setup the system and register cleanup actions, the tear down works in reverse.
• cleanup actions have a unique-tag, so if the same cleanup action gets scheduled again, it is only added once. However, scheduling it again, will update their priority and order.
• cleanup actions can enqueue additional cleanup actions. For example, with context.cext.cleanup_add_udev_rule() it will remove a rule file, but we also need to notify udev about the changes. That latter should only be done once (to save overhead). Hence, the udev-rule cleanup does not itself reload udev, instead it also enqueues a udev-update action. With the previous point, this udev-update action is only run once and with lowest priority (high numerical value).

In general, cleanup actions should never fail and be best effort. E.g. if there is an action to delete a file, then the action should succeed if the file does not exist.

The idea is to make cleanup actions more formalized. You register a cleanup action by calling context.cext.cleanup_add*(), and don't directly mess with context.cleanup dictionary.

More importantly, we should make much more uses of cleanup actions. For example, most of the after-scenario hooks of the tags should instead be implement as cleanup actions. Instead of

  def tag1_bs(context, scenario):
     setup_thing_a("arg")
     setup_thing_b()

  def tag1_as(context, scenario):
     cleanup_thing_b()
     cleanup_thing_a("arg")

Instead should do:

  def tag1_bs(context, scenario):
     context.cext.cleanup_add(lambda cext: cleanup_thing_a("arg"))
     setup_thing_a("arg")
     context.cext.cleanup_add(lambda cext: cleanup_thing_b())
     setup_thing_b()

The idea is that:

• cleanup actions allow more control about the order in which they are performed, with the default behavior being most sensible (run in reverse order).

• if there is an exception in the middle of the setup, then the cleanup cleanup actions still run. That is important, to undo the changes to the system, so that the next test can run in a clean state.

• the cleanup moves closer to the action that performs it. For nmci.tags this is not that important, because the before_scenario() and after_scenario() were close to each other already. However, you can also do this inside a behave step. E.g. have a step that creates a profile and register it's deletion right away.