[Infrastructures] state machines

[Daniel Hagerty]

 > If practicalities disagree with the theory, then something's wrong
 > with the theory.  In this case, the theory is that we can know every

    Investigating the converse is also useful -- if you can't
practically do what the theory says you can, what are you doing that
creates the practical obstacles?  "doctor, it hurts when I do this..."

 > When it comes down to it, your thesis relies on the answers to:
 > (a) do you know all the levels at which any possible action can
 > modify the system?
 > (b) can you reliably record *and later restore* the state at *all* of
 > those levels?  (keeping in mind that this may require actions to be
 > performed in a particular order, so simply thwacking the eeprom after
 > doing your disk restore might not completely restore the state if the
 > eeprom controls something that can affect the restore....)

    There are more questions.  Note that one of the problems a user
had was deleting a CNAME and discovering that his journal produced
different results depending on the existence of the CNAME.

 > You can do this with full machine virtualization, and perhaps someday
 > that will be a best practice.  Otherwise, unless you've carefully
 > inspected and dissected *everything* that touches your system, it's
 > not clear to me that you can say yes to both of the above questions.

    In the particular cadence example, you don't even need full
machine virtualization to see what's being done -- the program's only
means of interacting with the outside world is through the syscall
interface, which can be instrumented.  Concerned that it's touching
the eeprom?  You can prove that it isn't directly doing so by showing
that it never uses the interface to do so.  If you're concerned that
it's doing it through an intermediary, the communication with the
intermediary will show up, and you should perform the same operation
on the suspect intermediary.