Dataflow matrix machines (by Anhinga anhinga)

Yes, one needs a model which can specialize based on actual interactions.

Of course, non-verbal stage is particularly tricky... But it all should be doable eventually.