"Differentiable Finite State Machines"

[dmm]

Alex Mordvintsev (known for DeepDream and more recently for beautiful Neural Cellular Automata and Self-Organizing Textures) created this cool tutorial:

google-research.github.io/self-organising-systems/2022/diff-fsm/

"how differentiable optimization can be used to learn Finite State Machines (FSM) for solving toy string processing tasks"

"how simple regularization and initialization techniques can steer continuous optimization towards finding discrete deterministic solutions"

"experiments shown here may have some educational value, e.g. in demonstrating less conventional (and perhaps unexpected) uses of differentiable programming and some elegant JAX tricks."

He introduces two techniques to "sparsify" the system (to reduce the size of the state machine): penalty for entropy and addition of identity transform.

:-) I quote-retweeted a summary of this, and Alex retweeted my tweet (I felt honored by that), and that tweet of mine went "semi-viral" as a result :-)

Comments

[dmm]

https://twitter.com/ComputingByArts/status/1534220252355100673

:-) I am sure all those likes, retweets, and follows resulting from this are there mainly because Alex has retweeted this :-)

:-) I guess I have written a good "tweet summary" there :-)

[dmm]

Meanwhile I was to advertise his new remarkable animation inspired by horseshoe crabs (I think this is his latest exploration via his novel family of Neural Cellular Automata techniques, but he does not say so explicitly):

https://twitter.com/zzznah/status/1532395703527415809

[juan_gandhi]

Oh how interesting!