"Towards Categorical Foundations of Learning"

[dmm]

When one tries to use category theory for the applied work, a number of questions arise: Is it just too difficult to be used at all by me given my level of technical skills? Is it fruitful enough, and is the fruitfulness/efforts ratio high enough for all this to make sense?

I recently discovered Bruno Gavranović, a graduate student in Glasgow, whose work is promising in this sense. They are really trying hard to keep things simple and also trying to make sure that there are non-trivial applications. Here is one of his essays and papers (March 2021, so it's not the most recent one, but probably the most central):

www.brunogavranovic.com/posts/2021-03-03-Towards-Categorical-Foundations-Of-Neural-Networks.html

(I am posting this here because there are people who read this blog who are interested in applied category theory and like it, not because I am trying to convince those who formed a negative opinion of this subject. I am non-committal myself, I have not decided whether applied categories have strong enough fruitfulness/efforts ratio, but this particular entry seems to be one of the best shots in this sense, so I am going to try to go deeper with their work.)

Update: their collection of papers in the intersection between Category Theory and Machine Learning: github.com/bgavran/Category_Theory_Machine_Learning

Comments

[dmm]

My initial note to myself a couple of days ago:

Discovered Bruno Gavranović: https://twitter.com/bgavran3

https://twitter.com/bgavran3/status/1599185403579609088 https://arxiv.org/abs/2212.00542

https://twitter.com/bgavran3/status/1478901780994007044 https://arxiv.org/abs/2103.01931

https://scholar.google.com/citations?user=ofP7CgYAAAAJ

https://www.brunogavranovic.com/

Via https://twitter.com/bgavran3/status/1599474366148149248 replying to https://twitter.com/michael_nielsen/status/1599472810271059968

[dmm]

And my follow-up to myself today (then I decided it's probably time to make a post):



Looked at the new https://www.brunogavranovic.com/posts/2022-12-05-graph_neural_networks_as_parametric_cokleisli_morphisms.html

Yes, the whole framework is actually not difficult(!) and makes sense. The question is: is it useful?

"This paper makes a step forward in substantiating our existing framework described in Categorical Foundations of Gradient-Based Learning. If you’re not familiar with this existing work - it’s a general framework for modeling neural networks in the language of category theory. Given some base category with enough structure, it describes how to construct another category where morphisms are parametric, and bidirectional.

Even more specifically - it allows us to describe the setting where the information being sent backwards is the derivative of some chosen loss function.

This is powerful enough to encompass a variety of neural network architectures - recurrent, convolutional, autoregressive, and so on. What the framework doesn’t do is describe the structural essence of all these architectures at the level of category theory.

Our new paper does that, for one specific architecture: Graph Convolutional Neural Networks (GCNNs). We show that they arise as a morphisms for a particular choice of the base category - the CoKleisli category of the product comonad."

Yes, actually, this looks really good: https://www.brunogavranovic.com/posts/2021-03-03-Towards-Categorical-Foundations-Of-Neural-Networks.html