Sparsity in Neural Networks (first workshop with this title)

[dmm]

I am very interested in sparsity in neural nets and I am super-happy that the community around sparsity is growing, and the first workshop on the subject had just taken place.

The key organizer is D.C.Mocanu; I did an experimental PyTorch project building upon his work a couple of years ago.

I learned about this via the ML Collective reading group mailing list. The links are in the comments.

Comments

[dmm]

The open tutorial is based on a rather huge survey (given by its first author):

"Sparsity in Deep Learning: Pruning and growth for efficient inference and training in neural networks", https://arxiv.org/abs/2102.00554

So, yes, from an exotic field this topic became something big (although the idea of this standalone workshop still came from a rejection of a workshop proposal by one of the major conferences)

[dmm]

I learned about the new OpenAI Codex paper, https://dmm.dreamwidth.org/44860.html?thread=163132#cmt163132, from the beginning of this tutorial, and I had to switch to studying that paper.

Now this study is done, so can return to focusing on this workshop.