![[personal profile]](https://www.dreamwidth.org/img/silk/identity/user.png){kind=small}
dmmIt seems that people are gradually getting dissatisfied with the state of machine learning frameworks (and, in particular, with how huge and unwieldy they became, while not delivering enough). The most interesting new trend comes from Julia community, in particular, in connection with their new Flux framework. I highly recommend this blog post in this sense:
julialang.org/blog/2018/12/ml-language-compiler/
Perhaps, we can finally start getting away from Python dominance in machine learning. Transition to Julia does sound attractive, it's a very nice, performant language.
***
But even inside Google Brain, it seems that people are getting fed up with TensorFlow. Here is what their new "Reformer", ai.googleblog.com/2020/01/reformer-efficient-transformer.html, (a super-efficient incarnation of the famous class of attention-based Transformer linguistic models) is using instead of TensorFlow:
This framework: github.com/google/trax
Based on this experimental thing: github.com/google/jax ("Composable transformations of Python+NumPy programs: differentiate, vectorize, JIT to GPU/TPU, and more"; note that this is still using the TensorFlow XLA (Accelerated Linear Algebra) domain-specific optimizing compiler for linear algebra)
julialang.org/blog/2018/12/ml-language-compiler/
Perhaps, we can finally start getting away from Python dominance in machine learning. Transition to Julia does sound attractive, it's a very nice, performant language.
***
But even inside Google Brain, it seems that people are getting fed up with TensorFlow. Here is what their new "Reformer", ai.googleblog.com/2020/01/reformer-efficient-transformer.html, (a super-efficient incarnation of the famous class of attention-based Transformer linguistic models) is using instead of TensorFlow:
This framework: github.com/google/trax
Based on this experimental thing: github.com/google/jax ("Composable transformations of Python+NumPy programs: differentiate, vectorize, JIT to GPU/TPU, and more"; note that this is still using the TensorFlow XLA (Accelerated Linear Algebra) domain-specific optimizing compiler for linear algebra)
no subject
Date: 2020-02-08 05:32 pm (UTC)First of all, it is based on the new source-to-source system of automatic differentiation which claims to be the best in the world at the moment (it is certainly better than anything I've seen so far):
https://github.com/FluxML/Zygote.jl
https://arxiv.org/abs/1810.07951
(See, in particular, Section 5, "Related Work", of the latest version of the paper (March 2019).)
They say that the approach can support the full flexibility and dynamism of the Julia language, including control flow, nesting, mutation, recursion, closures, data structures (structs, dictionaries, etc.), higher-order functions, and other language constructs, and that the output is given to an existing compiler to produce highly efficient differentiated code.
The Flux itself is presented as "the ML library that doesn't make you tensor".
https://github.com/FluxML/Flux.jl
So, this seems to be a perfect fit for dataflow matrix machines, where we can truly benefit from not having to contort things into fixed-shape tensors and from the ability to take gradients of arbitrary source code.
This is a great framework for many other machine learning applications as well.
Their front page is very cool:
https://fluxml.ai/