On the essay on AI-generating algorithms

[dmm]

I am going to discuss the essay arxiv.org/abs/1905.10985 "AI-GAs: AI-generating algorithms, an alternate paradigm for producing general artificial intelligence" by Jeff Clune.

It is a rather involved text, although it does not have any formulas (22 pages + 12 pages of literature).

Jeff Clune got his PhD in 2010, founded Evolving AI Lab at the University of Wyoming and co-founded a start-up named Geometric Intelligence, which eventually became Uber AI Lab.

Together with Ken Stanley and other members of Uber AI Lab, he jump-started "deep neuroevolution" in the end of 2017: eng.uber.com/deep-neuroevolution/  (See also January 2019 review paper "Designing neural networks through neuroevolution" in Nature Machine Intelligence: www.nature.com/articles/s42256-018-0006-z ).

In January 2020, he joined OpenAI to lead a large-scale effort into research of AI-generating algorithms.

***

I am just going to write various quotes from that paper and discuss parts of this essay in the comments to this post now and in the coming days.

March 26 update: I wrote a follow-up essay, "Synergy between AI-generating algorithms and dataflow matrix machines", covering possible interplay between AI-GAs and DMMs: github.com/anhinga/2020-notes/tree/master/research-notes

Comments

[dmm]

Yes, I think the "third pillar" is one of his key innovations.

Thanks for mentioning the "Niche construction" - he mentions niches quite a bit in pages 11-16, but never references "Niche construction". I am going to include the Widipedia reference here for completeness:

https://en.wikipedia.org/wiki/Niche_construction

[timelets]

On the practical side, it would be amazing to build an AI system that develops customized meaningful curriculum for actual human intelligent agents. Moreover, if we could build a meaningful cirrocumuli for autistic children, based on some kind of AI approach, that would be a real breakthrough. Can we model the difference between worlds of normal and autistic children? Especially, in early stages when they can't speak yet.

[dmm]

Yes, one needs a model which can specialize based on actual interactions.

Of course, non-verbal stage is particularly tricky... But it all should be doable eventually.

[dmm]

I've created a "map" of this paper in the subsequent comments (in reality, one needs to overview Section 2, but I think I will not do that - Sections 2.1 and 2.2 are sufficiently compact and can be read in their entirety, and Section 2.3 is the very important core of the paper, but I am probably going to work with the first two pillars in the near future, so I'll postpone trying to master the third pillar till a later point.)