Dataflow matrix machines (by Anhinga anhinga)

WIRED published a story on Transformer invention

Thu, 21 Mar 2024 20:04:12 GMT

The history of the creation of "Attention Is All You Need", arxiv.org/abs/1706.03762

It's pretty intense; it's very interesting what it took to achieve that.

comments

6 months since GPT-4 release

Fri, 15 Sep 2023 03:32:54 GMT

A good way to mark this occasion is to try to read a new paper which seems to be a major breakthrough in understanding and harnessing the magic of Transformers:

"Uncovering mesa-optimization algorithms in Transformers"

"we demonstrate that minimizing a generic autoregressive loss gives rise to a subsidiary gradient-based optimization algorithm running inside the forward pass of a Transformer. This phenomenon has been recently termed mesa-optimization"

"Moreover, we find that the resulting mesa-optimization algorithms exhibit in-context few-shot learning capabilities,
independently of model scale. Our results therefore complement previous reports characterizing the
emergence of few-shot learning in large-scale LLMs"

comments

Let's understand Large Language Models better

Tue, 22 Aug 2023 14:57:18 GMT

This is a good starting point:

"A Mathematical Framework for Transformer Circuits", Dec 2021
transformer-circuits.pub/2021/framework/index.html

comments

Large Language Models and Transformers (a workshop this week)

Mon, 14 Aug 2023 16:04:13 GMT

simons.berkeley.edu/workshops/large-language-models-transformers

Youtube livestream (and, presumably, post-conference youtube recording) are available.

Some of the talks look really interesting.

comments

AI and East-West cultural differences

Thu, 20 Apr 2023 12:10:44 GMT

"A second difficulty in communicating alignment ideas was based on differing ontologies. A surface-level explanation is that Japan is quite techno-optimistic compared to the west, and has strong intuitions that AI will operate harmoniously with humans. A more nuanced explanation is that Buddhist- and Shinto-inspired axioms in Japanese thinking lead to the conclusion that superintelligence will be conscious and aligned by default. One senior researcher from RIKEN noted during the conference that “it is obviously impossible to control a superintelligence, but living alongside one seems possible.” Some visible consequences of this are that machine consciousness research in Japan is taken quite seriously, whereas in the West there is little discussion of it."

***

I think it's time for us to start asking if, for example, GPT-4-produced simulations have associated subjective experience.

We have a feed-forward transducer in an autoregressive mode; each time a new token is produced by the feed-forward Transformer, the whole dialog including the just produced token is fed again to the input of the model, so there is a recurrent dynamics here (cf. section 3.4 of "Transformers are RNNs: Fast Autoregressive Transformers with Linear Attention", arxiv.org/abs/2006.16236).

So I would not be too surprised if that process actually "feels what it says".

comments

Something is happening around Waluigi effect

Wed, 08 Mar 2023 02:18:07 GMT

I am not sure what (I missed the latest part of the story). But here is a beautiful petition on change.org which says this:

*****
Waluigi has been scorned by Nintendo yet again, being left out of the roster of Super Smash Bros Ultimate. However, there is still a chance for Waluigi to get his rightly deserved place in the spotlight. Waluigi should appear in the next edition of Higher Algebra.

Indeed, Waluigi fits naturally into the framework of stable ∞-categories, and would probably have been incorporated long ago were Nintendo not so notoriously protective of their copyright. For example, the discussion of the Waldhausen construction in §1.2.2 generalizes without much additional effort to the WAHldhausen construction. It is also worth noting that a careful treatment of the WAHll finiteness obstruction from the ∞-categorical perspective is sorely lacking from the literature.
*****

(I've read the original Waluigi effect paper. I am going to write more about all this in the comments.)

comments

GreaterWrong viewer for LessWrong; Conjecture.dev

Thu, 23 Feb 2023 03:44:58 GMT

I am reading more and more LessWrong in recent months (mostly, after the Simulator theory by Janus (work done while at Conjecture) has been posted there in September).

I still think the Simulator theory is probably the single most important research breakthrough of 2022.

These days LessWrong is dominated by writing related to AI safety (the topic is made particularly acute by the recent progress in LLMs: ChatGPT and even more capable Bing Chat; no consensus whatsoever, of course, but I do think that GPT-3 release in May 2020 is, in some sense, an equivalent of the nuclear fission discovery on 19 December 1938, and that ChatGPT performance (+ Bing Chat clearly drastically enhanced capabilities even compared to that) is, in the same sense, an equivalent of the first working nuclear reactor on 2 December 1942, if one goes by "AI today is what nuclear energy has been back then" analogy).

So, one thing which might be useful is that there is GreaterWrong alternative viewer (which looks different from LessWrong default viewer and which can be visually tuned in terms of presentation style; also different default front page for the site if one uses GreaterWrong). Which viewer is better might depend on your device (display, browser, etc).

Another thing, Conjecture people tend to produce some of the best, most interesting articles there.

I'll put a few links into the comments.

comments

Prompt engineering guide

Tue, 07 Feb 2023 07:41:37 GMT

A huge field already: github.com/dair-ai/Prompt-Engineering-Guide

comments

Technical interview with Neel Nanda

Sun, 05 Feb 2023 06:26:58 GMT

www.lesswrong.com/posts/r2yTwkGt3kbQG2mXi/axrp-episode-19-mechanistic-interpretability-with-neel-nanda

comments

New Year resolution

Sun, 08 Jan 2023 05:42:26 GMT

To read my https://twitter.com/home more regularly (that's absolutely the best source of info at the moment).

A small fraction of today's catch:

New work by Janus

A new involved take on AI safety/alignment

(What's the right way to organize all that information?)

Links are in the comments (I think the new work by Janus is more important even for alignment, and is just overall more important of the two topics of this post)...

comments

"MPLP: Learning a Message Passing Learning Protocol"

Mon, 26 Dec 2022 17:38:19 GMT

I have been looking at a recent rather remarkable paper which includes the DeepDream creator among its authors, and I've decided to check whether I missed any of his works; and I turns out there is this paper I really should be aware of. This really resonates with some of the thing I have been exploring this year.

arxiv.org/abs/2007.00970

"We present a novel method for learning the weights of an artificial neural network - a Message Passing Learning Protocol (MPLP). In MPLP, we abstract every operations occurring in ANNs as independent agents. Each agent is responsible for ingesting incoming multidimensional messages from other agents, updating its internal state, and generating multidimensional messages to be passed on to neighbouring agents. We demonstrate the viability of MPLP as opposed to traditional gradient-based approaches on simple feed-forward neural networks, and present a framework capable of generalizing to non-traditional neural network architectures. MPLP is meta learned using end-to-end gradient-based meta-optimisation. We further discuss the observed properties of MPLP and hypothesize its applicability on various fields of deep learning."

comments

update on AI progress and AI safety

Sun, 04 Dec 2022 16:18:56 GMT

AI-safety-wise, the write-up, My AI Safety Lecture for UT Effective Altruism by Scott Aaronson is very nice reasonably objective and theory-friendly overview of the current state of AI safety as a field of science.

AI-progress-wise, ChatGPT based on roughly speaking GPT-3.5 has been released recently, with people doing tons of interesting things with it, including meaningful writing and software generation... This seems to be another major step-up.

comments

Conferences; research updates

Thu, 17 Nov 2022 01:44:04 GMT

This week, Nov 17-18, Thu-Fri, 8am-11:45am Boston time, "Quantum physics and the first-person perspective": www.essentiafoundation.org/quantum-physics-and-the-first-person-perspective/seeing/

JuliaCon 2023, juliacon.org/2023/ the call for proposals is posted, deadline Dec 18: pretalx.com/juliacon2023/cfp

I've spent more quality time focusing of two breakthroughs in understanding the nature and the behavior of machine learning models which came from the "penumbra" of "prosaic alignment" start-ups and which I wrote about in my previous two posts.

"Grokking is (more or less) solved." I took brief notes between Oct 21 and Oct 23: github.com/anhinga/2022-notes/tree/main/Grokking-is-solved

"Generative autoregressive models are similators." I took extensive notes between Oct 5 and Oct 23: github.com/anhinga/2022-notes/tree/main/Generative-autoregressive-models-are-similators

I am continuing to develop thoughts related to these topics, I am going to gradually write more about those topics in the comments.

comments

Grokking is (more or less) solved

Sun, 16 Oct 2022 14:54:41 GMT

The most interesting conceptual AI advances seem lately to come from "prosaic alignment" start-ups. These are companies which believe that the current trend of improving Transformer models is likely to lead straight to AGI, and that better understanding of the nature and properties of these model is key to AI safety (and, of course, it's also key to better AI capabilities).

And it is often the case that the key elements of work are done by people "on the edge", "in the penumbra" of those alignment start-ups.

In the previous post I mentioned the key new understanding of large Transformer models as simulators. That work has been done "while at Conjecture", but is not listed as directly coming from Conjecture (one of those "prosaic alignment" start-ups). I think the key people involved are still at Conjecture, but they seem to be trying to keep some distance between Conjecture and this work. I am continuing to take notes of those materials and commit them to GitHub (see links in the comments to the previous post).

Here is another one of those stories. Grokking is a phenomenon, where small Transformers look at a part of a mathematical structure for quite a while, and then rather suddenly transition to understanding the whole of that mathematical structure including the part they never see in training. It has been discovered in 2021 and has been a subject of a number of follow-up attempts to understand it.

The recent breakthrough has been done in mid-August by Neel Nanda who left Anthropic (perhaps the most famous of the "prosaic alignment" start-ups) a few months ago. And it looks like he has more or less solved the mysteries behind this phenomenon. I am going to continue studying his writings more. The links are in the comments.

comments

Generative autoregressive models are similators

Wed, 21 Sep 2022 07:25:52 GMT

Вот, наконец, кажется возник правильный подход к пониманию природы моделей вроде GPT-3 и разнообразного волшебства, с этим связанного:

www.lesswrong.com/posts/vJFdjigzmcXMhNTsx/simulators

Он говорит, что надо перестать думать про эти модели в терминах более старых AI-систем.

comments

"Transformers are Sample Efficient World Models"

Wed, 07 Sep 2022 15:34:08 GMT

Another important paper from one of François Fleuret's collaborations: arxiv.org/abs/2209.00588

Previous important papers include "Transformers are RNNs: Fast Autoregressive Transformers with Linear Attention",arxiv.org/abs/2006.16236 and "Flatten the Curve: Efficiently Training Low-Curvature Neural Networks", arxiv.org/abs/2206.07144

comments

Open source code generator (an alternative to OpenAI Codex)

Tue, 09 Aug 2022 03:13:17 GMT

github.com/salesforce/CodeGen

One can also run one of these models via HuggingFace; it is based on "A Conversational Paradigm for Program Synthesis" paper, arxiv.org/abs/2203.13474

Someone has even created a fake GitHub Copilot based on that (useful for those who prefer VSCode): github.com/moyix/fauxpilot

comments

"A Mathematical Framework for Transformer Circuits"

Fri, 24 Dec 2021 16:27:12 GMT

Anthropic AI is an organization which has been created approximately a year ago by former OpenAI people who (I believe) have been unhappy about the current direction of OpenAI.

"Anthropic is an AI safety and research company that’s working to build reliable, interpretable, and steerable AI systems. Large, general systems of today can have significant benefits, but can also be unpredictable, unreliable, and opaque: our goal is to make progress on these issues."

They have just published their first major paper directed towards better understanding of Transformers. I am going to accumulate various links in the comments.

comments

"Deep Learning for AI" (the 2018 Turing Lecture)

Wed, 27 Oct 2021 19:03:18 GMT

It turns out that this is open-access: dl.acm.org/doi/10.1145/3448250

The nuances in that lecture are very interesting, shed various light in the disagreement between Hinton et al and Schmidhuber et al (this one is written from the Hinton et al side, obviously; their emphasis is that technical aspects are equally important and not subservient to "pioneering theory"; e.g. a lot of rather recent pre-2012 developments such as the practical understanding of the role of ReLU is what made the AlexNet breakthrough possible, and moreover things like "the very efficient use of multiple GPUs by Alex Krizhevsky" are also key, not just the neural architecture ideas).

There is a whole section on Transformers, I am going to include it in the comments verbatim.

The journal publication is July 2021, and there are references in the paper which are newer than 2018; I don't know how heavily the text itself has been edited since 2018.

comments

Recent presentations

Sun, 24 Oct 2021 04:43:22 GMT

- JuliaCon 2021 (July 30)

- ML Collective Research Jam #3 (Aug 4)

- ML Collective Research Jam #4 (Sep 22)

- Stuttgart Julia Programming Language Meetup (Oct 23)

comments

Compact Transformers

Fri, 20 Aug 2021 12:59:24 GMT

For those of us (like myself) who'd like to experiment with changing Transformer architecture on a home personal computer.

Links are in the comments.

comments

OpenAI Codex (next generation) - it looks like we are finally "there"

Thu, 12 Aug 2021 15:35:14 GMT

There was a live demo of the next generation of OpenAI Codex code-generating software on August 10.

My impression from it is that "we have finally arrived" - this is a programming tool which seems to be more useful than an extra entry-level software engineer on the team. This starts to address the key bottleneck of our times: our limited ability to create software.

This was always my threshold: can we create an AI software which can be "hired instead of a junior software engineer"? That was the main temporal uncertainty for me: how long would it take to reach that level? It looks like this has been accomplished.

We are rapidly approaching the situation when AI will actively participate in programming AI software, for better or for worse...

OpenAI Codex is now a part of OpenAI API (which is still a closed beta with a waitlist), and it will be possible to participate in an informal competition today from 10am Pacific time (1pm Eastern) till 1pm Pacific (4pm Eastern) and try it a bit.

Links are in the comments.

comments

GitHub Copilot ("we are getting there")

Tue, 29 Jun 2021 16:23:30 GMT

github.blog/2021-06-29-introducing-github-copilot-ai-pair-programmer/

"Today, we are launching a technical preview of GitHub Copilot, a new AI pair programmer that helps you write better code. GitHub Copilot draws context from the code you’re working on, suggesting whole lines or entire functions. It helps you quickly discover alternative ways to solve problems, write tests, and explore new APIs without having to tediously tailor a search for answers on the internet. As you type, it adapts to the way you write code—to help you complete your work faster.

Developed in collaboration with OpenAI, GitHub Copilot is powered by OpenAI Codex, a new AI system created by OpenAI. OpenAI Codex has broad knowledge of how people use code and is significantly more capable than GPT-3 in code generation, in part, because it was trained on a data set that includes a much larger concentration of public source code. GitHub Copilot works with a broad set of frameworks and languages, but this technical preview works especially well for Python, JavaScript, TypeScript, Ruby and Go."

If you are using Visual Studio Code often, it might make sense to try to sign-up for the technical preview phase...

comments