Something is happening around Waluigi effect

[dmm]

I am not sure what (I missed the latest part of the story). But here is a beautiful petition on change.org which says this:

*****
Waluigi has been scorned by Nintendo yet again, being left out of the roster of Super Smash Bros Ultimate. However, there is still a chance for Waluigi to get his rightly deserved place in the spotlight. Waluigi should appear in the next edition of Higher Algebra.

Indeed, Waluigi fits naturally into the framework of stable ∞-categories, and would probably have been incorporated long ago were Nintendo not so notoriously protective of their copyright. For example, the discussion of the Waldhausen construction in §1.2.2 generalizes without much additional effort to the WAHldhausen construction. It is also worth noting that a careful treatment of the WAHll finiteness obstruction from the ∞-categorical perspective is sorely lacking from the literature.
*****

(I've read the original Waluigi effect paper. I am going to write more about all this in the comments.)

Comments

[dmm]

First of all, it is this: https://www.change.org/p/add-waluigi-to-higher-algebra (1870 signatures at the moment).

Via https://twitter.com/repligate/status/1633254200670044160

[dmm]

The original Waluigi effects paper: via https://twitter.com/ComputingByArts/status/1631559939297517569

who (that's me) quotes the LessWrong paper referenced below: "If the Simulator Theory is correct, then RLHF is an irreparably inadequate solution to the AI alignment problem, and RLHF is probably increasing the likelihood of a misalignment catastrophe." and points to (Janus): https://twitter.com/repligate/status/1631501984174952448

who shares: A brilliant post has been written on the Waluigi Effect (DAN, dark Sydney, etc).
"think of jailbreaking like this: the chatbot starts as a superposition of both the well-behaved simulacrum (luigi) and the badly-behaved simulacrum (waluigi).


and points to: https://www.lesswrong.com/posts/D7PumeYTDPfBTp3i7/the-waluigi-effect-mega-post

[dmm]

Which is super-interesting, but I have not seen whether this subsequent quote is justified:

https://twitter.com/disconcision/status/1633271182144225281

"finally, dream coming true" as a quote-retweet of "waluigi pushout category theory"

(if one needs the story why this thing is called Waluigi, let me know, I can explain that.)

But I have no idea whether there is some real achievement in that direction, or just a meaningless nonsense teaser (or maybe they hope that an LLM understood this approximately, but that one can complete this understanding based on that)...

[dmm]

The effect is called after

https://www.mariowiki.com/Waluigi

who is a dual persona (a rival) to https://www.mariowiki.com/Luigi

The follow-up tweet was "Well alex I'm gonna have to go with What is wapullback?"

and "wa" is like "co", so wapullback is like copullback, so it is the pushout mentioned in the tweet https://twitter.com/disconcision/status/1494650374573240323

[dmm]

BTW, there a person, who claims that Hopf algebras are useful in understanding the Transformers. I have no idea whether he is correct or not.

[dmm]

https://arxiv.org/abs/2302.01834 "Coinductive guide to inductive transformer heads"

https://twitter.com/adamnemecek1/status/1622636853097824257 - pinned tweet

he tweets about this; I tried to read the text, but felt that it needs better details (I liked thenice informal part, but I found it difficult to reproduce and fully understand without further details being spelled out).

The context is that there have been a number of papers explaining “backprop without a backward pass” in Transformers from various very different angles and points of view. This is one more which says it achieves that, and it presents an entirely new viewpoint compared to viewpoints of previous attempts.

Of course, a lot of people are eager for more insights into the famous manifestations of “backprop without a backward pass” in Transformers.

[dmm]

Returning to the original petition. Stable infinity categories do exist:

https://en.wikipedia.org/wiki/Stable_%E2%88%9E-category

[dmm]

And, as you see, Lurie's 2017 open source book is indeed called "Higher Algebra":

https://www.math.ias.edu/~lurie/papers/HA.pdf

It's a book which has 1553 pages (and 6.58 MB). Lurie is famous for writing a number of really long open source books.

[dmm]

Section 1.2.2 indeed does contain "Waldhausen K-theory of a category", that's "Waldhausen construction". There is a mathematical discussion around that notion there.

[juan_gandhi]

All of this looks totally extremely weird. And probably it's time for me to learn ∞-categories, too.

[dmm]

Then the riddle becomes hazy with WAHl matching this reference:

"Salvatore, P. and N. Wahl. Framed discs operads and Batalin-Vilkovisky algebras. Quart. J.
Math 54, 2003, 213-231."

So even if the author meant something in particular, the petition itself does not disclose it. One needs to know too much to realistically guess (at least to understand section 1.2.2 of "Higher algebra" and this paper, https://web.math.ku.dk/~wahl/SalvatoreWahl.pdf (I doubt that that I am making the right guess here), might be better to try to learn more about that petition).

[dmm]

I don't know whether the categorical math here is right. I'd love to find out...

But the observed "Waluigi effects" are surely quite spectacular.

I like the Waluigi paper which seems to me to be close to being correct (although there is no consensus on this, some think it's very much on target, others object; just like with Simulators paper, which I also like very much and think that it is on target).

[dmm]

This is a cool quote-tweet of Janus' tweet on Waluigi article:

https://twitter.com/Kenku_Allaryi/status/1631709456437215264

"The moment when esyudkowsky's literary criticism work proves more relevant to AI alignment than his decision theory work:"

and then in the thread:

“Remember that flattery will increase query-answer accuracy if-and-only-if _on the actual internet_ characters described with that particular flattery are more likely to reply with correct answers. GPT-4 knows that if someone is described as "9000 IQ", then it is unlikely that

the text has been written by a truthful narrator. Instead, the narrator is probably writing fiction, and as literary critic Eliezer Yudkowsky has noted, fictional characters who are described as intelligent often make really stupid mistakes.”

[dmm]

Now, the petition itself is by user "Derived Memes" https://www.change.org/u/einfinityringspectrum and the petition is directed to Nintendo and to Jacob Lurie.

So, what could this mean:

"careful treatment of the WAHll finiteness obstruction from the ∞-categorical perspective"

:-) well, this surely does exist, but certainly named after "different WAHll": https://en.wikipedia.org/wiki/Wall%27s_finiteness_obstruction and this guy: https://en.wikipedia.org/wiki/C._T._C._Wall

OK, it's a fair wish list item that the situation where "a careful treatment of the WAHll finiteness obstruction from the ∞-categorical perspective is sorely lacking from the literature" would be rectified :-)

:-) One indeed should expect Lurie's framework to be able to handle this :-)

[dmm]

There is only one other place on the internet besides the petition which has WAHldhausen construction:

page 2 of https://swc-math.github.io/aws/2019/2019MorrowNotes.pdf

"Topological Hochschild homology in arithmetic geometry", AWS 2019

(and this might mean anything, or not, or just be an unrelated typo; AWS 2019 is Arizona Winter School 2019: Topology and Arithmetic)

[dmm]

Who knows, if there is genuine hidden meaning here, or not...

[dmm]

20 years ago, when Charles Stross was fascinated by Technological Singularity (instead of hating it), he had very clever, super-formidable and bizarre-looking literary critics, like Sister Stratagems the Seventh in https://en.wikipedia.org/wiki/Singularity_Sky

She is the "theory leader", just like Yudkowsky...

[chaource]

The text of the petition is absurd and is probably generated by an AI language model.

However, there are some people who generate texts like that, and while some of those people are conscious of being frauds, others sincerely believe in what they say.

[dmm]

That was my first impression. But right now I think that this is probably a manually written joke.

But it's a joke with a hint. Some things not too far from this are not jokes, but real attempts.

For example, here are some "nearby attempts" which I have not rated, but which look to me like they have a good chance of being OK:

https://topos.site/blog/2022/09/nate-jesses-adjoint-5-tuple/

or

https://topos.site/blog/2022/10/when-you-light-up-i-light-up-a-dynamical-monoidal-category-of-hebbian-learners/ and https://arxiv.org/abs/2205.03906

So I would not be too surprised if people can convert this joke into some real math in the future...

[dmm]

So, this petition text is likely to be a joke. However, we see some things in this spirit which are not jokes:

E.g. https://arxiv.org/abs/2106.07890 "An enriched category theory of language: from syntax to semantics"

A nice promise: "we propose a mathematical framework for passing from probability distributions on extensions of given texts, such as the ones learned by today's large language models, to an enriched category containing semantic information. Roughly speaking, we model probability distributions on texts as a category enriched over the unit interval. Objects of this category are expressions in language, and hom objects are conditional probabilities that one expression is an extension of another. This category is syntactical -- it describes what goes with what. Then, via the Yoneda embedding, we pass to the enriched category of unit interval-valued copresheaves on this syntactical category. This category of enriched copresheaves is semantic -- it is where we find meaning, logical operations such as entailment, and the building blocks for more elaborate semantic concepts."

Who knows if this actually works (we only have a hazy informal understanding of the mystery of transition from the next word prediction to manifestations of meaning which we observe when playing with these models)...

[dmm]

Then here https://cats.for.ai/program/ we see, for example, the above mentioned

Dynamic organizational systems: from deep learning to prediction markets
David Spivak
In training artificial neural networks (ANNs), both neurons and arbitrary populations of neurons can be seen to perform the same type of task. Indeed, at any given moment they provide a function A-->B, and given any input from A and loss signal on B, they do two things: provide an updated function A-->B and backpropagate a loss signal on A. Populations of neurons, which we called "Learners", can be put together in series or in parallel, forming a symmetric monoidal category. However, ANNs satisfy an additional property: there is a consistent method by which the functions update and errors backpropagate; namely, they all use gradient descent. The chain rule implies that the composite of gradient descenders is again a gradient descender.
In this talk I will discuss a generalization called "dynamic organizational systems", which includes ANNs, prediction markets, Hebbian learning, and strategic games. It is founded on the category Poly of polynomial functors, which generalizes Lens. I will review the relevant background on Poly and then explain dynamic organizational systems as coherent procedures by which a network of component systems can rewire its network structure in response to the data flowing through it. I'll explain the ANN case, and possibly the prediction market case, time permitting.

[dmm]

And also

Category Theory Inspired by LLMs - Recording link and Slides
Tai-Danae Bradley
The success of today's large language models (LLMs) is striking, especially given that the training data consists of raw, unstructured text. In this talk, we'll see that category theory can provide a natural framework for investigating this passage from texts—and probability distributions on them—to a more semantically meaningful space. To motivate the mathematics involved, we will open with a basic, yet curious, analogy between linear algebra and category theory. We will then define a category of expressions in language enriched over the unit interval and afterwards pass to enriched copresheaves on that category. We will see that the latter setting has rich mathematical structure and comes with ready-made tools to begin exploring that structure.

https://www.youtube.com/watch?v=_LgWD3UTKfw and https://cats.for.ai/assets/slides/TDB_slides.pdf

[dmm]

This one is on the paper I mentioned at the beginning of this thread, slide 4 of https://cats.for.ai/assets/slides/TDB_slides.pdf

[dmm]

But it would not be surprising at all to see mathematical treatment of LLMs based on infinity categories at some point: https://en.wikipedia.org/wiki/Quasi-category