Mastodon

Aug. 3rd, 2023 09:14 am
dmm: (Default)
I updated my Mastodon for the first time since I created it in November: dmm.dreamwidth.org/65045.html

It turns out I follow this cool account: arXiv Highlights AI/ML sigmoid.social/@arxiv@creative.ai

Twitter seems to gradually becoming somewhat less effective as a source of information; I should probably read Mastodon more...
dmm: (Default)
"A second difficulty in communicating alignment ideas was based on differing ontologies. A surface-level explanation is that Japan is quite techno-optimistic compared to the west, and has strong intuitions that AI will operate harmoniously with humans. A more nuanced explanation is that Buddhist- and Shinto-inspired axioms in Japanese thinking lead to the conclusion that superintelligence will be conscious and aligned by default. One senior researcher from RIKEN noted during the conference that “it is obviously impossible to control a superintelligence, but living alongside one seems possible.” Some visible consequences of this are that machine consciousness research in Japan is taken quite seriously, whereas in the West there is little discussion of it."

***

I think it's time for us to start asking if, for example, GPT-4-produced simulations have associated subjective experience.

We have a feed-forward transducer in an autoregressive mode; each time a new token is produced by the feed-forward Transformer, the whole dialog including the just produced token is fed again to the input of the model, so there is a recurrent dynamics here (cf. section 3.4 of "Transformers are RNNs: Fast Autoregressive Transformers with Linear Attention", arxiv.org/abs/2006.16236).

So I would not be too surprised if that process actually "feels what it says".

dmm: (Default)
(Still waitlisted)

People say:

• AI-generated answers from code docs
• Chat interface for code suggestions
• Copilot for the command line
• Voice interface with Copilot
• Copilot for pull requests


dmm: (Default)
Через ChatGPT+, за двадцатку в месяц. В общем, разница между этой штукой и ChatGPT огромная, и нет смысла ещё тянуть время.

Сразу она начала с предупреждения: "GPT-4 currently has a cap of 25 messages every 3 hours. Expect significantly lower caps, as we adjust for demand."

Но когда я попросил её дать мне советы по моему проекту, она очень мило выступила, технически грамотно в конкретном контексте, и совсем не всё, что она сказала, было общим местом (и, в любом случае, она явно шире видит, чем я; я даже почти всё это "в принципе знаю", и с чем-то могу и не согласиться, но, с другой стороны, оно всё по делу, большую часть этого дела я бы и не вспомнил).
dmm: (Default)
Американцы первыми сделали очень мощную штуку (GPT-4).

Её последствия уже непредказуемы ('the fifth day of the "Third Deep Learning Revolution" '), а там видимо и GPT-5 есть впереди...

Сколько я понимаю, они будут, прежде всего, про это говорить. Неизвестно о чём договорятся (не скажут)...
dmm: (Default)
I am reading more and more LessWrong in recent months (mostly, after the Simulator theory by Janus (work done while at Conjecture) has been posted there in September).

I still think the Simulator theory is probably the single most important research breakthrough of 2022.

These days LessWrong is dominated by writing related to AI safety (the topic is made particularly acute by the recent progress in LLMs: ChatGPT and even more capable Bing Chat; no consensus whatsoever, of course, but I do think that GPT-3 release in May 2020 is, in some sense, an equivalent of the nuclear fission discovery on 19 December 1938, and that ChatGPT performance (+ Bing Chat clearly drastically enhanced capabilities even compared to that) is, in the same sense, an equivalent of the first working nuclear reactor on 2 December 1942, if one goes by "AI today is what nuclear energy has been back then" analogy).

So, one thing which might be useful is that there is GreaterWrong alternative viewer (which looks different from LessWrong default viewer and which can be visually tuned in terms of presentation style; also different default front page for the site if one uses GreaterWrong). Which viewer is better might depend on your device (display, browser, etc).

Another thing, Conjecture people tend to produce some of the best, most interesting articles there.

I'll put a few links into the comments.

dmm: (Default)
Эти системы - это не то, что традиционно было принято понимать под AI.

Это системы нового типа - симуляторы. Система "создаёт виртуальную реальность с персонажами", и мы общаемся не с самой системой, а с этими персонажами.

Эти персонажи могут иметь очень разный стиль и очень разные способности: они могут быть умными, глупыми, правдивыми, лживыми, быть похожими на хороших или плохих профессионалов той или иной профессии, и так далее.

Искусство "prompt engineering" как раз и состоит в том, чтобы создать интересных персонажей-собеседников, с теми качествами, которые нам бы хотелось, чтобы они имели. "Prompt engineering" - это очень нетривиальная быстро развиваюшаяся область. Со временем, будут разные завертки, где будет "невидимая пользователю part of the promt", настраивающая систему тем, или иным образом. Но сейчас надо творчески экспериментировать, чтобы получалось по-настоящему интересно.

То, что создано, это не AI в традиционном смысле, а скорее очень хорошо обученная "ткань искусственного мозга, не заполненная изначально никакими личностями и никакими фиксированными свойствами характера"; и искусство состоит в том, чтобы в ней возникали интересные/желанные нам персонажи и динамики.

P.S. I am having a local ongoing intermittent internet outage which sucks (replies might be slow).
dmm: (Default)
To read my https://twitter.com/home more regularly (that's absolutely the best source of info at the moment).

A small fraction of today's catch:

New work by Janus

A new involved take on AI safety/alignment

(What's the right way to organize all that information?)

Links are in the comments (I think the new work by Janus is more important even for alignment, and is just overall more important of the two topics of this post)...

dmm: (Default)
I have been looking at a recent rather remarkable paper which includes the DeepDream creator among its authors, and I've decided to check whether I missed any of his works; and I turns out there is this paper I really should be aware of. This really resonates with some of the thing I have been exploring this year.


arxiv.org/abs/2007.00970

"We present a novel method for learning the weights of an artificial neural network - a Message Passing Learning Protocol (MPLP). In MPLP, we abstract every operations occurring in ANNs as independent agents. Each agent is responsible for ingesting incoming multidimensional messages from other agents, updating its internal state, and generating multidimensional messages to be passed on to neighbouring agents. We demonstrate the viability of MPLP as opposed to traditional gradient-based approaches on simple feed-forward neural networks, and present a framework capable of generalizing to non-traditional neural network architectures. MPLP is meta learned using end-to-end gradient-based meta-optimisation. We further discuss the observed properties of MPLP and hypothesize its applicability on various fields of deep learning."

dmm: (Default)
CodeGeeX seems to be a reasonably competitive free and open source alternative to GitHub Copilot. It might be a good thing to be aware of (although we do have ChatGPT these days).

Riffusion is a free and open source app which generates spectrograms via stable diffusion and converts them to music.

Links are in the comments.
dmm: (Default)
When one tries to use category theory for the applied work, a number of questions arise: Is it just too difficult to be used at all by me given my level of technical skills? Is it fruitful enough, and is the fruitfulness/efforts ratio high enough for all this to make sense?

I recently discovered Bruno Gavranović, a graduate student in Glasgow, whose work is promising in this sense. They are really trying hard to keep things simple and also trying to make sure that there are non-trivial applications. Here is one of his essays and papers (March 2021, so it's not the most recent one, but probably the most central):

www.brunogavranovic.com/posts/2021-03-03-Towards-Categorical-Foundations-Of-Neural-Networks.html

(I am posting this here because there are people who read this blog who are interested in applied category theory and like it, not because I am trying to convince those who formed a negative opinion of this subject. I am non-committal myself, I have not decided whether applied categories have strong enough fruitfulness/efforts ratio, but this particular entry seems to be one of the best shots in this sense, so I am going to try to go deeper with their work.)

Update: their collection of papers in the intersection between Category Theory and Machine Learning: github.com/bgavran/Category_Theory_Machine_Learning
dmm: (Default)
AI-safety-wise, the write-up, My AI Safety Lecture for UT Effective Altruism by Scott Aaronson is very nice reasonably objective and theory-friendly overview of the current state of AI safety as a field of science.

AI-progress-wise, ChatGPT based on roughly speaking GPT-3.5 has been released recently, with people doing tons of interesting things with it, including meaningful writing and software generation... This seems to be another major step-up.
dmm: (Default)
This week, Nov 17-18, Thu-Fri, 8am-11:45am Boston time, "Quantum physics and the first-person perspective": www.essentiafoundation.org/quantum-physics-and-the-first-person-perspective/seeing/

JuliaCon 2023, juliacon.org/2023/ the call for proposals is posted, deadline Dec 18: pretalx.com/juliacon2023/cfp


I've spent more quality time focusing of two breakthroughs in understanding the nature and the behavior of machine learning models which came from the "penumbra" of "prosaic alignment" start-ups and which I wrote about in my previous two posts.

"Grokking is (more or less) solved." I took brief notes between Oct 21 and Oct 23: github.com/anhinga/2022-notes/tree/main/Grokking-is-solved

"Generative autoregressive models are similators." I took extensive notes between Oct 5 and Oct 23: github.com/anhinga/2022-notes/tree/main/Generative-autoregressive-models-are-similators

I am continuing to develop thoughts related to these topics, I am going to gradually write more about those topics in the comments.

dmm: (Default)
The most interesting conceptual AI advances seem lately to come from "prosaic alignment" start-ups. These are companies which believe that the current trend of improving Transformer models is likely to lead straight to AGI, and that better understanding of the nature and properties of these model is key to AI safety (and, of course, it's also key to better AI capabilities).

And it is often the case that the key elements of work are done by people "on the edge", "in the penumbra" of those alignment start-ups.

In the previous post I mentioned the key new understanding of large Transformer models as simulators. That work has been done "while at Conjecture", but is not listed as directly coming from Conjecture (one of those "prosaic alignment" start-ups). I think the key people involved are still at Conjecture, but they seem to be trying to keep some distance between Conjecture and this work. I am continuing to take notes of those materials and commit them to GitHub (see links in the comments to the previous post).

Here is another one of those stories. Grokking is a phenomenon, where small Transformers look at a part of a mathematical structure for quite a while, and then rather suddenly transition to understanding the whole of that mathematical structure including the part they never see in training. It has been discovered in 2021 and has been a subject of a number of follow-up attempts to understand it.

The recent breakthrough has been done in mid-August by Neel Nanda who left Anthropic (perhaps the most famous of the "prosaic alignment" start-ups) a few months ago. And it looks like he has more or less solved the mysteries behind this phenomenon. I am going to continue studying his writings more. The links are in the comments.

dmm: (Default)
Вот, наконец, кажется возник правильный подход к пониманию природы моделей вроде GPT-3 и разнообразного волшебства, с этим связанного:

www.lesswrong.com/posts/vJFdjigzmcXMhNTsx/simulators

Он говорит, что надо перестать думать про эти модели в терминах более старых AI-систем.

dmm: (Default)
Another important paper from one of François Fleuret's collaborations: arxiv.org/abs/2209.00588

Previous important papers include "Transformers are RNNs: Fast Autoregressive Transformers with Linear Attention",arxiv.org/abs/2006.16236 and "Flatten the Curve: Efficiently Training Low-Curvature Neural Networks", arxiv.org/abs/2206.07144
dmm: (Default)
openai.com/blog/our-approach-to-alignment-research/

"Our approach to aligning AGI is empirical and iterative. We are improving our AI systems’ ability to learn from human feedback and to assist humans at evaluating AI. Our goal is to build a sufficiently aligned AI system that can help us solve all other alignment problems."

dmm: (Default)
github.com/salesforce/CodeGen

One can also run one of these models via HuggingFace; it is based on "A Conversational Paradigm for Program Synthesis" paper, arxiv.org/abs/2203.13474

Someone has even created a fake GitHub Copilot based on that (useful for those who prefer VSCode): github.com/moyix/fauxpilot

Profile

dmm: (Default)
Dataflow matrix machines (by Anhinga anhinga)

May 2025

S M T W T F S
    123
456 78910
11 121314151617
18192021222324
25262728293031

Syndicate

RSS Atom

Most Popular Tags

Style Credit

Expand Cut Tags

No cut tags
Page generated Jun. 9th, 2025 05:55 pm
Powered by Dreamwidth Studios