Dataflow matrix machines (by Anhinga anhinga)

Both 70B and 8B versions are very impressive on initial blind comparisons: chat.lmsys.org/?leaderboard

If Llama 3-70B-Instruct turns out to be indeed more or less equivalent to early GPT-4, this would have a lot of wide-ranging implications.

One can use Llama 3-70B-Instruct at www.meta.ai/ for free.

The history of the creation of "Attention Is All You Need", arxiv.org/abs/1706.03762

It's pretty intense; it's very interesting what it took to achieve that.

1 year since GPT-4 launch. Everything is progressing rapidly, the last few weeks have been intense.

GPT-4 now has some real competition from Claude 3 Opus and from Gemini models.

It also has some problems: in its default multimodal configuration, the system prompt is too long, and that can interfere with its thinking.

There is a URL for a non-multimodal "Classic" version, which might be better in this sense: chat.openai.com/g/g-YyyyMT9XH-chatgpt-classic

Unanimous judgement (stating that this is a federal issue which cannot be decided by states), but 4 justices had different opinions objecting to the decision overreach (starting from page 14).

A 20 page PDF: www.supremecourt.gov/opinions/23pdf/23-719_19m2.pdf

An overview of BlueSky and atproto by Steve Klabnik:

steveklabnik.com/writing/how-does-bluesky-work

This does look very attractive...

Some visual illusions are very strong, this one might be the strongest I've seen yet:

twitter.com/ComputingByArts/status/1743675596268601546

One can place the cursor at various parts of the image to verify that the wire frame is not moving relative to the screen.

I don't know if there is a detailed neuroscience understanding of this one.

May 11, 2025 update: Another version of that, presumably: x.com/algekalipso/status/1921416201642930357

Lean theorem prover seems to be the most convenient and practical (good enough for mathematicians to use it in real work, I am seeing plenty of such stories lately; there is a bit of controversy on whether to use Lean 3 or Lean 4).

The best quote according to "leanprover" twitter:

>Our favorite quote from the paper: "Monadic syntax is excellent for expressing stochastic algorithms, and working over finitely supported distributions avoids the need for integrability side conditions during proofs."

Links are in the comments.

The main difference between classical Transformers and the new generation (which includes GPT-4) is that the new generation seems to be "mixtures-of-experts", with each of their feedforward layers subdivided into "experts" and only some of its "experts" activated on each inference run.

I think this is key to both GPT-4 and Mixtral 8x7B (which I suspect is approximately an open-source mini-GPT-4 and which is the new leading open-source model roughly equivalent to GPT-3.5 in performance).

Of course, GPT-4 might have some extra magic secret sauce besides being (according to rumors) a "mixture-of-experts" and scale (given how difficult it has been to even reproduce its performance so far).

Hugging Face published a very nice tutorial recently: huggingface.co/blog/moe

"Are there evolutionary models which try to explain why the tendency to engage in self-deception and other numerous cognitive biases are not pruned away by evolutionary selection?"