"A Mathematical Framework for Transformer Circuits"

Anthropic AI is an organization which has been created approximately a year ago by former OpenAI people who (I believe) have been unhappy about the current direction of OpenAI.

"Anthropic is an AI safety and research company that’s working to build reliable, interpretable, and steerable AI systems. Large, general systems of today can have significant benefits, but can also be unpredictable, unreliable, and opaque: our goal is to make progress on these issues."

They have just published their first major paper directed towards better understanding of Transformers. I am going to accumulate various links in the comments.