WIRED published a story on Transformer invention

[dmm]

The history of the creation of "Attention Is All You Need", arxiv.org/abs/1706.03762

It's pretty intense; it's very interesting what it took to achieve that.

Comments

[dmm]

WIRED allowed me to read it for free (but I think they try to limit how much one can read for free; I don't know if anyone has already created a completely unrestricted copy).

https://www.wired.com/story/eight-google-employees-invented-modern-ai-transformers-paper/

[juan_gandhi]

Feeling lost in the brave new math. On one hand, things are so pathetically primitive. On the other hand, fck, they achieve so much! Looks more like medicine, compared to e.g. molecular biology.

Oh, and thank you for providing this information. The world is so different compared to the pre-covid epoch.

[dmm]

I think we'll do more sophisticated things again soon.

But I was stunned with their description of the last few weeks of working on that paper; that was intense...

[chaource]

Пусть Jakob Uszkoreit тамъ все ускоряетъ!

[timelets]

Thanks!

[chaource]

https://web.archive.org/web/20240320101528/https://www.wired.com/story/eight-google-employees-invented-modern-ai-transformers-paper/