Active Entries
- 1: "Narrow AGI" this year?
- 2: Tao on coordinate vs coordinate-free math reasoning
- 3: "Aging as a loss of goal-directedness"
- 4: New integrated mode for GPT-4 in ChatGPT+
- 5: Китайский новый год начнётся 10-го февраля
- 6: Automating the Search for Artificial Life with Foundation Models
- 7: "Anatomy of a Formal Proof"
- 8: C to safe Rust automatic translation using LLMs and dynamic analysis
- 9: GonzoML
- 10: Transformers as a Computational Model (workshop)
Style Credit
- Style: Neutral Good for Practicality by
Expand Cut Tags
No cut tags
no subject
Date: 2023-10-29 06:07 pm (UTC)(but, actually, positions are meaningful, so there is still a bit of privileged structure in the residual stream, just (perhaps) not within the embedding vectors (but perhaps even there, if we look closely, who knows))
~37:50 spectrum of how privileged a basis is, rather than a binary privileged vs non-privileged
(the truth is there are traces of various privileges in the residual stream as well)
~39:30 even ADAM privileges everything it interacts with, because of its weirdness ("ADAM sucks" says Neel Nanda, but I don't think it's necessarily so, perhaps this artificial thing is good, who knows(!)).