Citations; software progress

[dmm]

I have always been feeling somewhat awkward that my most cited paper has been an industrial pre-machine-learning paper on how to extract geographic references from a text.

Recently, our 2009 paper in American Mathematical Monthly on non-zero self-distances has finally surpassed it in the number of citations.

Meanwhile, my two papers I like the most have exactly zero citations.

5 months in a new job: among software achievements: now I know how to take autocomputed gradients with respect to variables assembled inside nested dictionaries. So I am no longer forced to reshape complicated tree-like-structures into flat arrays in order to use differentiable programming.

As a result, I can finally experiment with DMM training using gradient methods without putting too much labor into those experiments.

🇺🇦 🇺🇦 🇺🇦 Links are in the comments 🇺🇦 🇺🇦 🇺🇦

Comments

[dmm]

Gradients with respect to nested dictionaries:

JAX (Python):

https://github.com/anhinga/jax-pytree-example

Julia Flux (Zygote.jl):

https://github.com/anhinga/julia-flux-drafts/tree/main/arxiv-1606-09470-section3

This also can be done in Enzyme.jl (also Julia) to some extent, but Enzyme is still too fragile and restrictive.

No other differentiable programming framework can done this kind of thing, to the best of my knowledge.

[dmm]

The experiments with using gradients methods for DMM training are not open-source yet (alas, but open-sourcing them will have to wait).

[dmm]

GitHub shows "600 contributions in the last year" for the first time in my life. Unfortunately, almost a half of them are in private repositories.

I am getting very nice generalization properties and nice "intermediate sparsification" results in my latest series of experiments.

[dmm]

Nice "full sparsification" which generalizes well...

Ok, therefore June 27, 2022 is the day when the first DMM program synthesis primer has been successfully created.

[dmm]

661 contributions; left the job yesterday, disconnected from their org: minus 476 contributions = 185.

But the key experimental results are now open sourced.

[dmm]

400 contributions on GitHub in the last year (the last 5 weeks have been fruitful).

Pondering what to do with respect to trying to actually get some collaborative efforts on all this...

[dmm]

Reached 500 of them last week, before 52 of them expired; currently at 463.

Still pondering what to do with respect to trying to actually get some collaborative efforts on all this...

[dmm]

Had a great dialog which prompted me to start doing a straight port of 2017 Clojure version to Julia:

https://github.com/anhinga/late-2022-julia-drafts/tree/main/dmm-port-from-clojure

500 contributions on GitHub in the last year again.

[dmm]

1) Finally has implemented a "superfluid" architecture (unearthed and developed our 2018 design) with flexible "activation function" inside a neuron; so, finally, we have achieved a "fully type-free" version (the Clojure one has been "almost type-free")

2) The standard first self-referential experiment (network inducing a wave pattern in its own connectivity matrix) just started to work, so we can now say that a port from Clojure to Julia does exist

3) 554 contributions on GitHub in the last year at the moment (9 of them will expire soon)

[dmm]

600 contributions on GitHub in the last year at the moment.

Just finished an outline for a possible book on "Machine Learning in Julia" (I was approached and asked to explore this as a possible book project; keeping this one in a private repository for the time being).

[dmm]

Just reached 662 contributions on GitHub; finally got above Sep 17.

Not doing the book (I decided it does not make sense to undertake such a project now, and especially on the publisher's terms).

[dmm]

"666 contributions in the last year"

[dmm]

"700 contributions in the last year"

[dmm]

"800 contributions in the last year"

[dmm]

"888 contributions in the last year"

We have a new preprint, "Safety without alignment": https://arxiv.org/abs/2303.00752

[dmm]

"900 contributions in the last year"

Полгода назад кончилась та служебная деятельность; сейчас бы она мне очень мешала...

На самом деле, это была пятница, 16-е сентября, а в понедельник, 19-го сентября Скотт Александр опубковал заметку про деятельность Януса, и, как раз, оказалось уместно полностью переключиться на новую проблематику...

[dmm]

"999 contributions in the last year"

[dmm]

"1101 contributions in the last year" - actively using GPT-4 now

[dmm]

1199 contributions

[dmm]

Twitter: 490 Following 300 Followers

[dmm]

200 karma points on LessWrong

[dmm]

300 karma points

[dmm]

350 karma point; it's time to formulate some intermediate results in my thinking about all this:

https://github.com/anhinga/2023-notes/blob/main/non-anthropocentric-ai-safety/understanding-as-of-july-2023/README.md

[dmm]

"1300 contributions in the last year" on GitHub - used GPT-4 heavily today to help me with TikZ LaTeX slides.

[dmm]

(Enzyme is an LLVM level differentiable programming system. In addition to Julia, it has a supported front-end in Rust, which I have not tried; but theoretically it should usable with a variety of compilers from various languages as long as those compilers target LLVM; however, there is an official restriction of a program being "statically analyzable", and unofficial existing restrictions are much tighter than that at the moment).