Citations; software progress

[dmm]

I have always been feeling somewhat awkward that my most cited paper has been an industrial pre-machine-learning paper on how to extract geographic references from a text.

Recently, our 2009 paper in American Mathematical Monthly on non-zero self-distances has finally surpassed it in the number of citations.

Meanwhile, my two papers I like the most have exactly zero citations.

5 months in a new job: among software achievements: now I know how to take autocomputed gradients with respect to variables assembled inside nested dictionaries. So I am no longer forced to reshape complicated tree-like-structures into flat arrays in order to use differentiable programming.

As a result, I can finally experiment with DMM training using gradient methods without putting too much labor into those experiments.

🇺🇦 🇺🇦 🇺🇦 Links are in the comments 🇺🇦 🇺🇦 🇺🇦

Comments

[dmm]

Citations: https://scholar.google.com/citations?user=bCBmorYAAAAJ

"Partial metric spaces" is finally leading "A confidence-based framework for disambiguating geographic terms", with 214 citations vs 213 citations.

Meanwhile, my 2 best papers (both published in 2017) have zero citations:

"Dataflow matrix machines and V-values: A bridge between programs and neural nets": https://arxiv.org/abs/1712.07447

"Revisiting EPRL: All Finite-Dimensional Solutions by Naimark’s Fundamental Theorem": https://www.cs.brandeis.edu/~bukatin/revisiting-eprl.html

[dmm]

Other papers I wrote or co-wrote which I like:

Before 2000:

"Towards computing distances between programs via Scott domains", the free PDF is only available as Chapter 10 of my PhD Thesis "Mathematics of Domains": https://arxiv.org/abs/1512.03868

"Partial metrics and co-continuous valuations": https://link.springer.com/content/pdf/10.1007/BFb0053546.pdf (that's the one I could not present, getting travel documents was taking too long; this one became my third most cited one)

The rest of the material in "Mathematics of Domains".

***

After 2010:

"Some corollaries of the correspondence between partial metrics and multivalued equalities": this particular paper is available from me upon request (it's behind a paywall on the journal site), and it is the final accord of this cycle of work: https://www.cs.brandeis.edu/~bukatin/distances_and_equalities.html

***

The present cycle of work (this cycle of work gets zero external citations):

https://www.cs.brandeis.edu/~bukatin/partial_inconsistency.html

and

https://www.cs.brandeis.edu/~bukatin/dmm_next.html

[dmm]

Google Scholar just adjusted its counts (I lost 11, 783 is the new total).

The "Partial metric spaces" is now leading 213 vs 208.

[dmm]

Finally, 800 citations on Google Scholar (224 vs 211 for the two leading papers, so that 14 new citations for these two papers out of 17 new overall)

[dmm]

Gradients with respect to nested dictionaries:

JAX (Python):

https://github.com/anhinga/jax-pytree-example

Julia Flux (Zygote.jl):

https://github.com/anhinga/julia-flux-drafts/tree/main/arxiv-1606-09470-section3

This also can be done in Enzyme.jl (also Julia) to some extent, but Enzyme is still too fragile and restrictive.

No other differentiable programming framework can done this kind of thing, to the best of my knowledge.

[dmm]

The experiments with using gradients methods for DMM training are not open-source yet (alas, but open-sourcing them will have to wait).

[dmm]

GitHub shows "600 contributions in the last year" for the first time in my life. Unfortunately, almost a half of them are in private repositories.

I am getting very nice generalization properties and nice "intermediate sparsification" results in my latest series of experiments.

[dmm]

Nice "full sparsification" which generalizes well...

Ok, therefore June 27, 2022 is the day when the first DMM program synthesis primer has been successfully created.

[dmm]

661 contributions; left the job yesterday, disconnected from their org: minus 476 contributions = 185.

But the key experimental results are now open sourced.

[dmm]

400 contributions on GitHub in the last year (the last 5 weeks have been fruitful).

Pondering what to do with respect to trying to actually get some collaborative efforts on all this...

[dmm]

Reached 500 of them last week, before 52 of them expired; currently at 463.

Still pondering what to do with respect to trying to actually get some collaborative efforts on all this...

[dmm]

Had a great dialog which prompted me to start doing a straight port of 2017 Clojure version to Julia:

https://github.com/anhinga/late-2022-julia-drafts/tree/main/dmm-port-from-clojure

500 contributions on GitHub in the last year again.

[dmm]

1) Finally has implemented a "superfluid" architecture (unearthed and developed our 2018 design) with flexible "activation function" inside a neuron; so, finally, we have achieved a "fully type-free" version (the Clojure one has been "almost type-free")

2) The standard first self-referential experiment (network inducing a wave pattern in its own connectivity matrix) just started to work, so we can now say that a port from Clojure to Julia does exist

3) 554 contributions on GitHub in the last year at the moment (9 of them will expire soon)

[dmm]

600 contributions on GitHub in the last year at the moment.

Just finished an outline for a possible book on "Machine Learning in Julia" (I was approached and asked to explore this as a possible book project; keeping this one in a private repository for the time being).

[dmm]

Just reached 662 contributions on GitHub; finally got above Sep 17.

Not doing the book (I decided it does not make sense to undertake such a project now, and especially on the publisher's terms).

[dmm]

"666 contributions in the last year"

[dmm]

"700 contributions in the last year"

[dmm]

"800 contributions in the last year"

[dmm]

"888 contributions in the last year"

We have a new preprint, "Safety without alignment": https://arxiv.org/abs/2303.00752

[dmm]

"900 contributions in the last year"

Полгода назад кончилась та служебная деятельность; сейчас бы она мне очень мешала...

На самом деле, это была пятница, 16-е сентября, а в понедельник, 19-го сентября Скотт Александр опубковал заметку про деятельность Януса, и, как раз, оказалось уместно полностью переключиться на новую проблематику...

[dmm]

"999 contributions in the last year"

[dmm]

"1101 contributions in the last year" - actively using GPT-4 now

[dmm]

(Enzyme is an LLVM level differentiable programming system. In addition to Julia, it has a supported front-end in Rust, which I have not tried; but theoretically it should usable with a variety of compilers from various languages as long as those compilers target LLVM; however, there is an official restriction of a program being "statically analyzable", and unofficial existing restrictions are much tighter than that at the moment).

[timelets]

Funny, how publications and real stuff diverge. My most referenced publication turned out to be nothing much in practice, but the work we did almost unnoticed got eventually adopted by the the entire industry, litigated for years, etc.

Good luck with your new endeavor!

[dmm]

Yes, I think this is more often the rule, that it's pretty typical that they diverge...

Thanks!