OpenAI code generation breakthrough

[dmm]

In this video Microsoft CTO is interviewing OpenAI CEO starting from 25:00 mark (right before this mark he is talking about a huge computer system Microsoft created for OpenAI; the style of this overall Microsoft video does feel quite weird to my taste, but this fragment with Sam Altman is good):

twitter.com/matvelloso/status/1263193089310461952

At about 29:00 mark OpenAI demos their new transformer-based code-generating system trained on a large subset of GitHub. I'd say, it's quite impressive, it does feel like a breakthrough in coding-assisting tools. Some discussion here:

news.ycombinator.com/item?id=23250379

Generally speaking, people are saying lately that large modern transformer models only pretend to be sequence-to-sequence, but in reality they learn tons of structured linguistic information, see e.g. this informal essay-style paper and references therein:

arxiv.org/abs/2005.06420 "The Unstoppable Rise of Computational Linguistics in Deep Learning"

(This is not yet a artificial junior software engineer one can hire, but this OpenAI prototype is a considerable step in that direction. May 20, 2020 will be remembered as an important milestone.)

Comments

[dmm]

Of course, when an imperfect code-generator is used in this fashion, the code has to be easily readable and understandable by a human. Otherwise, it would be impossible to evaluate its quality.

And readable and understandable code is, by itself, a super-important goal - so there is harmony here; this system does generate nice code (achievable by censoring which repositories are admissible as training data).

Meanwhile Microsoft released this thing, which is "officially unrelated" to the OpenAI effort (but there will be an attempt to integrate a version of OpenAI system into IntelliCode in the future): https://arxiv.org/abs/2005.08025 "IntelliCode Compose: Code Generation Using Transformer"

[dmm]

Ah, so there was some functionality in this direction in the IntelliCode, although less spectacular:

https://hub.packtpub.com/microsofts-visual-studio-intellicode-gets-improved-features-whole-line-code-completions-ai-assisted-refactoring-and-more/

***

https://microsoft.github.io/prose/

https://www.microsoft.com/en-us/research/publication/on-the-fly-synthesis-of-edit-suggestions/

https://devblogs.microsoft.com/visualstudio/ai-assisted-developer-tools/

( https://twitter.com/amandaksilver/status/1191573487191838722 )