![[personal profile]](https://www.dreamwidth.org/img/silk/identity/user.png){kind=small}
OpenAI code generation breakthrough
May. 22nd, 2020 08:20 pmIn this video Microsoft CTO is interviewing OpenAI CEO starting from 25:00 mark (right before this mark he is talking about a huge computer system Microsoft created for OpenAI; the style of this overall Microsoft video does feel quite weird to my taste, but this fragment with Sam Altman is good):
twitter.com/matvelloso/status/1263193089310461952
At about 29:00 mark OpenAI demos their new transformer-based code-generating system trained on a large subset of GitHub. I'd say, it's quite impressive, it does feel like a breakthrough in coding-assisting tools. Some discussion here:
news.ycombinator.com/item?id=23250379
Generally speaking, people are saying lately that large modern transformer models only pretend to be sequence-to-sequence, but in reality they learn tons of structured linguistic information, see e.g. this informal essay-style paper and references therein:
arxiv.org/abs/2005.06420 "The Unstoppable Rise of Computational Linguistics in Deep Learning"
(This is not yet a artificial junior software engineer one can hire, but this OpenAI prototype is a considerable step in that direction. May 20, 2020 will be remembered as an important milestone.)
twitter.com/matvelloso/status/1263193089310461952
At about 29:00 mark OpenAI demos their new transformer-based code-generating system trained on a large subset of GitHub. I'd say, it's quite impressive, it does feel like a breakthrough in coding-assisting tools. Some discussion here:
news.ycombinator.com/item?id=23250379
Generally speaking, people are saying lately that large modern transformer models only pretend to be sequence-to-sequence, but in reality they learn tons of structured linguistic information, see e.g. this informal essay-style paper and references therein:
arxiv.org/abs/2005.06420 "The Unstoppable Rise of Computational Linguistics in Deep Learning"
(This is not yet a artificial junior software engineer one can hire, but this OpenAI prototype is a considerable step in that direction. May 20, 2020 will be remembered as an important milestone.)
