OpenAI Codex (next generation) - it looks like we are finally "there"

[dmm]

There was a live demo of the next generation of OpenAI Codex code-generating software on August 10.

My impression from it is that "we have finally arrived" - this is a programming tool which seems to be more useful than an extra entry-level software engineer on the team. This starts to address the key bottleneck of our times: our limited ability to create software.

This was always my threshold: can we create an AI software which can be "hired instead of a junior software engineer"? That was the main temporal uncertainty for me: how long would it take to reach that level? It looks like this has been accomplished.

We are rapidly approaching the situation when AI will actively participate in programming AI software, for better or for worse...

OpenAI Codex is now a part of OpenAI API (which is still a closed beta with a waitlist), and it will be possible to participate in an informal competition today from 10am Pacific time (1pm Eastern) till 1pm Pacific (4pm Eastern) and try it a bit.

Links are in the comments.

Comments

[dmm]

The next one is super-impressive:

def compute_a_column(csv_contents: str, index: int)
    """
    csv_contents has lines
    lines contain words separated by commas
    form a list of one word from each line,
    where the selected word is situated at index position within its line
    """
    return [line.split(',')[index] for line in csv_contents.split('\n')[1:]] 

The system has even correctly figured out that I'd rather omit line 0.

[dmm]

Then it got less brilliant, generated wrong solution here

def compute_difference_between_two_dates(date1: str, date2: str)
    """
    compute difference between two dates in days
    """
    return abs((date1 - date2).days)

and then when I wanted to try this, the interface (Firefox, being quite flaky today with this challenge) just stopped working (tells me that I still have some unused assists, but does not work):

def convert_date_to_day(date: str):
    """
    date looks like 2014-05-20
    convert in to the number of days
    """

[dmm]

In some sense, one should remember that Codex is non-deterministic, and even if the first attempt did not produce a good result, one can always try again (100 repeated attempts might yield a pretty good performance, per Codex paper).

In the competition, the stand-alone Codex was not as fast as many of the participants, but it eventually solved all 5 problems on its own.

They did offer bonus problem number 6 to the leaders. I wonder, if they offered it to the stand-alone Codex as well...

[dmm]

Oh, they've now removed problem 6 related updates from the dashboard (I guess not everyone noticed it, and it seemed unfair to the winners of the originally defined run)

[dmm]

And it was not completely wrong, it just did not take the type into account.
Something like this is the correct solution, so it just ignored my type hint:

def compute_difference_between_two_dates(date1, date2):
    d1 = datetime.strptime(date1, "%Y-%m-%d")
    d2 = datetime.strptime(date2, "%Y-%m-%d")
    return abs((d1 - d2).days)