DeepMind’s AlphaCode explained—should programmers be worried?

DeepMind, a Google’s British AI subsidiary, has revolutionized the world of technology even more by creating an AI ready to challenge and beat humans in a programming contest: AlphaCode.  The system itself is using models of language to generate codes on an unprecedented scale, followed by a filtering to promising programs. 

According to DeepMind’s website, “AlphaCode achieved an estimated rank within the top 54% of participants in programming competitions by solving new problems that require a combination of critical thinking, logic, algorithms, coding, and natural language understanding”. It’s really simple: when AlphaCode is facing a problem, it shows up with a bunch of solutions and then picks the best ten solutions. 

But this is really promising. As stated by Professor Ernest Davis, from the computer science department at New York University, “this is a breakthrough in the sense that they are solving programming problems much more complex than earlier systems”, like OpenAI Codex. 

But not everything is a sea of roses. According to the same professor, since the job of AlphaCode is to come up with a range of solutions for a problem and find the best ones, the amount of data to create complex programs is going to be huge. 

AlphaCode is the kind of deep learning system that is generally known as a sequence-to-sequence model (seq2seq). This algorithm model grabs a sequence of values (letters, pixels, numbers, etc.) and then produces another sequence of values. 

In line with DeepMind’s paper, AlphaCode uses an encoder-decoder Transformer architecture. In recent years, Transformers have become popular due to the fact that they are, obviously, an upgrade of their predecessors, because they require much less memory and compute requirements and can handle large sequences of data. As stated by Ben Dickson, on TechTalks Newsletter, “the encoder part of AlphaCode creates a numerical representation of the natural language description of the problem. The decoder part takes the embedding vector produced by the encoder and tries to generate the source code of the solution”. 

In order for AlphaCode to become a real thing, DeepMind’s scientists used a combination of unsupervised pre-training and supervised fine-tuning. This type of approach happens when there isn’t enough labeled data or data annotation is expensive and time-consuming. 

According to Mike Mirzayanov, founder of Codeforces, “I can safely say the results of AlphaCode exceeded my expectations. I was skeptical because even in simple competitive problems it is often required not only to implement the algorithm, but also (and this is the most difficult part) to invent it. AlphaCode managed to perform at the level of a promising new competitor. I can’t wait to see what lies ahead!”. 

AI vs humans

If we took the examples of Deep Blue and AlphaGo, the AI systems that beat the world champions at chess and Go, they excel at that task, but you can’t put them competing with other humans on other tasks that require careful planning and strategizing, because they are not programmed to do that. The same thing happens in competitive programming, because a human being spends years studying programming and has the abilities to solve other tasks, sometimes even simpler tasks, but AlphaCode is only programmed to do that. 

In the opinion of Dzmitry Bahdanau, from ServiceNow, “AlphaCode is a useful contribution to AI research, but AI coding at the level of a skilled human is still light-years away”. So, will AlphaCode send a lot of software programmers to unemployment? Only time will tell, but the truth is that we should be more focused on what these types of programming devices and software programmers could do together, instead of putting them against each other to see who is the best. 

To conclude and quoting again Ben Dickson, from TechTalks Newsletter, “AlphaCode should be recognized for what it is: a code generator that can propose good candidate solutions for well-formulated problem statements. It should also be recognized for what it is not: the digital equivalent of a human programmer”. 

Let me know in the comments what your take on AlphaCode is.

PS: looking for a job in AI? Check these out.