Software 2.0 and AI as an Engineering Discipline

In this post, we break down Stanford associate professor of computer science Chris Re’s recent talk on “Software 2.0” and AI as an engineering discipline and bring SVP of Unity Technologies Danny Lange into the conversation — a computer scientist who has worked on machine learning for IBM, Microsoft, Amazon Web Services and Uber. Danny gives his own take on this approach to AI and zero-code deep learning.

In a recent talk given to the Stanford Human-Centered AI institute, associate professor of computer science at Stanford Chris Re stated that weird new things are happening in software — the practices that once took center stage for AI researchers, particularly the obsession with models, have run their course.

Re claims that “models have become a commodity,” explaining that the most valuable ways people working in machine learning can spend their time is not innovating in models, but rather spending time where even the most powerful machine learning models still fall short, the fine-grained work, subtle interactions and disambiguations of terms.

This shift in focus can be seen in software as a whole, with Tesla’s AI scientist Andrej Karpathy calling AI “Software 2.0,” a new kind of software that “can be written in much more abstract, human unfriendly language, such as the weights of a neural network.”

Re’s main message is that Software 2.0 is turning AI into an engineering discipline with radically different systems and an attention to new “failure modes” of AI, and we now need “a new theory and practice to maintain and understand these systems and make them robust over time.”

This discipline has less emphasis on data structures, allowing engineers to focus on "monitoring the quality and improving supervision," instead of spending time tweaking hyper-parameters.

“There’s no mention of a model, there’s no mention of parameters, there’s no mention of traditional code” — it’s a zero-code deep learning approach.

Re observed that thanks to Software 2.0, “some machine learning teams actually have no engineers writing in those lower-level frameworks like TensorFlow and Pytorch.”

We asked SVP of Unity Technologies Danny Lange what his thoughts are on Software 2.0 and zero-code deep learning. Danny, a computer scientist who has worked on machine learning for IBM, Microsoft, Amazon Web Services and Uber, said:

“I understand Chris’ point, however he is a bit ahead of the curve with his statement. It’s not that simple to build a strong model. We often talk about large data sets but in complex models it’s still the details that count.”

Danny explained that at Unity, they’ve replaced TensorFlow with PyTorch since it’s an easier system to use.

“We are not yet ready for “no-code,” that said, we are working on a Model-as-a-Service within Computer Vision, and it’s our hope that they can be used by a broader range of users.”