Lessons from a Nobel Laureate (on deep learning and academia/industry)
Last week I saw John Jumper give a presentation on his team’s development of AlphaFold 1/2/3. He started explaining how AlphaFold1 worked, essentially a convolutional neural network. He spent most of the talk explaining the many changes required to produce AlphaFold2, which went from mediocre predictive accuracy to changing the whole field.
I took away two main points.
There was no single trick that lead to the big performance gain of AF2 over AF1. No single bit of intuition that lead to such a huge shift in performance. Instead, lots of good ideas each of which increased performance slightly.