In 2014, one of the main worries of a deep learning researcher was whether neural networks can perform certain tasks at all.
It was a breakthrough – and many could hardly believe it – that a the neural network could translate from English to German on par with a phrase-based system. Less than a decade later, GPT4 can not only translate into many languages without any explicit translation training,but can perform almost all NLP tasks better than previous specialized systems and explain its own work in the process. I will show how the field and the techniques evolved in the last decade demonstrate GPT4 to showcase the current state of the art, and discuss how we learn from the past decade to decide what to focus on in the future.
Lukasz is a deep learning researcher at OpenAI and was previously part of the Google Brain team. He recently worked on GPT4 and Codex models. Earlier, he co-invented Transformers and other neural sequence models and co-authored the TensorFlow system. Before working on machine learning, Lukasz was a tenured researcher at the University of Paris Diderot and worked on logic and automata theory. He received his PhD from RWTH Aachen University in 2008 and his MSc from the University of Wroclaw, Poland.