Published on U of T News
The University of Toronto’s Geoffrey Hinton has been named one of three recipients of the prestigious A.M. Turing Award – dubbed by some “the Nobel Prize of computing” – for his decades-long work advancing the field of artificial intelligence.
Hinton, a U of T University Professor Emeritus who also works at Google, was named alongside two of his friends and longtime collaborators – Yann LeCun, of Facebook and New York University, and Yoshua Bengio, of Université de Montréal.
The Association for Computing Machinery lauded the three for making “conceptual and engineering breakthroughs” in deep neural networks – an area of AI inquiry that was once deemed fanciful, but is now considered transformational and is touching the lives billions of people.
"It's very nice that the group of us have been recognized as having a big impact,” said Hinton, who has sometimes been referred to as the “godfather” of deep learning, as the field is known.
“It's sort of an acceptance by the computer science community that this stuff isn't crazy.”
The A.M. Turing Award, named for British mathematician Alan Turing, who first articulated the mathematical foundation and limits of computing, is the premier international award for achievements in computing and includes a $1 million prize funded by Google.
It’s the second time the award has been given to a U of T researcher. University Professor Emeritus Stephen Cook won it in 1982 “for his advancement of our understanding of the complexity of computation in a significant and profound way.” Alumnus and former faculty member William Morton Kahan also won the award in 1989 while he was a faculty member at the University of California, Berkeley. Former postdoctoral researcher Silvio Micali, now a faculty member at MIT, won in 2012.
“I want to congratulate Geoffrey Hinton, Yann LeCun and Yoshua Bengio for their incredible work – and sheer perseverance – in advancing the field of artificial intelligence over so many years,” said U of T President Meric Gertler.
“Their determination to follow paths others had rejected is testament to the importance of supporting basic research, which may not always produce results quickly – but can lead to profound discoveries over time that forever change our world.”
In recent years, the deep learning techniques Hinton, LeCun and Bengio helped pioneer have been used to achieve breakthroughs in a range of applications – everything from voice recognition software in people’s phones to algorithms that can spot cancerous tumours in medical scans. That, in turn, has sparked a gold rush for Canadian talent, prompting nearly a dozen multinational companies to set up AI labs in cities like Toronto, Montreal and Edmonton.
It wasn’t always this way. Hinton recalled a time when he, LeCun and Bengio were relegated to the fringes of computer science research for their insistence that computers could learn in a way that mimics the human brain.
"We stuck to that when almost everyone else in artificial intelligence and computer science didn't,” Hinton said.
“It wasn't quite the same in psychology, where a lot of people believed in neural networks. But if you looked at the people doing computer science and machine learning, they became thoroughly disenchanted with them."
Hinton said he first met LeCun at an academic conference in Paris in 1985. Intrigued by his work, he later agreed to become his external thesis adviser. There was only one small problem: “The exam was in French and I didn't speak French, so the only thing I understood in his thesis was the equations,” Hinton said. “But that was the important part."
LeCun would later do post-doctoral research in Hinton’s Toronto lab before going on to work at AT&T’s Bell Labs, where his early work in convolutional neural networks would be used to read hand-written text. At the time, however, few people believed the technique could be applied to photographs of three-dimensional scenes.
Hinton’s relationship with Bengio, who also spent time at Bell Labs, would develop a few years later, though neither one can pinpoint the exact date.
Regardless, all three men were moving in the same circles and, research-wise, were on the same page.
Beginning in 2004, the trio’s work really started to come together. That’s when Hinton set up a program with the Canadian Institute for Advanced Research (CIFAR) focused on machine learning and computer vision, which included both LeCun and Bengio.
Now meeting on a regular schedule, the three began regularly sharing ideas that informed each other’s work. In particular, Hinton recalled holding unofficial meetings in advance of a conference the three of them attended every year in Vancouver.
"Those were very important meetings,” Hinton said. “We would discuss all sorts of unpublished ideas and it was relatively small group of people, so you didn't need to waste time describing basic things.
“That was very useful for making progress."
Hinton said the CIFAR program was key to advancing the group’s research. And he stressed it wouldn’t have happened without the help of past CIFAR head Chaviva Hosek, who was a professor at the Munk School of Global Affairs & Public Policy and “who got the point very quickly,” and another senior CIFAR executive named Mel Silverman, a University Professor Emeritus of medicine, who helped sketch out a research proposal.
"The funding allowed us to work together again."
For Hinton, a big public breakthrough came in 2012 after he and two of his students, Alex Krishevsky and Ilya Sutskever, entered an image recognition and classification competition called ImageNet and shocked the computer vision community by almost halving the next-best error rate.
“We showed that the computer vision community was completely wrong and that Yann's convolutional nets, with a few extra tricks, could do much better than they could at recognizing things in photographs," Hinton said.
Deep learning had finally arrived, although it’s not like Hinton, LeCun or Bengio were the least bit surprised.
"We all had very similar basic beliefs about how neural networks were the right things to do,” Hinton said, “and that they would win in the end."