Speaker: Yoshua Bengio
Department of Computer Science and Operations Research
University of Montreal
Title: Experimental Investigations into Deep Architectures
Abstract: Whereas theoretical work suggests that deep architectures might be
more efficient at representing highly-varying functions, training deep
architectures was unsuccessful until the recent advent of algorithms
based on unsupervised pre-training. Even though these new algorithms
have enabled training deep models, many questions remain as to the
nature of this difficult learning problem and on the limitations of
current algorithms for deep architectures. We attempt to shed some
light on these issues in several ways, mostly using hypothesis -
experiment iterations, starting from a few more questions. What do
different successful approaches to deep architectures have in common?
Is unsupervised pre-training helping to train deep architectures
because it acts like a regularizer or does it help with optimization?
Why are purely supervised deep neural networks so difficult to train?
Is there something wrong with Contrastive Divergence and Gibbs
sampling in RBMs that explains very poor mixing sometimes observed?
Can nature still teach us more tricks about neural networks and
learning the kind of complicated abstractions that we hope deep
architectures will capture? I will briefly describe some of the
recent investigations at U. Montreal towards answering these
For Additional Information, contact:
Hugo Larochelle, http://www.cs.toronto.edu/~larocheh/