Speaker: Dani Yogatama, Carnegie Mellon University
Title: Learning to Represent Language: Embeddings and Optimization
The performance of a machine learning model heavily depends on how the data is represented in the model. For example, when working with text data we can represent words as strings, binary vectors, or real vectors. Recent advances have sought to automate these choices to save human costs and achieve better results.
In this talk, I will first discuss how to encode prior knowledge into a representation learning model with structured regularizers. I will show an application of this technique to learn a word embedding model that encourages hierarchical ordering of word meanings (e.g., "professor" is a hyponym of "academic").
I will then talk about how to efficiently perform representation learning in a setting where we need to work with streams of datasets. I will present a method to quickly decide how best to represent a dataset by treating this problem as an optimization over a very large discrete set of choices. I will show how we can use Bayesian optimization to solve it efficiently and briefly describe an improved Bayesian optimization algorithm that can generalize across multiple datasets.
I will conclude by discussing applications of my work on representation learning in real-world systems and outlining future directions.
Dani Yogatama received his PhD from Carnegie Mellon University in 2015. Prior to CMU, he was a Monbukagakusho fellow at the University of Tokyo. He is currently a research scientist at Baidu Silicon Valley AI Lab. His research interests include machine learning, large-scale optimization, and applications to natural language processing and speech recognition.