|
|
Course Motivation and Description
Machine learning enables computers to learn about the world around us but also holds fundamentally hard challenges associated with the so-called curse of dimensionality: the huge number of possible observations, events, or configurations of variables. Deep learning has been introduced to face that challenge by adding to the rich science of machine learning the notion of deep representation, the idea that better models can be learned if the machine constructs and discovers rich and abstract representations of the data. Past and future advances in deep learning hold incredible promises of technological advances on the path towards AI. This realization has strongly influenced information technology markets recently and there are already impressive fallouts from these investments in science and technology.
This tutorial will cover some of the main current topics in deep learning research and applications, starting from the theoretical underpinnings of distributed representations and depth, as well as a detailed description of the most commonly used method for obtaining parameter gradients, i.e., the backpropagation algorithm. It will show how these ideas are incorporated in convolutional neural networks (for images) and recurrent neural networks (for capturing sequential structure). Although the deep learning breakthroughs started with unsupervised learning, most of the current applications have focused on supervised learning, as many challenges but also major promises remain, in the land of deep unsupervised learning. A brief introduction will be given to the current state-of-the-art in this area and how these ideas are motivated the point of view of geometry (manifold learning) and the discovery of underlying causal factors. The tutorial will close with the lighter subject of applications of deep learning in industry, with a focus on computer vision and image processing.
Slides
Slides part 1
Slides part 2
Instructors
Yoshua BENGIO (PhD in Computer Science, McGill University, 1991) did two post-docs at M.I.T. (Michael Jordan) and AT&T Bell Labs (Yann LeCun), then became professor at the Department of Computer Science and Operations Research at Université de Montreal. He authored two books and around 200 publications, the most cited being in the areas of deep learning, recurrent networks, probabilistic learning, natural language and manifold learning. He is among the most cited Canadian computer scientists and is or has been associate editor of the top journals in machine learning and neural networks. Since '2000 he holds a Canada Research Chair in Statistical Learning Algorithms, since '2006 an NSERC Industrial Chair, since '2005 is a Fellow of the Canadian Institute for Advanced Research. He is on the the NIPS foundation board and has been program chair and general chair for NIPS. He has co-organized the Learning Workshop for 14 years and co-created the new International Conference on Learning Representations. His current interests are centered around a quest for AI through machine learning, and include fundamental questions on deep learning and representation learning, the geometry of generalization in high-dimensional spaces, manifold learning, biologically inspired learning algorithms, and challenging applications of statistical machine learning.
Roland MEMISEVIC (PhD in Computer Science, University of Toronto, 2008) held positions as research scientist at PNYLab, Princeton, as post-doc at the University of Toronto and at ETH Zurich, and as a junior professor at the University of Frankfurt, Germany. In 2012, he joined the University of Montreal as an assistant professor in Computer Science. His research interests are in deep learning and computer vision with a focus on approaches that extend deep learning beyond object recognition towards more general tasks in vision and AI. His scientific contributions include approaches to learning motion and transformation patterns from images and videos, and approaches to learning invariance from data. He presented his work at conferences such as NIPS, CVPR, ICCV, ICML, AAAI, and in journals including PAMI, Neural Networks, Neural Computation. He served as a program committee member or reviewer for most of these and other conferences and journals in machine learning and computer vision. Roland Memisevic has been invited speaker at numerous deep learning events and tutorials.
|
|
