R202, Astronomy-Mathematics Building, NTU
Speaker(s):
Rio Yokota (Global Scientific Information and Computing Center, Tokyo Institute of Technology)
Organizer(s):
Weichung Wang (National Taiwan University)
一、 課程背景與目的：
This course is part of the series of events of the Medical AI Research Collaboration Hub (MARCH).
MARCH is a year-long educational and research development program aimed to guide students and researchers along medical artificial intelligence projects with real-world impact for healthcare patients and providers around the world. The program is organized around a workshop series in key topic areas to provide participants with comprehensive technical knowledge and thought leadership in the area of medical AI. Throughout the program, participant teams will form and be mentored by the organizers and invited faculty as they develop their project. Participants will have access to a broad range of resources provided by our sponsor and organizing institutions, including events held both in Taiwan and the United States.
Our goal is to provide an environment where true international, interdisciplinary collaboration can foster the development of new AI technologies to meet critical needs in healthcare. We believe strongly that success in this field requires long-term, deep collaboration between medicine, mathematics, and computer science. Our program is therefore open to physicians, researchers, students, and scientists, all of whom have a key role to play in the future of healthcare technology.
二、課程之大綱：
Deep neural networks are usually trained using first order stochastic gradient descent. In this short course, we investigate the possibility of using second order optimization methods such as natural gradient decent, generalized Gauss-Newton, and Newton methods. We will discuss the similarities and differences between these second order methods through the relationship between curvature (Riemann geometry of the parameter space) and covariance (Bayesian statistics of the parameter distribution). The course will also have a hands-on component where demonstrations will be given using a PyTorch implementation. We will also discuss fast approximation techniques and their distributed parallel implementation.
三、課程詳細時間地點以及方式：
2019/07/01 9:00am-12:00pm
Introduction to optimization methods for deep learning (1.5 hours) + hands-on (1.5 hours)
2019/07/02 9:00am-12:00pm
Second order optimization methods (1.5 hours) + hands-on (1.5 hours)
2019/07/03 9:00am-12:00pm
Fast approximation for second order methods (1.5 hours) + hands-on (1.5 hours)
2019/07/04 9:00am-12:00pm
Distributed parallel training with second order methods (1.5 hours) + hands-on (1.5 hours)
