“Training, Validation, Problem Diagnosis, and Troubleshooting of Using Deep
Neural Networks in Life Science Applications” given by Prof. Chun-Biu Li (Stockholm University).
Deep neural networks (DNNs) constitute the building blocks for many modern machine learning and data science applications, such as regression, classification, dimensionality reduction, generative models, deep reinforcement learning, etc. This course provides a comprehensive training on how to practically train and use DNNs properly for statistically modeling. To be self-contained, an overview of basic concepts of DNNs relevant to life science applications will be included in the course. Instead of mathematical technicalities, the course will then focus on establishing intuitive but deep understanding of the training and validation processes of DNNs. Topics covered will include choosing appropriate state-of-the-art network architectures, model hyper-parameters, optimization methods, practices in monitoring the training processes, identifying improper training and troubleshooting, imposing suitable regularizations and their effects, training time complexity and possible speedups, validation of the resulting DNNs and troubleshooting, effective scrutiny of the DNN outputs to facilitate biological interpretations, etc.
Date:
Online: 17, 18 Apr. (3 hours each in the afternoon)
On-site: 23 – 25 Apr. (please bring your own laptop)
(A casual party with poster session (poster presentation is encouraged but not mandatory) is planed in the evening of 23 Apr)
Venue: Room 380, Electrical Engineering and Applied Physics (EIPE) Building Annex 1, Aobayama Campus, Tohoku University
（東北大学 青葉山キャンパス 電気情報応物系１号館別館 380号室）
Target audience: PhD students, postdocs, young faculty members.
Language: English
Participation fee: Free
Registration form (deadline: 10 Apr)
Registration is limited to those who can attend all classes.
We limit the number of participants to 20 (first come, first served).
Please refer to the syllabus below.
Contact:
Prof. Shoichi Toyabe / 鳥谷部 祥一 (Tohoku University)
toyabe@tohoku.ac.jp
——-
Time Table (TENTATIVE)
ONLINE Section (Please find the Zoom URL in the e-mail sent to the participants)
| Apr 17 (Thu) | Apr 18 (Fri) | |
| 14:00 – 17:00 | Overview of basic concepts of deep learning | Overview of basic concepts of deep learning |
ON-SITE Section
| Apr 23 (Wed) | Apr 24 (Thu) | Apr 25 (Fri) | |
| 09:30 – 12:00 | Optimization Methods | Regularization Methods | Evaluation Methods |
| 12:00 – 13:30 | Lunch | Lunch | Lunch |
| 13:30 – 16:00 | Discussion, Practice & Short Presentations | Discussion, Practice & Short Presentations | Discussion, Practice & Short Presentations |
| 16:00 – 17:00 | Welcome mixer | Lab Tour | |
| 17:00 – | Izakaya party |
Syllabus (TENTATIVE)
Learning Outcomes:
After completion, participants are expected to be able to
• Explain basic and modern concepts of DNNs
• Select appropriate training, validation methods and software to model a given life science dataset
• Compare strengths and weaknesses of various DNNs architectures, training, regularization and validation schemes
• Perform problem diagnosis and troubleshooting when applying DNNs to real life science problems
• Present analyses and results logically in both written report and oral presentation
Prerequisites:
• The camp targets junior researchers (PhD students or above) conducting biological studies who may have little or small experience in applying deep learning to analyze and model their data.
• Basic knowledge in linear algebra, calculus, probability and statistics with levels equivalent to those provided in bachelor courses for science students
• Have experience in coding (preferable in R, Python or Matlab), and using machine learning/AI libraries and functions
Tentative Plan (2 online and 3 on-site meetings):
* The following plan can be subject to change depending on participants’ learning progress and interests
Meeting 1 (Online): Overview of basic concepts of deep learning
Overview of DNN architectures (feedforward, convolutional, recurrent, gating, residual, U-net, attention, encoder-decoder, etc.), various input formats and output functions, various activation functions, etc.
Meeting 2 (Online): Continue on basic concepts of deep learning
Overview of DNN architectures (feedforward, convolutional, recurrent, gating, residual, U-net, attention, encoder-decoder, etc.), various input formats and output functions, various activation functions, etc.
Meeting 3 (On-site): Optimization
Stochastic gradient descent and its generalizations, setting hyperparameters, parameter initialization, vanishing and exploding gradients, time complexities and possible speedups.
Meeting 4 (On-site): Regularization
Various regularizations, including norm penalties, data augmentation, noise injections, early stopping, dropout, batch and layer normalizations, etc., and their effects.
Meeting 5 (On-site): Evaluation
Various evaluation techniques, including the monitoring of training/validation curves, parameter updates, and the transformation of representations across layers, error quantifications, model selections, diagnosis of training problems.
Practice Sections (On-site)
Participants divided into groups according to their backgrounds to carry out projects. They are encouraged to bring data/model from their own studies and apply what they learn from the meeting. They can also pick up existing data/model from well known databases to carry out the analysis. Participants are expected to bring their laptops to the on-site sections.