| 01 |
3/3 |
Intro to NLP, Deep Learning Basics [slides] |
|
|
| 02 |
3/8 |
Deep Learning Basics (2), Tokenization, Word Embedding, Text Classification [slides] |
|
MNIST, [Mikolov et al., 2014] |
| 03 |
3/10 |
Text Classification, Tokenization, Word Embedding, Recurrent Neural Networks [slides] |
|
|
| 04 |
3/15 |
Training, LSTM [slides] |
|
[Hochreiter & Schmidhuber, 1997] |
| 05 |
3/17 |
Jupyter Notebook, Token Classification, NER, MRC [slides] |
Assignment 1 is up |
Vanishing Gradients and Fancy RNNs |
| 06 |
3/22 |
Text Generation (Machine Translation, Summarization), Encoder-Decoder, Decoder Attention [slides] |
|
[Cho et al., 2014], [Bahdanau et al., 2015], [Luong et al., 2015] |
| 07 |
3/24 |
Encoder Attention, Transformer [slides] |
|
[Hermann et al., 2015], [Wang & Jiang, 2017], [Seo et al., 2017], [Vaswani et al., 2017] |
| 08 |
3/29 |
Transformer [slides] |
|
[Vaswani et al., 2017], Annotated Transformer |
| 09 |
3/31 |
Annotated Transformer [slides] |
Assignment 1 is due |
Annotated Transformer |
| 10 |
4/5 |
LayerNorm, Teacher Forcing, Beam Search, Byte Pair Encoding (BPE), Language Model [slides] |
Assignment 2 is up |
[Ioffe & Szegedy, 2015], [Ba et al., 2016] |
| 11 |
4/7 |
Language Model, Syntactic Parsing [slides] |
|
Syntactic Parsing |
| 12 |
4/12 |
Syntactic Parsing, Semantic Parsing [slides] |
|
Syntactic Parsing, [Zhong et al., 2017] |
| |
4/14 |
No lecture (Minjoon can’t make it) |
|
|
| |
4/19 |
No lecture (midterm week) |
Assignment 2 is due |
|
| |
4/21 |
No lecture (midterm week) |
|
|
| 13 |
4/26 |
Semantic Parsing, NLP paper analysis [slides] |
Assignment 3 is up |
[Chen et al., 2016], [Rajpurkar et al., 2016] |
| 14 |
4/28 |
NLP paper analysis discussions [slides] |
|
|
| 15 |
5/3 |
Pretrained Language Model [slides] |
|
[Peters et al., 2018], [Howard & Ruder, 2018] [Radford et al., 2018], [Devlin et al., 2018] |
| |
5/5 |
No lecture (어린이날) |
|
|
| 16 |
5/10 |
Pretrained LM and Tools [slides] |
Assignment 3 is due |
|
| 17 |
5/12 |
Pretrained LM Tools [slides] |
Assignment 4 is up (5/14 Fri) |
transformers |
| 18 |
5/17 |
Open-domain QA and Final Project Tutorial [slides] |
Final Project is up |
[Chen et al., 2017], [Karpukhin et al., 2020] |
| |
5/19 |
No lecture (석가탄신일) |
|
|
| 19 |
5/24 |
Nearest Neighbor Search, Large Language Models [slides] |
|
faiss, [Malkov & Yashunin, 2016], [Yang et al., 2019], [Liu et al., 2019], [Lewis et al., 2020] |
| 20 |
5/26 |
Large Language Models [slides] |
Assignment 4 is due (5/28 Fri 11pm) |
[Lan et al., 2020], [Clark et al., 2020], [Raffel et al., 2020], [Xu et al., 2020], [Sun et al., 2020], [Radford et al., 2019], [Roberts et al., 2020], [Hong et al., 2021] |
| 21 |
5/31 |
In-context Learning, Recent Trend in NLP [slides] |
|
[Kaplan et al., 2020], [Brown et al., 2020], [Ramesh et al., 2021] |
| |
6/2 |
No lecture (work on your final project!) |
|
|
| 22 |
6/7 |
Final project presentation |
|
|
| 23 |
6/9 |
Final project presentation |
|
|
| |
6/14 |
No lecture (finals week) |
|
|
| |
6/16 |
No lecture (finals week) |
Final Project Report is due |
|