KAIST AI605 - Deep Learning for NLP (Fall 2021)
Links
- Schedule & Materials
- Lecture Videos on YouTube
- Q&A on GitHub Discussions
- Instructions for NAVER Students
- KAIST AI605 - Spring 2021
Time & Location
Mon & Wed 16:00-17:15 via Zoom
- Visit KLMS or email a TA (or the instructor) for the invitation
- The class will be fully virtual
- All lectures and materials will be in English
Instructor
Minjoon Seo
-
Office: KAIST Seoul Campus Building 9 Room 202
-
Office Hours: MW 17:30-18:00 via email
TAs
Miyoung Ko
Yongrae Jo
Minki Kang
Jaehyeong Jo
Grading Policy
There is no exam in this class, and the final grade has the following breakdown:
- Assignments: 80% or 40%
- Project: 0% or 40%
- Participation: 20%
Note that you can choose either (1) complete all the four assignments, or (2) complete two assignments of your choice and complete the project (recommended if you are already working on an NLP project). If you complete all of the four assignments and the project, we will choose the option that gives you a higher grade.
The final grade will be determined by the following policy (after rounding to the nearest integer in percent):
- 93% or higher: A0 or A+ (only few students will get A+)
- 90% to 92%: A-
- 87% to 89%: B+
- 83% to 86%: B0
- 80% to 82%: B-
- 77% to 79%: C+
- 73% to 76%: C0
- 72% or lower: C- or lower (fail)
Late Submission Policy
- We will give you 7 no-penalty late days that can be used across all assignments (but not the final project).
- After you used all late penalty days, there will be -10% penalty for every late day (24 hours). That is, if the assignment or the project is due at 11pm and you submit at 11:30pm on the next day, 20% will be deducted.
- Note that when there are bonus questions, you can still achieve 100% (or higher) with a late assignment.
- An assignment that is more than 7 days late will not be accepted and you will receive 0% for that assignment.
Course Description
This course covers recent advances in natural language processing area driven by deep learning. Topics include (but are not limited to)
- Recurrent Neural Networks
- Encoder-Decoder
- Transformer
- Language Model
- Pretrained Language Model (e.g. BERT)
- Text Classification (e.g. Sentiment Analysis)
- Sequence Tagging (e.g. NER, Question Answering)
- Sequence Generation (e.g. Summarization, Machine Translation, Semantic Parsing)
- Zero-shot Learning (e.g. GPT-3)
- Modern NLP Libraries (e.g. Hugging Face)
Assignments
- Assignment 1: Implement RNNs to create a sequence classifer
- Assignment 2: Implement a passage retrieval system using
faiss - Assignment 3: Implement Transformer to create a sequence-to-sequence model
- Assignment 4: Using BERT to create a sequence classifier and a token classifier (for QA)
Project
The deliverable for the project will be a 4-to-8-page report. You can choose your own topic for the project, but if you don’t have one, we encourage you to work on creating an open-domain question answering system on EfficientQA dataset (detailed description will be available). In this case, we recommend you to complete Assignment 2 and 3.
Textbook
There is no required textbook for this course but I highly recommend Speech & Language Processing, whose pdf version is available for free, for your reference.