2/5 Week 1

• Tue 2/6: Overview of NLP and the class
  • Warm up: Least squares with least thinking
  • Reference (optional)
    • Advances in natural language processing
    • Talking to computers in natural language
    • Computational linguistics and deep learning
  • Demos: CoreNLP, AllenNLP, NLTK
• Thu 2/8: Applied machine learning: loss functions, bag of words, features, bag of vectors
  • Reference
    • Machine learning basics: 5.1, 5.2, 5.3
  • Reference (optional)
    • Percy’s slides on machine learning
    • Baselines: for comparisons

2/12 Week 2

• Tue 2/13: Feedforward neural networks and backprop
• Thu 2/16: Computation graphs and modules
  • Reference: section 3.2 of Autograd in ML
• Reference on background (optional)
  • Andrew Ng’s deep learning note
  • Hinton and Rumelhart, 1986: the original backprop paper for perspective
• Section on numpy
• Assignment 1 (due 2/26 11:55pm)

2/19 Week 3

• Basic model structures
  • windowed, convnet, RNN, deep RNN, bidirectional RNN, encoder/decoder
  • Collobert et al., NLP (almost) from scratch
  • Unreasonable effectiveness of RNNs
• More on RNN-like models
  • LSTM, GRU, SRU, identity initialization, clipping
  • Reference
    • Chapter on RNN
    • Understanding LSTMs
• Section: machine learning in practice

2/26 Week 4

• Word representations
  • Turney and Pantel 2010, From Frequencies to Meaning
  • word2vec, glove
• Project discussions (optional)
• Section: project ideas
• 2/26: Assignment 1 due

3/5 Week 5

• Encoder/decoder
  • seq2seq
  • Neural Machine Translation by Jointly Learning to Align and Translate
  • Attention and Augmented Recurrent Neural Networks
• Regularization and optimization
  • 7.1-7.12, 8.1-8.5 Deep learning
• Assignment 2 (due 3/27 11:55pm)
• Project planning (Due 11:55pm on 3/6)

3/12 Midterm week

• He He: Tuesday, March 13, 2018 - 12:30pm to 1:30pm
• Tue 3/13 1:45-2:45pm: Midterm review
• Thu 3/15 1:30pm at GREEN0S6: Midterm
  • The midterm
  • Solutions

3/19 Recess

3/26 Week 6

• 3/27: Canceled
• 3/29: Grammars and parsing
  • CFG, PCFG, constituency parsing
  • Dependency parsing
• CS Colloquiums
  • Danqi Chen: Monday, March 26, 2018 - 12:30pm to 1:30pm
  • Justin Johnson: Thursday, March 29, 2018 - 12:30pm to 1:30pm

4/2 Week 7

• 4/3: Semantic parsing
  • Readings
    • Executable semantic parsing
• 4/5: Generative models

4/9 Week 8

• 4/10: Theory of word embeddings (Misha)
• 4/12: Generative models continued
• 4/12: Project topics
  • Machine reading
    • DrQA [paper] [code]
  • Bag of tricks
  • Tagging sentences
• Project check-in 1

4/16 Week 9

• 4/17: Learning language games
  • Paper, Slides
• Jacob Andreas: Monday, April 16, 2018 - 12:30pm to 1:30pm
• Assignment 3 (due 5/3 11:55pm)

4/23 Week 10

• Bandit learning
  • Stochastic computation graph
  • RL tutorial
• MDPs
  • Chapters 3 and 4 of Sutton and Barto

4/30 Week 11

• Deep RL
  • ICML RL tutorial
• Learning to communicate
• Project checkin 2

5/7: Reading period

5/7: 12-2pm at Banana Corridor: Project poster session

5/15: Dean’s date