COS 302 / SML 305 / ECE 305: Mathematics for Numerical Computing and Machine Learning

TA: David Xu(OH: Tues 2-4pm, COS 003)
TA: Andre Niyongabo Rubungo (OH: Mon 1:30-2:30pm, common space outside of CS 241; Mon 5-6pm, common space outside of CS 241)
TA: Haichen Dong (OH: Mon 7-8pm, zoom; Tue 11am-12pm, Friend 010)

Lab TA: Aditya Palaparthi (OH: Sun 10-11am, zoom; Mon 3-4pm, common space outside of CS 241; Tues 8-9pm zoom)
Lab TA: Kevin Kim (OH Mon 8-9pm, Lewis Library 406)
Lab TA: Evan Wang (OH: Tues 4:30-6:30pm, zoom)

Precept 01 (Haichen): Thu 10:00-10:50am in Friend 110
Precept 02 (Andre): Thu 12:30-1:20pm in Sherrerd H 001
Precept 03 (David): Thu 9:00-9:50am in Friend 110

Announcements

Linear Algebra

Week 1: 4-8 September

Assignment 1 Out
Assignment 2 Out

Course Logistics

[required] Reading: Course Syllabus

Vectors

[required] Reading: MML 2.0
[optional] Lecture Video: Vector Basics (15:03)
[optional] Video: 3Blue1Brown on vectors

Matrices

[required] Reading: MML 2.2
[optional] Lecture Video: Matrix Basics (18:44)
[optional] Video: 3Blue1Brown on matrix multiplication

Week 2: 11-15 September

Assignment 1 Due 6pm ET Wednesday 13 Sept 2023
Assignment 3 Out

Systems of Linear Equations

[required] Reading: MML 2.1
[required] Reading: MML 2.3
[optional] Lecture Video: Solving Linear Systems (36:44)

Matrix Inversion

[optional] Blog post: Don't invert that matrix.
[optional] Lecture Video: Matrix Inversion (25:06)

Vector Spaces

[required] MML 2.4
[optional] Lecture Video: Vector Spaces (16:11)

Week 3: 18-22 September

Assignment 2 Due 6pm ET Wednesday 20 Sept 2023
Assignment 4 Out

Linear Independence, Basis and Rank

[required] MML 2.5-2.6
[optional] 3Blue1Brown video on basis vectors
[optional] Lecture Video: Linear Independence, Basis, and Rank (14:00)

Linear Maps

[optional] Lecture Video: Linear maps (9:35)
[required] MML 2.7-2.8
[optional] 3Blue1Brown video on linear transformations and matrices

Change of Basis

[optional] Lecture Video: Change of Basis (11:00)
[required] MML 2.7-2.8
[optional] 3Blue1Brown video on change of basis

Norms and Inner Products

[optional] Lecture Video: Norms and Inner Products (18:58)
[required] MML 3.0-3.3

Week 4: 25-29 September

Assignment 3 Due 6pm ET Wednesday 27 Sept 2023
Assignment 5 Out

Orthogonality

[optional] Lecture Video: Orthogonality and Projection (17:16)
[required] MML 3.4-3.8

Gram-Schmidt Orthogonalization

[optional] Lecture Video: Gram-Schmidt Orthogonalization (15:15)
[required] MML 3.4-3.8
[optional] F20 Lecture by Szymon Rusinkiewicz: Gram-Schmidt Orthogonalization

Determinant and Trace

[optional] Lecture Video: Matrix Invariants (12:28)
[required] MML 4.0-4.1
[optional] F20 Lecture by Szymon Rusinkiewicz: Matrix Trace and Invariants
[optional] 3Blue1Brown video on determinants

Week 5: 2-6 October

Assignment 4 Due 6pm ET Wednesday 4 Oct 2023
Assignment 6 Out

Eigenvectors and Eigenvalues

[optional] Lecture Video: Eigenvectors and Eigenvalues (26:10)
[required] MML 4.2
[optional] 3Blue1Brown video on eigenvectors and eigenvalues

Matrix Factorization

[optional] Lecture Video: Modeling Data with Matrix Factorization (12:07)
[required] MML 4.6-4.7
[optional] F20 Lecture by Szymon Rusinkiewicz: LU and Cholesky Decomposition, Part I, Part II

Week 6: 9-13 October

Midterm Exam in class Oct 11
No precept Thurs/Fri.
Assignment 7 Out

Singular Value Decomposition

[optional] Lecture Video: SVD Basics (24:56)
[required] MML 4.5
[optional] F20 Lecture by Szymon Rusinkiewicz: Singular Value Decomposition

Probability

Week 7: 23-27 October

Assignment 5 Due 6pm ET Wednesday25 Oct 2023

Random Variables

[optional] Lecture Video: Why is Probability Important in Machine Learning (6:54)
[optional] Lecture Video: Probability Spaces and Random Variables (7:02)

Probability Density and Mass Functions

[optional] Lecture Video: Probability Density and Mass Functions (6:56)
[optional] Lecture Video: Some Useful Probability Distributions (5:32)
[required] MML 6.0-6.2
[optional] Video: 3Blue1Brown on the Binomial Distribution

Expectations

Week 8: 30 October to 3 November

Assignment 6 Due 6pm ET Wednesday 1 Nov 2023
Assignment 8 Out

Joint Probability

[optional] Lecture Video: Basics of Joint Probability (6:53)
[required] MML 6.3
[optional] Video: 3Blue1Brown on Bayes' Theorem
[optional] Metacademy: Bayesian Machine Learning Roadmap

Independence and Dependence

[optional] Lecture Video: Independence and Dependence (7:43)
[required] MML 6.4

The Gaussian Distribution

[optional] Lecture Video: The Gaussian Distribution (10:13)
[required] MML 6.5

Week 9: 6-10 November

Assignment 7 Due 6pm ET Wednesday8 Nov 2023
Assignment 9/a> Out

Inequalities and Limits

[optional] Lecture Video: Useful Inequalities and Limit Theorems (13:58)

Pseudo-Random Numbers

[required] MML 6.7

Monte Carlo

[required] MML 6.5
[optional] F20 Lecture by Szymon Rusinkiewicz: Monte Carlo Integration

Information Theory

[optional] Lecture Video: Information Theory Basics (16:22)

Optimization

Week 10: 13-17 November

Assignment 8 Due 6pm ET Wednesday 15 Nov 2023
Assignment 10 Out

Differentiation

[optional] Lecture Video: Why is Differentiation Important to Machine Learning? (3:09)
[optional] Lecture Video: Differentiation Basics (4:44)
[optional] Lecture Video: Partial Derivatives (4:17)
[optional] Lecture Video: Best Affine Approximation (5:26)
[required] MML 5.0-5.5
[optional] MML 5.6-5.8
[optional] F20 Lecture by Szymon Rusinkiewicz: Differentiating Vector- and Matrix-Valued Functions

Week 11: 20-22 November (short week)

No lecture Weds, no precept Thurs/Fri.

Optimization Basics

[optional] Lecture Video: Why is the Gradient the Direction of Steepest Ascent? (2:41)
[optional] Lecture Video: Optimization Basics (8:05)
[required] MML 7.0

Week 12: 27 November to 1 December

Assignment 9 Due 6pm ET Wednesday 29 Nov 2023
Assignment 11 Out

Stochastic Gradient Descent

[required] MML 7.1

Automatic Differentiation

[required] MML 5.6

Lagrange Multipliers

[required] MML 7.2

Week 13: 4 December to 8 December

Assignment 10 Due 6pm ET Wednesday 6 Dec 2023

Higher-order Optimization

[optional] Video: Mathematical Monk on Newton's method

Convex Optimization

[required] MML 7.3
[optional] Lecture Video: Convex Optimization (21:32)

Reading Week: 11 December to 15 December

Assignment 11 Due 6pm ET Friday 15 Dec 2023 (Dean's Date)