Introduction

The course will provide an overview of fundamental concepts and algorithms in machine learning. Mathematical and coding exercises will be provided to deepen understanding of these concepts and algorithms. The topics include:

Regression/Classification Basics: Linear regression, logistic regression, probability interpretations, generalized linear models, etc.
Optimization: convexity, (stochastic) gradient descent, momentum method, line search, second-order methods, gradient-free optimizations, etc.
Neural Networks: classic neural network architectures and training techniques using PyTorch and TensorFlow.

Course format

We will interleave lecture and coding. We will use Google colab as the platform to do exercises.

Instructors

Lab organizer: Dr. Yangchen Pan (Email: yangchen.eng.ox@gmail.com)
TAs: TBD

Course Policy

Grading

Naming rules: for example, for exercise1, you may have: firstname-lastname-ex1.ipynb

Email: yangchen.eng.ox@gmail.com. Please make sure the files are named correctly as specified above.

Choose three exercise files in total to submit, at least one of them should be from optimization part. Please do not submit anything from the NN part, as the answers were already provided.

Email subject line: Firstname-Lastname-CWMweek7

Please include the links to access your file in your email, so make sure your file is authorized to view for anyone with the link.

Attendance

At the end of each day, before leaving, please ask the instructor or TAs to review your completed exercises. After that, please sign the attendance sheet.

Assignments should be completed individually, however, discussion is encouraged.

Break

Lunch break: 11:50 am - 1:20 pm
Weds afternoon

Syllabus

We will go through the following content in 5 days. We will use [google colab] as a platform for exercises.

NOTE 1: for each colab file below, you should make a copy in your own google drive to edit & run.

NOTE 2: some questions are “optional”—you are suggested but not required to do.

NOTE 3: Students registering for the Week 7 session should not work on the exercises below without prior notice, as there might be significant updates.

Course Intro & background review

Course Introduction
Colab/Markdown/latex intro [env intro]
Coding exercise: Python Basics [Numpy] [Strings] [Lists] [Dictionaries] [Tuples]
Background knowledge review: [math review]

Regression/classification

Linear Models Lecture [slides]
Linear regression, [Exercise 1] [Exercise 1-conti]
Probability interpretation of regression
Logistic regression, [Exercise 2]
Generalized linear models (GLM), [Exercise 3]
Quick review on object-oriented programming [OOP reading]

Neural networks

Slides for Neural networks [PDF]
Neural Network Numpy [Exercise]
Pytorch Tutorial [Tutorial]
Neural Network Pytorch [Exercise] [sol]
Convolutional Neural Network Pytorch [Exercise] [sol]

Optimization

Optimisation Lecture [slides]
Exercise 4 [Non-Stochastic Optimization - Regression]
Exercise 5 [Stochastic Optimization - Classification]