Introduction

The course will provide an overview of fundamental concepts and algorithms in machine learning. Mathematical and coding exercises will be provided to deepen understanding of these concepts and algorithms. The topics include:

Regression/Classification Basics: Linear regression, logistic regression, probability interpretations, generalized linear models, etc.
Optimization: convexity, (stochastic) gradient descent, momentum method, line search, second-order methods, gradient-free optimizations, etc.
Neural Networks: classic neural network architectures and training techniques using PyTorch.

Learning outcomes

The learning outcomes of my CWM AI/ML with python are:

Learn the fundamental concepts of machine learning
Gain hands-on experience implementing machine learning algorithms to solve real-world problems
Achieve a certain level of theoretical familiarity with machine learning algorithms to prepare for the next stage of study
Improve logic/mathematical thinking capabilities

Prerequisite

There are only optional prerequisites, not mandatory ones, as the course is intended to be self-contained. Relevant programming and math backgrounds will be reviewed on the first day of the course.

A certain level of familiarity with Python is recommended but not mandatory, as it can typically be learned while working on the exercises.
Some math background is helpful: basic concepts in linear algebra, probability, and statistics, such as singular value decomposition, basis, and parameter estimation.

Course format

We will interleave lecture and coding. We will use Google colab as the platform to do exercises.

Instructors

Lab organizer: Dr. Yangchen Pan (Email: yangchen.eng.ox@gmail.com)
TAs: Jindong Gu, Zhiyao Luo, Kevin Sun

Course Policy

Grading

Naming rules: for example, for exercise1, you may have: firstname-lastname-ex1.ipynb

Email: yangchen.eng.ox@gmail.com. Please make sure the files are named correctly as specified above.

Email subject line: Firstname-Lastname-CWMweek7

Please include the links to access your file in your email, so make sure your file is authorized to view for anyone with the link.

Please submit 9 completed questions, with at least 2 from Part I, 3 from Part II, and 1 from Part III, before 5 PM on June 8th.

Assignments should be completed individually, however, discussion is encouraged.

Attendance

Attendance is mandatory unless there are very special reasons. Any absences will be reported to the student office.
At the end of each day, before leaving, please check with the lab organizer and sign the attendance sheet.

Break

Lunch break: 11:50 am - 1:20 pm
Weds afternoon

Syllabus

We will go through the following content in 5 days. We will use [google colab] as a platform for exercises.

NOTE 1: for each colab file below, you should make a copy in your own google drive to edit & run.

NOTE 2: some questions are “optional”—you are suggested but not required to do.

NOTE 3: please do not work on the exercises below before your registered session officially starts, as there might be significant updates.

Course Intro & background review

Course Introduction
Colab/Markdown/latex intro [env intro]
Coding exercise: Python Basics [Numpy] [Strings] [Lists] [Tuples]
Background knowledge review: [math review]
Quick Introduction to optimization: [opt-notes] [in-lecture exercise]

Part I: Regression/classification linear models

Linear Models Lecture [slides]
Linear regression [Exercise 1] [Exercise 1-conti]
Probability interpretation of regression
Logistic regression [Exercise 2]
Generalized linear models (GLM) [Exercise 3]

Part II: Optimization

Optimisation Lecture [slides]
Non-Stochastic Optimization - Regression [Exercise 4]
Pytorch Tutorial [Tutorial]
Stochastic Optimization - Classification [Exercise 5]

Part III: Neural networks

Neural networks Lecture [slides]
Quick review on object-oriented programming [OOP reading]
Neural Network Numpy [Exercise 6]
Neural Network Pytorch [Exercise 7]
Convolutional Neural Network Pytorch [Exercise 8]