Introduction

The course will provide an overview of fundamental concepts and algorithms in machine learning. Mathematical and coding exercises will be provided to deepen understanding of these concepts and algorithms. The topics include:

• Regression/Classification Basics: Linear regression, logistic regression, probability interpretations, generalized linear models, etc.
• Optimization: convexity, (stochastic) gradient descent, momentum method, line search, second-order methods, gradient-free optimizations, etc.
• Neural Networks: classic neural network architectures and training techniques using PyTorch.

Learning outcomes

The learning outcomes of my CWM AI/ML with python are:

• Learn the fundamental concepts of machine learning
• Gain hands-on experience implementing machine learning algorithms to solve real-world problems
• Achieve a certain level of theoretical familiarity with machine learning algorithms to prepare for the next stage of study
• Improve logic/mathematical thinking capabilities

Prerequisite

While there are no formal mandatory prerequisites, the course assumes a solid willingness to engage with programming and mathematically rigorous material.

Prior experience with Python is strongly recommended, as students will be expected to work with programming assignments and computational exercises.

A reasonable mathematical background is also expected, particularly in linear algebra, probability, and statistics.

That said, students from a wide range of backgrounds have successfully completed the course in previous years. Those with less prior experience in programming or mathematics typically needed to invest additional time and effort, but were still able to succeed.

Course format

We will interleave lecture and coding. We will use Google colab as the platform to do exercises.

Instructors

• Lab organizer: Dr. Yangchen Pan (Email: yangchen.eng.ox@gmail.com)
• Senior Demonstrator: Dr. Yosra Mekki
• TAs: Aniq Rahman, Samuel Barnett, Yishun Lu, Yuanzhe Xu.

Course Policy

Grading

Please email me a total of 11 questions (including optional questions, if you wish), with at least one question from each part (Part I: Linear Models, Part II: Optimization, and Part III: Neural Networks). You may submit more than 11 questions, but no bonus marks will be awarded; in that case, please indicate which 11 questions should be graded.

Naming rules: for example, for exercise1, you may have: firstname-lastname-ex1.ipynb

Email: yangchen.eng.ox@gmail.com. Please make sure the files are named correctly as specified above.

Email subject line: Firstname-Lastname-CWMweek6

Please include the links to access your file in your email, so make sure your file is authorized to view for anyone with the link.

Submission deadline: The submission deadline is 5:00 pm on Friday. Submissions received after this time may not be considered. If you require a late submission due to exceptional circumstances, please contact the lab organiser by email.

Assignments should be completed individually, however, discussion is encouraged.

Attendance

1. Attendance is mandatory unless there are very special reasons. Any absences will be approved by the student office.

2. At the end of each day, before leaving, please make sure that you sign the attendance sheet.

Break

1. Lunch break: 12:20 pm - 2:00 pm

2. Weds afternoon

Syllabus

We will go through the following content in 5 days. We will use [google colab] as a platform for exercises.

NOTE 1: for each colab file below, you should make a copy in your own google drive to edit & run.

NOTE 2: some questions are “optional”—you are suggested but not required to do.

NOTE 3: please do not work on the exercises below before your registered session officially starts, as there might be significant updates.

NOTE 4: AI-based coding tools, such as ChatGPT, Copilot, and similar, are strictly forbidden in this lab. To turn it off, once you open a Colab file, go to Tools > Settings > AI Assistance > Hide Generative AI Features, and deselect the other options.

Course Intro & background review

• Course Introduction

• Colab/Markdown/latex intro [env intro]

• Coding exercise: Python Basics [Numpy] [Lists]

• Background knowledge review: [math review]

• Quick Introduction to optimization: [opt-notes]

Part I: Regression/classification linear models

• Linear Models Lecture [slides]

• Linear regression [Exercise 1] [Exercise 1-conti]

• Probability interpretation of regression

• Logistic regression [Exercise 2]

• Generalized linear models (GLM) [Exercise 3]

Part II: Optimization

• Optimisation Lecture [slides]

• Non-Stochastic Optimization - Regression [Exercise 4]

• Pytorch Tutorial [Tutorial]

• Stochastic Optimization - Classification [Exercise 5]

Part III: Neural networks

• Neural networks Lecture [slides]

• Quick review on object-oriented programming [OOP reading]

• Neural Network Numpy [Exercise 6]

• Neural Network Pytorch [Exercise 7]

• Convolutional Neural Network Pytorch [Exercise 8]