Machine Learning School

M9-AUTODIFF-ANN: Introduction to Neural Networks and Automatic Differentiation

11h 0min
Artificial neuron and its function, linear separability, activation functions; artificial neural networks, types of architectures; neural networks for regression and classification; how to compute the gradients

Main Content
Acquisition
1 Lecture content
Content: – Artificial neural networks can be trained using gradient descent; – Artificial neuron, activation functions; – What the artificial neuron does + linear separability, ... – Multiple layers of neurons and universal approximation; – Feed-forward/recurrent, layered/non-layered architectures; – Neural networks for classification and regression; – How to compute the gradients: autodiff; – Motivation: autodiff vs. symbolic and numeric differentiation; – Autodiff: the principle + graphical illustrations; – Backprop through common operations (graphically): – Defining new operations, incl. the caching of intermediate results; – Autodiff: a numeric example;

2h 0min
Practice
2 Colab Notebooks
A set of colab notebooks, regarding especially these topics: – Autodiff: an illustrative visual notebook; – Also contains the definition of new operations and caching of intermediate results; – Autodiff vs. symbolic vs. numeric differentiation; – Classification and regression using a multi-layered perceptron; – ...

3h 0min
0
Investigation
3 Independent study time + review
The estimated additional time required for studying the material independently, using the lecture videos/slides and also referencing other literature and material, as necessary. Facilitates correct understanding of the material. This activity also includes the time required for review before exams.

5h 30min
Assessment
4 Quiz activities
Quiz activities meant to provide quick, unassessed feedback to students regarding their grasp of the material.

30 min