MTH 448/563 Data-Oriented Computing

Fall 2019

Day 23 = Day -6

Wednesday, November 13

Testing our first classifier

Copy and paste this starter code into a Jupyter notebook: handwritten_character_classifier_starter.py.txt.

Exercise: look at some of the errors.

Exercise: test it on these images: handwriting_f19_test/testpngs.zip.

Improving our classifier

How can we improve the weights? To do that, it would be useful to have a continuous, even differentiable, measure of how bad the classifier is.

Softmax and cross-entropy

How to make something usable out of the score array?

Exercise: compute partial derivatives of softmax function.

loss function

Gradient descent

Copy and paste this starter code into a Jupyter notebook: gradient_descent_starter.py.txt.

Back-propagation of gradient

Back-propagation of gradient in our single-layer network

Summary of notation:

c       number of classes
j,m     general class indices
n       number of training images
i       general image index
hw      number of pixels
q       general pixel index
WT     array of weights (c by hw)
X       array of flattened images (hw by n)
S       array of image class scores (c by n)  = WTX
P       array of image class "probabilities" (output of softmax) (c by n)
l       array of individual image losses (length n)
L       overall (scalar) loss