Thursday, April 27, 2017

The shifted averages of each character class:

Recognition of the character represented in a given image by forming the "dot products" of that image with each of the above shifted averages and saying the one that gives the largest value is our character prediction worked not too badly.

But can we do better? If we call the shifted average images "weights", can we find modified weights that will give us better accuracy in recognition? How? Perhaps we could define a scalar measure of success and then try to maximize it by following the gradient of that measure with respect to the 10x125x100 = 125000 variables under our control (the pixel values of the set of all the weights images).

Exercise: Use a program like Gimp or Inkscape to draw a path of steepest descent (negative gradient of altitude, orthogonal to level sets a.k.a. contours) on the following image starting from the tip of the pointer, in order to "find" a low point in the neighborhood.

We can implement gradient descent in Tensorflow.

Download the Jupyter notebook supplied with this excellent video tutorial by Magnus Erik Hvass Pedersen.

As we watch and discuss the video, each student should ask at least one question via the quiz form.

Excellent explanation by Brandon Rohrer: (the 13 minutes up to discussion of gradient computation that we can skip, then from 21:34 to 23:30).

**Quiz: "rohrer1"** As we watch and discuss the video, each student should ask at least one question
via the quiz form.

This weekend, please carefully watch this tutorial by Brandon Rohrer, and
ask a question to **Quiz: "rohrer2"**.

Download the Jupyter notebook supplied with this excellent video tutorial by Magnus Erik Hvass Pedersen.

**Quiz: "hvass01"** As we watch and discuss the video, each student should ask at least one question
via the quiz form.

Another great tutorial by Magnus Erik Hvass Pedersen <https://www.youtube.com/watch?v=HMcx-zY8JSg>`_. Download the Jupyter notebook supplied with it.