# Day 22

Thursday, April 27, 2017

# From last time ...

The shifted averages of each character class:

Recognition of the character represented in a given image by forming the "dot products" of that image with each of the above shifted averages and saying the one that gives the largest value is our character prediction worked not too badly.

But can we do better? If we call the shifted average images "weights", can we find modified weights that will give us better accuracy in recognition? How? Perhaps we could define a scalar measure of success and then try to maximize it by following the gradient of that measure with respect to the 10x125x100 = 125000 variables under our control (the pixel values of the set of all the weights images).

## A measure of error for our recognition method, and gradient descent

### softmax

Exercise: Use a program like Gimp or Inkscape to draw a path of steepest descent (negative gradient of altitude, orthogonal to level sets a.k.a. contours) on the following image starting from the tip of the pointer, in order to "find" a low point in the neighborhood.

We can implement gradient descent in Tensorflow.

## Gradient descent (or steepest descent) in TensorFlow

Download the Jupyter notebook supplied with this excellent video tutorial by Magnus Erik Hvass Pedersen.

## Quiz: "hvass01"

As we watch and discuss the video, each student should ask at least one question via the quiz form.

# For next time:

## Recognition by multilayer networks: deep learning I

Excellent explanation by Brandon Rohrer: (the 13 minutes up to discussion of gradient computation that we can skip, then from 21:34 to 23:30).

Quiz: "rohrer1" As we watch and discuss the video, each student should ask at least one question via the quiz form.

## Convolutional neural networks

This weekend, please carefully watch this tutorial by Brandon Rohrer, and ask a question to Quiz: "rohrer2".

Download the Jupyter notebook supplied with this excellent video tutorial by Magnus Erik Hvass Pedersen.

Quiz: "hvass01" As we watch and discuss the video, each student should ask at least one question via the quiz form.