MTH 448/563 Data-Oriented Computing

Fall 2019

Day 22 = Day -7

Monday, November 11

Neural networks

Engineering good features can a lot of work! Can we avoid doing it?

A network with a single fully-connected layer.

A linear function.

What weights should we use (at least start with)?

Exercise: let's see how we do by applying argmax to the output of the network.

Some code from last week to let us get started quickly

from PIL import Image
from glob import glob
import matplotlib.pyplot as plt
%matplotlib inline
import numpy as np

def charclass(png):     # extract character class name from file name
        return png.split('__')[1][:-4]

pngs = sorted( glob('pngs/*.png') )
h,w,_ = np.array([0])).shape

The training images: zip file of your handwriting (no changes since last Wed).

How can we improve the weights? Would like a continuous, even differentiable, "loss" function.

Softmax and cross-entropy

Exercise: compute partial derivatives of softmax function.