Fall 2019

# Day 22 = Day -7

Monday, November 11

## Neural networks

Engineering good features can a lot of work! Can we avoid doing it?

A network with a single fully-connected layer.

A linear function.

What weights should we use (at least start with)?

Exercise: let's see how we do by applying argmax to the output of the network.

### Some code from last week to let us get started quickly

from PIL import Image
from glob import glob
import matplotlib.pyplot as plt
%matplotlib inline
import numpy as np

def charclass(png):     # extract character class name from file name
return png.split('__')[1][:-4]

pngs = sorted( glob('pngs/*.png') )
h,w,_ = np.array(Image.open(pngs[0])).shape


The training images: zip file of your handwriting (no changes since last Wed).

How can we improve the weights? Would like a continuous, even differentiable, "loss" function.

### Softmax and cross-entropy

Exercise: compute partial derivatives of softmax function.