Fall 2019

# Day 26 = Day -3

Monday, November 25

# over-fitting

The loss on our training set can continue to get smaller without actual improvement in performance.

What's happening and how can we fix it?

## regularization

example: L2 penalty on W

## drop-out

ignore random selections of nodes during learning

# efficiency

Can be more efficient to use just a subset ("mini-batch") of the inputs (images) for each step.

epoch = one cycle through all the mini-batches

# Tensorflow

Simplification of network construction and automatic computation of gradient.

Reproduction of our "from-scratch" network in Tensorflow (2.0)

# Convolutional network layers

A gentle introduction by Brandon Rohrer on Youtube

Convolution is a misnomer - actually cross-correlation (though they are simply related)

## 1D

1D cross-correlation

(g*f)(τ) =  − ∞g(t − τ)f(t)dt

Exercise: on paper.

## 2D

natural generalization to functions of 2 variables. Examples: images!

Application: "scantron" code I wrote for grading multiple-choice exams, now being used in MTH 311

2D example. Copy and paste this: 2d_cross-correlation.py, and here is the image maple_leaf_small.jpg (though you can use any grayscale image you like).

# Regression problem

SUBJECT OF REPORT 6

Prediction of continuum-valued variables rather than categorical, for example, the positions of the pupils in images like this:

Code to draw a collection of examples like the one below: draw_one_eye.py

Acknowledgments: student Edwin Medrano, Adrian Rosebrock.

A lot of the needed code: one_eye_starter.py (copy and paste to notebook).

## How to assess accuracy

Classification problem: confusion matrix gives more detail than overall accuracy

Regression problem: is there an analog of the confusion matrix?