Report 6: Eye-tracking project

Goal: use the convolutional neural network presented in class on Nov 25 to determine the (x,y) location of the center of the pupil in images of eyes.

Base problem: use the single-eye photos, part for training and the remainder for testing.


Essential ingredients

Define and compute some measure of overall accuracy on the test set.

Visual presentation of accuracy: plot predicted x vs. actual x and predicted y vs actual y on the test set. This is the continuum analog of a confusion matrix.

Study of the effect of some network hyper-parameter on performance (either speed or accuracy). Some examples are numbers of filters in each layer, size (or even presence) of fully connected layers, minibatch size, learning rate.

Study of robustness with respect to altering the images. Alterations that could be studied are color of iris or background (skin), size of iris and/or pupil, addition of noise, smudging, etc. I can help with image preparation.

Optional extensions

Can a single fully connected layer do the basic job as well as the convolutional multilayer network? If so, is it as robust to alterations.

Can we get the locations of both pupils in two-eye images like these?:


Here are 2000 such images.