4005-759-01: Pattern Recognition (RIT CS, 20081)

Department of Computer Science
Pattern Recognition (Topics in Artificial Intelligence, Fall 2008)
 4005-759-01 (Calendar Description)
Home --- Syllabus --- Schedule & Slides --- Assignments --- Resources

Instructor: Richard Zanibbi, Office hours: 2-3:50pm Tuesdays and Thursdays, Room 70-3551 (Golisano College)
Lectures: 10-11:50am Tuesdays and Thursdays, Room 70-2590 (Golisano College)

Project: Handwritten Digit Recognizer

Weight: 20% of final grade

For the course project, students will work in teams to construct a pattern recognizer for handwritten digits. Students will use the MNIST dataset for experiments. This data set contains 60,000 training images, and 10,000 testing images. Yann LeCun, one of the individuals that developed the MNIST dataset was also involved in the creation of algorithms for recognizing the set, perhaps most famously the LeNet5 Convolutional Neural Network.

Students may use any language of their choosing for the project. If you do not have a strong preference, MATLAB is recommended, because it provides a simple, complete environment for implementing algorithms, running experiments, and visualizing results. The course web pages provide links to available libraries for pattern recognition under the "Resources" link; there are others available elsewhere.

Please consult the "Resource" page in the course web pages for materials on carrying out research and writing research papers.

Part I. Proposal

Due: October 2, 2008 (start of class) Weight: 5% of final grade

Each team will provide a proposal for their digit recognizer (maximum 5 pages single-spaced, including references). It must include:

  • A brief description of the problem and data set
  • A brief description of three to five references which present various state-of-the-art techniques for recognizing MNIST. the
  • Problem description, including challenges in recognizing hadnwritten (and particulalthe MNIST digits
  • Features to be used in your system for classification
  • Pattern classification and/or machine learning algorithms to be used in your system
  • A (rough) design for your experiment: what error metrics will you use, what variables will be manipulated, and how will classifiers be compared?

The instructor will use this to check that experiments are well-defined, appropriate, and executable within the quarter. Reports will be graded for clarity, completeness, and correctness. The proposal does not need to describe an implemented system.

Part II. Final Report

Due: October 30, 2008 (start of class) Weight: 15% of final grade

Each team will provide a technical report (maximum 10 pages, including references) summarizing the outcome of their experiment, and comparing their results to published results. The report will include:

  • Problem description, including challenges in recognizing handwritten digits
  • Features, algorithms, error metrics, etc. used for experiments
  • Experimental design ('methods')
  • Results: comparison of error rates for different algorithms, etc., and comparison with other published results
  • Discussion, where you are encouraged to conjecture about possible influences on the results (i.e. you can make claims that you do not have to provide evidence for in this section), and identify further experiments of interest. You do not need a formal conclusion; the discussion will serve this purpose as well.
  • References

The performance of your algorithms is important, but do not worry if your algorithm is not performing as well as or better than the state-of-the-art: you primarily need to make a serious attempt at creating an effective algorithm, and then be able to intelligently discuss the outcome of your experiment. Reports will be graded for clarity, completeness, correctness, and reproducibility: the report should provide enough detail to allow someone working in pattern recognition to repeat the experiment.

You will likely find it helpful to visualize data sets and/or decision boundaries using 2 or 3-dimensional projections, and you should use figures and tables to summarize your results and clarify your presentation.