CS 4803A/8803A
Pattern Recongition

Spring 2003

Bunger Henry 360 (where the heck is that?)
Tues. and Thurs. 3:00-4:30

Description
This course introduces techniques for Pattern Recognition. The course presumes a reasonable background in probablility and linear algebra. The syllabus includes basic PR including Bayesian decision and estimation, non-parametric methods, multi-class classifiers, eigenvector and other feature selection methods, and EM techniques. Time permitting we will also cover additional topics of interest including boositng and sequence analysis via HMMs.

Instructor: Aaron Bobick; afb@cc.gatech.edu; CCB 241; (404)894-8591 (never pick it up - email much better...); Office Hours: For now, drop by or send email to schedule an appointment.
Teaching Assistant: Mario Romero; mromero@cc.gatech.edu.; Office hours: Tu & Th 11-12, GVU lab

Course Administrator: Wanda Abbott; wanda@gvu.gatech.edu; GVU Office, CCB 244 CCB; (404)894-0075

General Information

This course will teach you the basic techniques of Pattern Recognition. By the end of the cousre you should be able to implement a pretty standard PR system, and also have enough basis to understand more complex approaches.

The text for the class is Pattern Classification by Duda, Hart, and Stork. This is the second edition of the text, and should be in the bookstore.

For he first time ever, this class will NOT be recorded as an eClass. Previous years notes are The lectures for this class are being captured. To find them go to this eclass page. But, with the help of digital photography, we'll try to capture as many white board shots as possible.

Requirements, Collaboration, and Grading

The grades will be assessed as follows:

Problem Sets (not all PS are created equal)	50%
Mid-term	20%
Final project	20%
Class Participation	10%

There will be 5 or so bi-weekly problem sets that will involve some Matlab and hopefully some thinking. Collaboration on problem sets is encouraged at the "white board interaction" level. That is, share ideas and technical conversation, but write your own code. All problem sets should be in on time. One late problem set is accepted late (but before the next one is due) without excuse. After that, get prior permission.

There will be a "mid-term" somewhere around 2/3 the way through, just to make sure we're all on the same page.

There will be occasional readings assigned that will typically cover technical material that you may be responsible for.

Undergrads and grads will be graded on separate curves; more is expected from a graduate project than an undergraduate project.

Approximate Syllabus

This course is designed to be an senior level or first year or two grad student course covering basic pattern recognition, plus some more modern techniques. The goal is that by the end of the semester you know enough Pattern Recognition to be dangerous: can attempt a real problem but probably need to dig deeper to handle a real example. Also, you will have exposure to some more current approaches that are currently being researched. Should allow you to take a more advanced PR or machine learning course. When possible we will use examples from computer vision, something close to my heart.

[Wk 1-2] Review of probability and some linear algebra
[Wks 2-3] Bayesian Decision making; linear discriminants, separability, multi-class discrimination; quadratic classifiers.
[Wks 4-6] Bayesian estimation; non-parametric estimation; maximum likelihood, MAP
[Wks 7-8] Linear discriminants again; eigen vector analysis,
[Wks 9-10] Clustering, unsupervised learning, K-means and E/M, neural nets.
[Wks 11-12] Other topics: Sequence analysis, HMMs
[Wk 13] Project presentations.

As the classes happen, there will be a table here:

Date	Title	Assignments	Handouts	Captured board shots
January 7	Introduction to Pattern Recognition	Read: DHS Ch1, Appendices A.1,2,4,5	Syllabus (on line)
January 9	Prob Review: Discrete Events, Bayes Rule, TB test			Screen Shots Jan 9
January 14	Prob Review: Continuous Densities, Vectors		PS 1 (on line)
January 16	Prob Review: Expectation, Functions of rv's
January 21	Moments, Gaussians, Intro to Bayes Decision, LRT	Read DHS Chap 2.1-2.6		Screen Shots Jan 21
January 23	Expected Loss, Bayes Risk			Screen Shots Jan 23
January 28	Gaussian Decision Functions		PS 2 (on line)	Screen Shots Jan 28
January 30	Error bounds, ROC			Screen Shots Jan 30
February 4	Noisy features, parameter estimatoin	Read DHS 3.1 - 3.5	Biased variance estimate; Freeman & Brainard (on line)
February 6	ML Parameter estimation/Bayes Param Estimation
February 11	NO CLASS			none
February 13	Quantum Computing Lecture			none
February 18	Bayesian Param Estmation/Loss function approach	Really read Freeman & Brainard
February 20	Finish Freeman and Brainard; Bayes Class Estimation		PS 3 (on line)
February 25	PCA			Mario Notes Feb 25
February 27	Fisher MDF		Turk and Pentland (on line and paper)	Mario Notes Feb 27
March 11	Eigen faces and Start Non-parametric	DHS 4.1-4.4		Mario Notes March 11
March 13	More Non-parametric Estimation	DHS 4.5	PS 4 (on line)	Mario Notes March 13
March 18	NO CLASS			none
March 20	K-NN estimation and K-NN rule			Mario Notes March 20
March 27	Linear Discriminant Functions	DHS 5.1-5.4		Mario Notes March 27
April 1	Perceptrons		PS 5 (on line)	Mario Notes April 1
April 3				Mario Notes April 3

Some extra readings

There will be additional required readings. When they are available electronically you will be able to find them here:

Here is a PDF of how the ML estimate of variance is not unbiased.
Freeman and Brainard, Bayesian Decision Theory, the Maximum Local Mass Estimate, and Color Constancy, PDF
Turk and Pentland, "Eigenfaces for Recognitoin", CVPR 1991, PDF,

Problem sets

Problem Set 1: Handed out Jan 14, Due January 23, PDF version

Problem Set 2: Handed out Jan 28 Due Feb 11 The PDF file is here. The data set of question 4 is here but since it's binary data you need to right click (or ctl-click for you die hard mac types) and select "save Link target as..." or "Download link to disk".

Problem Set 3: Handed out Feb 20th, Due February 28th (Friday - but extension granted until Monday after Spring Break, March 10) PDF version

Problem Set 4: Handed out Mar 13 Due March 27th. (Warning: Long problem set!) The PDF is here. The data sets of question 1 are training and testing but since it's binary data you need to right click (or ctl-click for you die hard mac types) and select "Download link to disk". The ascii data sets for problem 3 are class1, class2, and class3. These can just be downloaded to a local ascii copy.

Problem Set 5: Handed out April 1 Due April 10th. The PDF is here. The two class data set are classes classA and classB in percept2.mat and the three class data (class1, class2, and class3) are in percept3.mat.

Project data

For those without their own project data, here are some to play with. All these files are ASCII (in case not using MATLAB) so you should just down load them and then load with a command such as LOAD easy.ascii The files are the easy data, some slightly harder data, and labels. The data are 4 rows of 40 dimensional data for each of 25 subjects (data matrices are 100 rows by 40 columns). See how well you do by leave one out, or some other test.

Contact Information:

Aaron Bobick

afb@cc.gatech.edu

College of Computing

Georgia Institute of Technology

Atlanta, GA 30332-0280

Tel: 404-894-8591

email: afb@cc.gatech.edu

CS 4803A/8803A Pattern Recongition