ECSE6110 Pattern Recognition
Term: Spring 2018
Instructor: Dr. Chengjiang Long
Time: Tuesday and Friday, 2:00pm – 3:20pm
Building/Room: JEC 4107
Office Hour: Tuesday and Friday 3:20pm—4:00pm by appointment
Office Hour Location: JEC 6045
Course Assistant: IIYoung Son
Course Website: www.chengjianglong.com/teachings.html
Course Overview: This course will give an introduction to the large and diverse field of pattern recognition. Topics include Introduction to Pattern Recognition, Probability theory and Linear algebra review, Basic graph concepts and Belief Network, Bayesian decision theory, Maximumlikelihood estimation, Bayesian methods, Nonparametric techniques, Principal Component Analysis, Fisher Discrimination,
Linear models for regression, Linear models for classification and support vector machines, Bagging, Random Forests and Boosting, Introduction to Neural Networks and Multilayer Neural Network,
Introduction to deep learning: Deep Feedforward Network and Convolutional Neural Network, Unsupervised Learning and clustering. On completion of the course, students should be sufficiently familiar with the formal theoretical structure, notation, and vocabulary of pattern recognition to be able to read and understand current technical literature. They will also have experience in the design and implementation of pattern recognition systems and be able to use those methods to program and solve practical problems.
Prerequisites: Basic probability and statistics, some linear algebra, basic programming skills. Working familiarity with Matlab, Python, Java or C/C++ will be expected.
Text Books: Rechard O. Duda, Peter E. Hart, and David G. Stork, “ Pattern Classification”, 2nd Edition, John Wiley, 2001.
Auxiliary Text Books: Trevor Hastie, Robert Tibshirani and Jerome Friedman, “ The Elements of Statistical Learning”, 2nd Edition, Springer, 2009.
David Barber, “ Bayesian Reasoning and Machine Learning”, Cambridge University Press, 2012.
Ian Goodfellow and Yoshua Beio and Aaron Courville, “ Deep Learning”, MIT Press, 2016.
Grading: The students will be graded based on course participation (5%), four homework assignments (20%, some of them need programming), a midterm exam (20%), a final exam (20%), and a
final project/presentation (35%). Final grade: A 90% to 100%; B80% to 89%; C 60% to 79%; F  < 60%.
Late submission policy: Exponential penalty  late for one day loses half, two day loses another half of the remaining, and so on and so forth.
Topics:
 Introduction to Pattern Recognition.
 Probability theory and Linear algebra review.
 Basic graph concepts and Belief Network.
 Bayesian decision theory.
 Maximumlikelihood estimation.
 Bayesian methods.
 Nonparametric techniques.
 Dimension reduction: Principal Component Analysis;
 Fisher Discrimination.
 Linear models for regression.
 Linear models for classification and support vector machines.
 Bagging, Random Forests and Boosting.
 Introduction to Neural Networks and Multilayer Neural Network
 Introduction to deep learning: Deep Feedforward Network and Convolutional Neural Network.
 Unsupervised learning and clustering.
Course schedule:
Class  Date  Topic  Reading  Homework/Project  Slides 
0  1/16/2018  Short class    
1  1/19/2018  Introduction to Pattern Recognition  Duda Ch 1   Lecture_1 
2  1/23/2018  Probability theory and Linear algebra review  Duda Ch 2   Lecture_2 
3  1/26/2018  Bayesian decision theory  Duda Ch 2.34   Lecture_3 
4  1/30/2018  Bayesian decision theory and Max Likelihood Estimation  Duda Ch 2.56, 3.12, Barber Ch 8   Lecture_4 
5  2/2/2018  Bayesian Esitmation Methods and Naive Bayes Classifier  Duda Ch 3.35, Barber Ch 13  HW 1 assigned  Lecture_5 
6  2/6/2018  NonParametric Methods – Parzen Estimation  Duda Ch 4.13   Lecture_6 
7  2/9/2018  NonParametric Methods – KNN  Duda Ch 4.45  HW 1 due  Lecture_7 
8  2/13/2018  Dimensionality – PCA and Fisher Discrimination  Duda Ch 3.78, Barber Ch 1516  HW 2 assigned  Lecture_8 
9  2/16/2018  Linear Discriminant Functions (1)  Duda Ch 5.12, Barber Ch 17   Lecture_9 
10  2/20/2018  Linear Discriminant Functions (2)  Duda Ch 5.34, Barber Ch 17, HTF Ch 12  HW3 assigned, HW2 due  
11  2/23/2018  Perceptron Classification and Learning  Duda Ch 5.57, HTF Ch 4   
12  2/27/2018  Minimum SquaredError Classification  Duda Ch 5.89  HW3 due  
13  3/2/2018  Bagging; Random Forests; Boosting (1)  HTF Ch. 10 and 15   
14  3/6/2018  Bagging; Random Forests; Boosting (2)  HTF Ch. 10 and 15   
15  3/9/2018  Midterm exam   Midterm exam  
 3/13/2018  Spring break    
 3/16/2018  Spring break    
16  3/20/2018  Presentation of Project Proposals to Class   Final Project Proposal  
17  3/23/2018  Basic graph concepts and Belief Network  Duda Ch. 2.56, 3.10   
18  3/27/2018  Introduction to Nueral Networks  Duda Ch 6.1   
19  3/30/2018  Multilayer Neural Networks  Duda Ch 6.2   
20  4/3/2018  Multilayer Neural Networks: Backpropagation  Duda Ch 6.34   
21  4/6/2018  Multilayer Neural Networks: Implementation  Duda Ch 6.58   
22  4/10/2018  Introduction to Deep Feedforward Nework  Ian Ch 6   
23  4/13/2018  Introduction to Regularization and Optimization  Ian Ch 78   
24  4/17/2018  Introduction to Convolutional Neural Network  Ian Ch 9  HW4 assigned  
25  4/20/2018  Unsupervised learning  Duda Ch 10.15, HTF Ch 14   
26  4/24/2018  Clustering Algorithms  Duda Ch 10.67  HW4 due  
27  4/27/2018  Hierarchical and Online Clustering  Duda Ch 10.914   
28  5/1/2018  Presentation of Project Reports to Class   Final Project Presentation  
29  TBA  Final exam   Final exam  
