ECSE-6110 Pattern Recognition
Term: Spring 2018
Instructor: Dr. Chengjiang Long
Time: Tuesday and Friday, 2:00pm – 3:20pm
Building/Room: JEC 4107
Office Hour: Tuesday and Friday 3:20pm—4:00pm by appointment
Office Hour Location: JEC 6045
Course Assistant: II-Young Son
Course Website: www.chengjianglong.com/teachings.html
This course will give an introduction to the large and diverse field of pattern recognition. Topics include Introduction to Pattern Recognition, Probability theory and Linear algebra review, Basic graph concepts and Belief Network, Bayesian decision theory, Maximum-likelihood estimation, Bayesian methods, Nonparametric techniques, Principal Component Analysis, Fisher Discrimination,
Linear models for regression, Linear models for classification and support vector machines, Bagging, Random Forests and Boosting, Introduction to Neural Networks and Multilayer Neural Network,
Introduction to deep learning: Deep Feedforward Network and Convolutional Neural Network, Unsupervised Learning and clustering. On completion of the course, students should be sufficiently familiar with the formal theoretical structure, notation, and vocabulary of pattern recognition to be able to read and understand current technical literature. They will also have experience in the design and implementation of pattern recognition systems and be able to use those methods to program and solve practical problems.
Basic probability and statistics, some linear algebra, basic programming skills. Working familiarity with Matlab, Python, Java or C/C++ will be expected.
Rechard O. Duda, Peter E. Hart, and David G. Stork, “ Pattern Classification”, 2nd Edition, John Wiley, 2001.
Auxiliary Text Books:
Trevor Hastie, Robert Tibshirani and Jerome Friedman, “ The Elements of Statistical Learning”, 2nd Edition, Springer, 2009.
David Barber, “ Bayesian Reasoning and Machine Learning”, Cambridge University Press, 2012.
Ian Goodfellow and Yoshua Beio and Aaron Courville, “ Deep Learning”, MIT Press, 2016.
The students will be graded based on course participation (5%), four homework assignments (20%, some of them need programming), a midterm exam (20%), a final exam (20%), and a
final project/presentation (35%). Final grade: A-- 90% to 100%; B--80% to 89%; C-- 60% to 79%; F -- < 60%.
Late submission policy:
Exponential penalty -- late for one day loses half, two day loses another half of the remaining, and so on and so forth.
- Introduction to Pattern Recognition.
- Probability theory and Linear algebra review.
- Basic graph concepts and Belief Network.
- Bayesian decision theory.
- Maximum-likelihood estimation.
- Bayesian methods.
- Nonparametric techniques.
- Dimension reduction: Principal Component Analysis;
- Fisher Discrimination.
- Linear models for regression.
- Linear models for classification and support vector machines.
- Bagging, Random Forests and Boosting.
- Introduction to Neural Networks and Multilayer Neural Network
- Introduction to deep learning: Deep Feedforward Network and Convolutional Neural Network.
- Unsupervised learning and clustering.
|1||1/19/2018||Introduction to Pattern Recognition||Duda Ch 1||Lecture_1|
|2||1/23/2018||Probability theory and Linear algebra review||Duda Ch 2||Lecture_2|
|3||1/26/2018||Bayesian decision theory||Duda Ch 2.3-4||Lecture_3|
|4||1/30/2018||Bayesian decision theory and Max Likelihood Estimation||Duda Ch 2.5-6, 3.1-2, Barber Ch 8||Lecture_4|
|5||2/2/2018|| Bayesian Esitmation Methods and Naive Bayes Classifier||Duda Ch 3.3-5, Barber Ch 13||HW 1 assigned||Lecture_5|
|6||2/6/2018||Non-Parametric Methods – Parzen Estimation||Duda Ch 4.1-3||Lecture_6|
|7||2/9/2018||Non-Parametric Methods – KNN||Duda Ch 4.4-5||HW 1 due||Lecture_7|
|8||2/13/2018||Dimensionality – PCA and Fisher Discrimination||Duda Ch 3.7-8, Barber Ch 15-16||HW 2 assigned||Lecture_8|
|9||2/16/2018||Linear Discriminant Functions (1)||Duda Ch 5.1-2, Barber Ch 17||Lecture_9|
|10||2/20/2018||Linear Discriminant Functions (2)||Duda Ch 5.3-4, Barber Ch 17, HTF Ch 12||HW3 assigned, HW2 due|| |
|11||2/23/2018||Perceptron Classification and Learning||Duda Ch 5.5-7, HTF Ch 4|| |
|12||2/27/2018||Minimum Squared-Error Classification||Duda Ch 5.8-9||HW3 due|| |
|13||3/2/2018||Bagging; Random Forests; Boosting (1)||HTF Ch. 10 and 15|| |
|14||3/6/2018||Bagging; Random Forests; Boosting (2)||HTF Ch. 10 and 15|| |
|15||3/9/2018||Midterm exam||Midterm exam|| |
|3/13/2018||Spring break|| |
|3/16/2018||Spring break|| |
|16||3/20/2018||Presentation of Project Proposals to Class||Final Project Proposal|| |
|17||3/23/2018||Basic graph concepts and Belief Network||Duda Ch. 2.5-6, 3.10|| |
|18||3/27/2018||Introduction to Nueral Networks||Duda Ch 6.1|| |
|19||3/30/2018||Multilayer Neural Networks||Duda Ch 6.2|| |
|20||4/3/2018||Multilayer Neural Networks: Backpropagation||Duda Ch 6.3-4|| |
|21||4/6/2018||Multilayer Neural Networks: Implementation||Duda Ch 6.5-8|| |
|22||4/10/2018||Introduction to Deep Feedforward Nework||Ian Ch 6|| |
|23||4/13/2018||Introduction to Regularization and Optimization||Ian Ch 7-8|| |
|24||4/17/2018||Introduction to Convolutional Neural Network||Ian Ch 9||HW4 assigned|| |
|25||4/20/2018||Unsupervised learning||Duda Ch 10.1-5, HTF Ch 14|| |
|26||4/24/2018||Clustering Algorithms||Duda Ch 10.6-7||HW4 due|| |
|27||4/27/2018||Hierarchical and On-line Clustering||Duda Ch 10.9-14|| |
|28||5/1/2018||Presentation of Project Reports to Class||Final Project Presentation|| |
|29||TBA||Final exam||Final exam|| |