The main assignment in this course is presentation of original research. Examples of projects undertaken by groups of students in the Spring of 2003 are given in the abstracts and reports below. The information is given for the benefit of students wishing to take the course in the future. Please do not cite the reports as literature, and contact the students directly for more information and pointers to publications resulting from this work.
Support Vector Machines for Segmental Minimum Bayes Risk Decoding
Veera Venkataramani and
Sourin Das
Segmental Minimum Bayes Risk (SMBR) Decoding is an approach whereby we use a decoding criterion that is closely matched to the evaluation criterion (Word Error Rate) for speech recognition. This involves the refinement of the search space into manageable confusion sets (ie, smaller sets of confusable words). We propose using Support Vector Machines (SVMs) as a discriminative model in the refined search space. The hope is that we will be able to use SVMs effectively when the search problem is broken down into sequence of independent and simpler problems. Our first approach will be to use SVMs to make hard decisions, ie, the SVMs will output a word for each confusion set.
Kernel-Based Spike Sorting
Jacob Vogelstein and
Kartik Murari
The objective of this project is spike sorting of neural data recorded from a single electrode, by kernel-based classification techniques. Algorithmic issues part of the study include: missing labels in SVM training that are filled in using expectation-maximization, and non-stationarity in spike characteristics that are accounted for by gradually discounting training data over time.
The neural spike data are courtesy of Prof. Wang (BME) and Prof. Johnson (MBI).
Invariance by Incrementals
Edward Choi and
Udayan Mallik
Support Vector Machines (SVMs) using Kernel learning methods have been used effectively for many pattern recognition tasks. One way to make SVMs more effective classifiers is to incorporate prior knowledge about the classification task into the learning process. Invariances are one example of prior knowledge.
Three methods have been used effectively to use invariances in classification (Scholkopf and Smola, Learning with Kernels, MIT Press, 2002). Invariances may be used to transform training examples to create virtual examples, thus expanding the space on which the learning algorithm operates, or alternatively, knowledge about invariance may be used to change the learning algorithm itself to encode the necessary invariances into the error function. Third, one may exploit invariance in a problem by mapping the data into a representation which makes use of invariant properties of the problem space.
A slightly modified approach has been proposed, which uses development in incremental learning methodology in conjunction with creating virtual examples via invariant transforms in order to utilize invariance in classification tasks. Thus the computational cost of the virtual-examples approach due to an enlarged training set can be weighted against the required level of invariance necessary for the classification task. As learning is on-line, at any intermediate stage the SVM will have a degree of accuracy respective to invariance. This depends upon the duration of time spent training on virtual examples, which in this approach corresponds directly to the amount of virtual examples generated by invariance-motivated transformations. The investigation of this approach is the topic of this study.