As a part of the Deep Learning Seminar course, we had to train a deep neural network to predict the year in which a yearbook photograph was taken. We fine-tuned VGGNet to do this both using standard cross-entropy loss (classification loss) and a linear combination of cross-entropy and L1 loss. We also attempted to visualize pixels most relevant for classification and the network's view of each class.

The huge improvements in the performance of various NLP tasks using word, and more recently, sentence embeddings prompts one to attack any problem that has not yet been shown to benefit using these methods. In our class project for NLP, we attempted to perform text summarization by clustering of sentence vectors. We experimented with different choices of embeddings and different techniques for selecting a cluster representative. We outperformed a simple baseline but could not match state-of-the-art performance.

Visual Question Answering is the task where a machine is given an image, and a question in natural language based on it, and is expected to answer the question in natural language. A simple neural baseline for this task uses a CNN to encode the image, an LSTM to encode the question and uses weights over these vectors to perform a classification over the 1000 most common answers. We extended this baseline to take in two additional cues - one was a bounding box retrieved when the question is passed as a query to a natural language object retrieval pipeline. The second was a region in the image that humans would typically find salient, and hence would be likely to ask questions about. We improved performance over the baseline but could not meet the state-of-the-art.

If we want robots to directly read a tutorial from the web and follow the instructions, it must be able to abstract away the nuances of language in each individual tutorial and store the instructions in a common format. We wished to discover whether it was possible learn such a format in an unsupervised manner given only tutorials corresponding to the same task. To this end, we attempted to use Hidden Markov Models to model cooking recipes. We experimented with different possible language models to act as observation probabilities but could not identify a good model.

Over the last few years, the focus of research in Recommender Systems has shifted from simply predicting ratings accurately to more holistic metrics that examine other aspects of the recommendation process. For my senior thesis under Dr. B. Ravindran, I attempted to improve the performance of recommender systems on their aggregate diversity, which examines the overall number of items in the inventory that the system manages to recommend and the relative number of times different itms get recommended. I first examined this problem as a course project for my data mining course, when I developed a heuristic solution to the problem that improved aggregate diversity by exploiting additional availabel context information such as user demographics. In my senior thesis, I identified the deficiencies of metrics that would intuitively be chosen to measure the aggregate diversity of a recommender system, attempted to identify a metric that overcame these defects and develop algorithms to optimize the same.

For the project in our Reinforcement Learning course, we attempted to use Reinforcement Learning techniques to exploit structural information when performing coreference resolution, as opposed to modelling it as a classification task. The coreference chain identified so far can be seen as a partial clustering. Clustering evaluation mechanisms can then be used to generate a reward which could be used by a Reinforcement Learning algorithm to learn generalizable rules for coreference resolution and can be interpreted as a metric learning task. We compared the performance of a number of standrd RL algorithms and were able to improve precision on the task over the baseline.

As a part of the Natural Language Processing course, we tried to improve upon techniques that make use of Latent Semantic Analysis for text summarization. This was done by attempting to use standard relevance scores, that determine the importance of a sentence to the summary of the document obtained using LSA, discounted by the similarity of the sentence in question to already selected sentences, much like what Maximal Marginal Relevance does. We also looked at different vector spaces in which sentences could be represented such as binary word vectors, TF.IDF. word vectors and term frequency word vectors weighted by unigram probabilities of words.

This was my project for the social network analysis course. We were exploring how a graphical representation of a dataset could be exploited to identify useful points for active learning. In active learning, the system identifies difficult data points, which it requests a human to label. We created an induced hypergraph for a normal relational database by grouping points into hyperedges based on attribute values. We hypothesized that influencers in this structure would make good choices for training points as label propagation techniques could then be used to learn other labels from them and experimented with different techniques of identifying influencers and propagation.

A different solution to a classic problem - we designed a spell checker that makes use of a modified version of Jaccard similarity of character bigrams of words, rather than edit distance to rank corrections. The system can also make use of additional context when surrounding words in the text are provided.

Wait-freedom is the strongest non-blocking guarantee of progress in a concurrent system. During our concurrent programming course, we tried to design a concurrent binary search tree that provides this guarantee. The main challenge in this was to identify all possible interleavings of operations from different threads that could result in a loss of correctness and incorporate a preventive for attaining it by a mechanism that did not involve waiting. We ended up with a design that, as far as we could make out, was theoretically wait-free except for a periodic cleanup operation, but spawned a very large number of threads to create a monitoring mechanism.