Siamese Neural Networks for One-shot Image Recognition

1 Siamese Neural Networks for One-shot Image RecognitionGregory of computer Science, University of Toronto. Toronto, Ontario, process of learning good features for ma-chine learning applications can be very compu-tationally expensive and may prove difficult incases where little data is available. A prototyp-ical example of this is theone-shot learningset-ting, in which we must correctly make predic-tions given only a single example of each newclass. In this paper, we explore a method forlearningsiamese Neural networkswhich employa unique structure to naturally rank similarity be-tween inputs. Once a network has been tuned,we can then capitalize on powerful discrimina-tive features to generalize the predictive power ofthe network not just to new data, but to entirelynew classes from unknown distributions.

2 Using aconvolutional architecture, we are able to achievestrong results which exceed those of other deeplearning models with near state-of-the-art perfor-mance on One-shot classification exhibit a strong ability to acquire and recognizenew patterns. In particular, we observe that when presentedwith stimuli, people seem to be able to understand newconcepts quickly and then recognize variations on theseconcepts in future percepts (Lake et al., 2011). Machinelearning has been successfully used to achieve state-of-the-art performance in a variety of applications such asweb search, spam detection, caption generation, and speechand Image Recognition . However, these algorithms oftenbreak down when forced to make predictions about data forwhich little supervised information is available.

3 We desireto generalize to these unfamiliar categories without neces-sitating extensive retraining which may be either expensiveor impossible due to limited data or in an online predictionsetting, such as web of the32ndInternational Conference on MachineLearning, Lille, France, 2015. JMLR: W&CP volume 37. Copy-right 2015 by the author(s).Figure of a 20-way One-shot classification task usingthe Omniglot dataset. The lone test Image is shown above the gridof 20 images representing the possible unseen classes that we canchoose for the test Image . These 20 images are our only knownexamples of each of those particularly interesting task is classification under therestriction that we may only observe a single example ofeach possible class before making a prediction about a testinstance.

4 This is calledone-shot learningand it is the pri-mary focus of our model presented in this work (Fei-Feiet al., 2006; Lake et al., 2011). This should be distinguishedfromzero-shot learning, in which the model cannot lookat any examples from the target classes (Palatucci et al.,2009). One-shot learning can be directly addressed by develop-ing domain-specific features or inference procedures whichpossess highly discriminative properties for the target a result, systems which incorporate these methods tendto excel at similar instances but fail to offer robust solutionsthat may be applied to other types of problems. In this pa-per, we present a novel approach which limits assumptionson the structure of the inputs while automatically acquir-ing features which enable the model to generalize success-fully from few examples.

5 We build upon the deep learn- Siamese Neural Networks for One-shot Image RecognitionFigure general strategy. 1) Train a model to discriminatebetween a collection of same/different pairs. 2) Generalize toevaluate new categories based on learned feature mappings framework, which uses many layers of non-linearitiesto capture invariances to transformation in the input space,usually by leveraging a model with many parameters andthen using a large amount of data to prevent overfitting(Bengio, 2009; Hinton et al., 2006). These features arevery powerful because we are able to learn them withoutimposing strong priors, although the cost of the learningalgorithm itself may be ApproachIn general, we learn Image representations via a supervisedmetric-based approach with Siamese Neural Networks , thenreuse that network s features for One-shot learning withoutany our experiments, we restrict our attention to characterrecognition, although the basic approach can be replicatedfor almost any modality (Figure 2).

6 For this domain, weemploy large Siamese convolutional Neural Networks whicha)are capable of learning generic Image features usefulfor making predictions about unknown class distributionseven when very few examples from these new distribu-tions are available;b)are easily trained using standardoptimization techniques on pairs sampled from the sourcedata; andc)provide a competitive approach that does notrely upon domain-specific knowledge by instead exploitingdeep learning develop a model for One-shot Image classification, weaim to first learn a Neural network that can discriminatebetween the class-identity of Image pairs, which is thestandardverificationtask for Image Recognition . We hy-pothesize that Networks which do well at at verificationshould generalize to One-shot classification.

7 The verifica-tion model learns to identify input pairs according to theprobability that they belong to the same class or differ-ent classes. This model can then be used to evaluate newimages, exactly one per novel class, in a pairwise manneragainst the test Image . The pairing with the highest scoreaccording to the verification network is then awarded thehighest probability for the One-shot task. If the featureslearned by the verification model are sufficient to confirmor deny the identity of characters from one set of alpha-bets, then they ought to be sufficient for other alphabets,provided that the model has been exposed to a variety ofalphabets to encourage variance amongst the learned Related WorkOverall, research into One-shot learning algorithms is fairlyimmature and has received limited attention by the machinelearning community.

8 There are nevertheless a few key linesof work which precede this seminal work towards One-shot learning dates back tothe early 2000 s with work by Li Fei-Fei et al. The au-thors developed a variational Bayesian framework for One-shot Image classification using the premise that previouslylearned classes can be leveraged to help forecast futureones when very few examples are available from a givenclass (Fe-Fei et al., 2003; Fei-Fei et al., 2006). More re-cently, Lake et al. approached the problem of one-shotlearning from the point of view of cognitive science, ad-dressing One-shot learning for character Recognition witha method called Hierarchical Bayesian Program Learning(HBPL) (2013). In a series of several papers, the authorsmodeled the process of drawing characters generatively todecompose the Image into small pieces (Lake et al.)

9 , 2011;2012). The goal of HBPL is to determine a structural ex-planation for the observed pixels. However, inference un-der HBPL is difficult since the joint parameter space is verylarge, leading to an intractable integration researchers have considered other modalities ortransfer learning approaches. Lake et al. have some veryrecent work which uses a generative Hierarchical Hid-den Markov model for speech primitives combined witha Bayesian inference procedure to recognize new words byunknown speakers (2014). Maas and Kemp have some ofthe only published work using Bayesian Networks to pre-dict attributes for Ellis Island passenger data (2009). Wuand Dennis address One-shot learning in the context of pathplanning algorithms for robotic actuation (2012).

10 Lim fo-cuses on how to borrow examples from other classes inthe training set by adapting a measure of how much eachcategory should be weighted by each training exemplar inthe loss function (2012). This idea can be useful for dataSiamese Neural Networks for One-shot Image RecognitionFigure simple 2 hidden layer Siamese network for binaryclassification with logistic predictionp. The structure of the net-work is replicated across the top and bottom sections to form twinnetworks, with shared weight matrices at each where very few examples exist for some classes, pro-viding a flexible and continuous means of incorporatinginter-class information into the Deep Siamese Networks for ImageVerificationSiamese nets were first introduced in the early 1990s byBromley and LeCun to solve signature verification as animage matching problem (Bromley et al.)