A Simple Feature Augmentation for Domain Generalization

1 A Simple Feature Augmentation for Domain GeneralizationPan Li1 , Da Li2,3 , Wei Li1 Shaogang Gong1, Yanwei Fu4 and Timothy M. Hospedales2,31 Queen Mary University of London2 Samsung AI Center, Cambridge3 University of Edinburgh4 Fudan University{ , , topical Domain Generalization (DG) problem askstrained models to perform well on an unseen target domainwith different data statistics from the source training do-mains. In computer vision, data Augmentation has provenone of the most effective ways of better exploiting the sourcedata to improve Domain Generalization .}

2 However, existingapproaches primarily rely on image-space data augmenta-tion, which requires careful Augmentation design, and pro-vides limited diversity of augmented data. We argue thatfeature Augmentation is a more promising direction for find that an extremely Simple technique of perturbingthe Feature embedding with Gaussian noise during train-ing leads to a classifier with Domain - Generalization perfor-mance comparable to existing state of the art. To modelmore meaningful statistics reflective of cross- Domain vari-ability, we further estimate the full class-conditional featurecovariance matrix iteratively during training.

3 Subsequentjoint stochastic Feature Augmentation provides an effectivedomain randomization method, perturbing features in thedirections of intra-class/cross- Domain variability. We verifyour proposed method on three standard Domain generaliza-tion benchmarks, Digit-DG, VLCS and PACS, and show itis outperforming or comparable to the state of the art in allsetups, together with experimental analysis to illustrate howour method works towards training a robust IntroductionDeep learning methods demonstrate exceptional perfor-mance in different fields of computer vision, such as ob-ject recognition, semantic segmentation or object detection.

4 Equal contributions. Corresponding oneClass twoPerturbed instancesClassifierFigure 1: Illustrative schematic of our stochastic featureaugmentation method. The trained vanilla model has lim-ited robustness as Simple perturbed Feature instances, asmight be encountered experiencing Domain -shift, could in-duce the classifier to make a mistake. During the training,we persistently perturb the Feature embedding, which leadsto classification mistakes. In order to discriminate these er-roneous perturbed instances, the Feature spare must adapt toseparate the classes with a more robust decision new boundary is in turn more robust to , these machine learning systems performancedrops dramatically when encountering test data, which isstatistically different from the training data [5].

5 This issueis known as the Domain shift problem, which Domain gener-alization (DG) research aims to address. Models with goodDG properties are crucial in practical applications since thedistribution of testing data in deployment is inevitably dif-ferent from training data collected for model fitting [17],whether due to either the expense or Simple impossibility ofcollecting representative training research traces back to a decade ago [3]. Since thena variety of methods were proposed to push the DG bound-ary, including learning Domain -invariant features [28, 14],extracting the underlying Domain knowledge [15, 20], and8886meta-learning inspired methods [21, 2, 6, 22].

6 Among exist-ing DG strategies, data Augmentation based approaches [33,38, 44] have become popular. Data Augmentation is alreadywidely used to reduce overfitting in conventional supervisedlearning [18], by inserting predefined class-preserving op-erations, such as transformation, cropping, rotation, flip-ping. Intuitively, augmenting the source Domain data withdiverse samples better representing the breadth of plausibledomains also leads to improved Generalization to novel do-mains, especially when there are only a few known sourcedomains to start with. However, existing Augmentation -based approaches primarily rely on image-space augmen-tations, which are non-trivial to design due to the diffi-culty of specifying- or learning how to synthesize images innew domains.

7 Existing approaches include perturbing in-puts by gradient-descent on the signal from a Domain clas-sifier [33], generating adversarial samples [38] and using animage synthesis network to generate novel images that foola Domain classifier [44]. These approaches are all compu-tationally expensive and complex. In contrast, Feature -leveldata Augmentation have also proven effective recently in asupervised learning context [37]. We are inspired by theseideas to explore Feature -level Augmentation solutions to DG,as shown in Fig. this paper, we first show that an extremely Simple fea-ture Augmentation of perturbing latent features using whiteGaussian noise already leads to comparable performance torecent state-of-the-art.

8 This strategy outperforms the abovementioned highly engineered approaches that rely on train-ing image-to-image generation networks, or gradient-basedadversarial sample generation; while being extremely sim-ple to implement and much faster to run. Nevertheless,while Feature Augmentation helps to enhance the breadthof seen domains for training, a limitation is that one cannot add too much noise without risking inducing a non-class-preserving Augmentation , which then has the counter-productive effect of introducing label-noise. To enable moremeaningful and class-preserving augmentations, we aim toestimate the natural directions of correlation that already ex-ist in a given source dataset.

9 Specifically, we estimate thefeature covariance online during training using moving av-erage and then use it to simulate a joint (multivariate Nor-mal) noise distribution across the features. Estimating class-conditional covariance further ensures that the learned noisefollows the class-preserving but inter- Domain proposed method potentially applies to any base DGmethod. We show that it improves the vanilla method toachieving the state of the art performance on three bench-marks, Digit-DG, VLCS and PACS. Furthermore, we showthe Feature evolution from pretrained Vanilla to our SFAtrained features to understand how the proposed method es-sentially improves model Related Domain GeneralizationVarious methods have been proposed in the DG litera-ture, spanning shallow [15, 28, 14] and deep [20, 27, 21,2, 6, 39, 44, 46] learning methods.

10 Representative workscan be categorized into: Domain invariant/agnostic modellearning [28, 14, 27, 23, 15, 20], meta-learning based DGmethods [21, 2, 6, 22], data Augmentation based DG meth-ods [33, 38, 44], and self-supervision based methods [4, 39].Our work is most relevant to the data Augmentation basedDG methods. [33] designed a special Bayesian architectureand required one-step of back-propagation to generate im-age perturbations. [38] targeted finding the hardest (adver-sarial) samples for the current model and appending theminto the training data, which requires a costly minimax op-timization at each iteration.