FCNet: A Convolutional Neural Network for …

1 FCNet: A Convolutional Neural Network forCalculating functional connectivity fromfunctional MRIAtif Riaz, Muhammad Asad, S. M. Masudur Rahman Al-Arif, EduardoAlonso, Danai Dima, Philip Corr and Greg SlabaughCity, University of London, of functional brain connectivity patterns us-ing functional MRI has received significant interest in the neuroimag-ing domain. Brain functional connectivity alterations have widely beenexploited for diagnosis and prediction of various brain disorders. Overthe last several years, the research community has made tremendousadvancements in constructing brain functional connectivity from time-series functional MRI signals using computational methods. However,even modern machine learning techniques rely on conventional correla-tion and distance measures as a basic step towards the calculation of thefunctional connectivity .

2 Such measures might not be able to capture thelatent characteristics of raw time-series signals. To overcome this short-coming, we propose a novel Convolutional Neural Network based model,FCNet, that extracts functional connectivity directly from raw fMRItime-series signals. The FCNet consists of a Convolutional Neural networkthat extracts features from time-series signals and a fully connected net-work that computes the similarity between the extracted features in aSiamese architecture. The functional connectivity computed using FCNetis combined with phenotypic information and used to classify individu-als as healthy controls or neurological disorder subjects. Experimentalresults on the publicly available ADHD-200 dataset demonstrate thatthis innovative framework can improve classification accuracy, which in-dicates that the features learnt from FCNet have superior : functional connectivity , CNN, fMRI, Deep IntroductionIn recent literature, functional magnetic resonance imaging (fMRI) has become apopular neuroimaging modality to explore the functional connectivity (FC) pat-terns of the brain.

3 Specifically, the resting state FC has shown to reflect a robustfunctional organization of the brain. Many studies [1 3] have shown promisingoutcomes in the understanding of brain disorders like schizophrenia, attentiondeficit hyperactivity disorder (ADHD) and Alzheimer s disease by studying brain2 Riaz et networks in resting state fMRI. The human brain can be viewed asa large and complicated Network in which the regions are represented as nodesand their connectivity as edges of the Network . FC is viewed as a pair-wiseconnectivity measurement which describes the strength of temporal coherence(co-activity) between the brain regions. A number of recent studies have shownFC as an important biomarker for the identification of different brain disorderslike ADHD [1], schizophrenia [3] and many methods have been developed for extracting the FC from temporalresting state fMRI data such as correlation measures [3], clustering [1] and graphmeasures [2].

4 Most of the existing techniques, including modern machine learn-ing methods like clustering, rely on conventional distance-based measures forcalculating the strength of similarity between brain region signals. These mea-sures act as hand-crafted features towards determining the FC and, may not beable to capture the inherent characteristics of the time-series Convolutional Neural Network (CNN) provides a powerful deep learningmodel which has been shown to outperform existing hand-crafted features basedmethods in a number of domains like image classification, image segmentationand object recognition. The strength of a CNN comes from its representationlearning capabilities, where the most discriminative features are learned duringtraining. A CNN is composed of multiple modules, where each module learnsthe representation from one lower level to a higher, more abstract level.

5 Toour knowledge, CNNs have not been investigated to determine the FC of brainregions. In this work, our motivation is to construct the FC patterns from fMRIdata by exploiting the representation learning capability of a CNN. Particularly,we are interested to determine if a CNN can capture the latent characteristicsof the brain signals. Compared with other methods, our approach calculates theFC directly from pairs of raw time-series fMRI signals, naturally preserving theinherent characteristics of the time-series signal in the constructed training, FCNet requires pairs of fMRI signals and a real value indicatingthe degree of FC. Training data is produced using a generator that selects pairs oftime-series signals that are considered functionally connected, and those that arenot. This data is used to train a Siamese Network [4] architecture to predict FCfrom an input signal pair.

6 We demonstrate the expressive power of the featuresextracted from the FCNet in a classification framework that classifies individualsas healthy control or disorder proposed framework has several stages and is illustrated in Fig 1. Thefirst stage is to train the proposed FCNet using the data generated by a datagenerator (Fig 1a). The FCNet learns to infer the FC between the brain the FCNet is trained, the next step is to use the FCs to distinguish healthycontrol and disorder subjects. This is accomplished by the classification pathways(Fig 1b, c). During training, the fMRI signal from a training subject is fed intothe trained FCNet, which generates a FC map of the brain regions. Then anElastic Net (EN) [5] is used to extract the most discriminative features fromthe FC. The process combines variable shrinkage and grouped feature features are concatenated with phenotypic information to create a finalFCNet3 Training datasetFCNet functional connectivityEN Feature SelectionConcatenation of phenotypic dataSVM Classifier (Training)Test datasetFCNet functional connectivityFeaturesConcatenation of phenotypic dataSVM Classifier (Testing)(predicted)(a) FCNet Training(b) TrainingGeneratorFCNetPairsLabelsPairsLa bels(c) TestingPairs generationPairs generationFig.

7 Of the proposed method. In (a), FCNet is trained from the datagenerated by the generator. In the training pipeline (b), functional connectivity (FC)is generated through FCNet. Next, discriminant features are selected and are con-catenated with phenotypic data, then employed to train a SVM classifier. The testingpipeline is shown in (c). After FC is calculated, features are selected and concatenatedwith phenotypic data. A trained SVM is employed for map. The feature map is used to train a SVM classifier which learns toclassify between healthy control and disorder subjects. Once the classificationpath of Fig 1b is trained, it can be used to classify test subjects as shown in contributions of this work include: 1) a novel CNN-based deep learningmodel for extraction of functional connectivity from raw fMRI signals 2) a learn-able similarity measure for calculation of functional connectivity and 3) improvedclassification accuracy over the state-of-the-art on the ADHD-200 Data and preprocessingThe resting state fMRI data evaluated in this work is from the ADHD-200 con-sortium [6].

8 Different imaging sites contributed to the dataset. The data is com-prised of resting state functional MRI data as well as phenotypic consortium has provided a training dataset, and an independent testingdataset separately for each imaging site. We have used data from three sites:NeuroImage (NI), New York University Medical Center (NYU) and Peking Uni-versity (Peking). All sites have a different number of subjects. Additionally,imaging sites have different scan parameters and equipment, which increases thecomplexity and diversity of the dataset. This data has been preprocessed as partof the connectome project1and brain is parcellated into 90 regions using the au-tomated anatomical labelling atlas [7]. A more detailed description of the dataand pre-processing steps appears on the connectome website. We have integratedphenotypic information of age, gender, verbal IQ, performance IQ and Full4 IQfor NYU and Peking (for NeuroImage, phenotypic information of IQs was notavailable).

9 Et onnb) Feature extractor networkC1C2C3C4C5C1C2C3C4C5 Cross-entropy onn + softmaxc) Similarity networka) FCNetd) LegendsFunctional ConnectivityFig. of the FCNet. (a) FCNet with coupled feature extractor Network (one Network for each brain region) and the similarity Network which measures thedegree of similarity between the two regions. (b) The feature extractor Network whichincludes multiple layers namely Convolutional (Conv), Batch Normalization (B-Norm),Pooling (pool), Fully Connected ( ) and Leaky-ReLU (L-ReLU). (c) The simi-larity measure Network . (d) Legends for feature extractor functional connectivity through FCNetIn this work, we propose a novel deep CNN for the calculation of FC. Ourproposed method calculates FC directly from raw time-series signals instead ofrelying on conventional similarity measures like correlation or distance is a deep- Network architecture for jointly learning a feature extractornetwork that captures the features from the individual regional time-series signaland a learnable similarity Network that calculates similarity between the FCNet is presented in Fig 2 and individual networks are detailed feature extractor Network :This Network extracts features fromindi-vidualbrain region time-series signals and is comprised of multiple layers thatare common in CNN models to learn abstract representations of features.

10 Here,we use a Leaky Rectified Linear Unit (ReLU) as the non-linearity function, dueto its faster convergence over ReLU [8]. The Network accepts time-series signalof length 172. All pooling layers pool spatially with pool length of 2. For allconvolution layers, we use kernel size of 3 and the number of filters are 32, 64,96, 64, 64 for layersC1,C2,C3,C4,C5 respectively. The last fully connectedlayer in the Network has 32 similarity measure Network :This Network employs a Neural networkto learn the FC betweenpairsof extracted features from two brain is in contrast to conventional methods that use hand-crafted computationslike correlation or distance based measures. The input to this Network are theabstracted features extracted from two regions. The Network computes their FC,which relates to the similarity between the two regions.