MSRF-Net: A Multi-Scale Residual Fusion Network ... - arXiv

1 1 MSRF-Net: A Multi-Scale Residual FusionNetwork for Biomedical Image SegmentationAbhishek Srivastava, Debesh Jha, Sukalpa Chanda, Umapada Pal, H avard D. Johansen,Dag Johansen, Michael A. Riegler, Sharib Ali, P al HalvorsenAbstract Methods based on convolutional neural networkshave improved the performance of biomedical image segmenta-tion. However, most of these methods cannot efficiently segmentobjects of variable sizes and train on small and biased datasets,which are common in biomedical use cases. While methods existthat incorporate Multi-Scale Fusion approaches to address thechallenges arising with variable sizes, they usually use complexmodels that are more suitable for general semantic segmentationcomputer vision problems.

2 In this paper, we propose a novelarchitecture called MSRF-Net, which is specially designed formedical image segmentation tasks. The proposed MSRF-Netis able to exchange Multi-Scale features of varying receptivefields using a dual- scale dense Fusion block (DSDF). Our DSDF block can exchange information rigorously across two differentresolution scales, and our MSRF sub- Network uses multipleDSDF blocks in sequence to perform Multi-Scale Fusion . Thisallows the preservation of resolution, improved informationflow, and propagation of both high- and low-level features toobtain accurate segmentation maps.

3 The proposed MSRF-Netallows to capture object variabilities and provides improvedresults on different biomedical datasets. Extensive experiments onMSRF-Net demonstrate that the proposed method outperformsmost of the cutting-edge medical image segmentation state-of-the-art methods. MSRF-Net advances the performance onfour publicly available datasets, and also, MSRF-Net is moregeneralizable as compared to state-of-the-art Terms Medical image segmentation, colonoscopy,MSRF-Net, Multi-Scale Fusion , genearalizationI. INTRODUCTIONMEDICAL image segmentation is an essential task inclinical diagnosis. It has been extensively studied bythe medical image analysis community [1] [3].

4 The semanticsegmentation results can help to identify regions-of-interest forlesion assessment, such as polyps in the colon, to inspect ifthey are cancerous and remove them if necessary. Thus, thesegmentation results can help to detect missed lesions, preventdiseases, and improve therapy planning and treatment. Thesignificant challenge in medical imaging is the requirement ofA. Srivastava is with Computer Vision and Pattern Recognition Unit, IndianStatistical Institute, Kolkata, IndiaD. Jha is with SimulaMet, Oslo, Norway and UiT The Arctic Universityof Norway, Troms , Norway (corresponding email: Chanda is with stfold University College, Halden, NorwayU.)

5 Pal is with Indian Statistical Institute, Kolkata, IndiaH. D. Johansen and D. Johansen are with UiT The Arctic University ofNorway, Troms , NorwayM. A. Riegler is with SimulaMet, Oslo, NorwayS. Ali is with the Department of Engineering Science, University of Oxford,and Oxford NIHR Biomedical Research Centre, Oxford, UKP. Halvorsen is with SimulaMet, Oslo, Norway and Oslo MetropolitanUniversity, Oslo, NorwayS. Ali and P. Halvorsen: Shared senior authorshipa large number of high-quality labeled and annotated datasets,which is a key factor in achieving the desired algorithmic goalfor automated medical image manual annotation of the medical dataset is very time-consuming, requires collaborations with experienced medicalexperts, and is costly.

6 During the annotation of the regionsin medical images (for example, polyp in still frames), theguidelines and protocol are set based on which expert performsannotation. However, there might exist discrepancies amongthe experts while considering a particular area in the lesionas cancerous or non-cancerous. Additionally, lack of standardannotation protocols for various imaging modalities and lowimage quality can also influence annotation quality. Otherfactors such as the annotator s attentiveness, a display device,image-annotation software and data misinterpretation due tolightning conditions can also affect the quality of alternative solution to manual image segmentation isan automated computer aided segmentation based decision-making system that can provide a faster, more accurate, andmore reliable solution to transform clinical procedures andimprove patient care.

7 Computer aided diagnosis will reducethe expert s burden and also reduce the overall treatment to the diverse nature of medical-imaging data, computeraided diagnosis based segmentation models must be robust tovariations in imaging the past years, convolutional neural networks (CNNs)based approaches have overcome the limitations of traditionalsegmentation methods [4] in various medical imaging modal-ities such as X-ray, computed tomography (CT), magneticresonance imaging (MRI), endoscopy, wireless capsule en-doscopy, dermatoscopy, and in high-throughput imaging likehistopathology and electron microscopy. Modern semanticand instance segmentation architectures are usually encoder-decoder based networks [5], [6].

8 The success of deep encoder-decoder based CNNs is largely due to their skip connections,which allow propagation of deep, semantically meaningful,and dense feature maps from the encoder Network to the de-coder sub-networks [7], [8]. However, encoder-decoder basedimage segmentation architectures have limitations in optimaldepth and design of the skip connections [9]. The optimaldepth of the architectures can vary from one biomedicalapplication to another. The number of samples in the datasetused in training also contributes to the limitation on thecomplexity of the Network . The design of skip connections aresometimes unnecessarily restrictive, demanding the Fusion ofthe same- scale encoder and decoder feature maps.

9 Moreover, [ ] 16 May 20212traditional CNN methods do not make use of the this paper, we propose a novel medical image segmenta-tion architecture, calledMSRF-Net, which aims to overcomethe discussed limitations. MSRF-Net utilizes a novel dual- scale dense Fusion (DSDF) block that performs dual scalefeature exchange and a sub- Network that exchanges multi -scalefeatures using the DSDF block. The DSDF block takes twodifferent scale inputs and employs a Residual dense block thatexchanges information across different scales after each convo-lutional layer in their corresponding dense blocks. The denselyconnected nature of blocks allows relevant high- and low-levelfeatures to be preserved for the final segmentation map pre-diction.

10 We also propose adding a complimentary gated shapestream that can leverage the combination of high- and low-level features to compute shape boundaries accurately. Themulti- scale information exchange in our Network preservesboth high- and low-resolution feature representations, therebyproducing finer, richer, and spatially accurate segmentationmaps. Further, layers of Residual networks allow redundantDSDF blocks to die out, and only the most relevant extractedfeatures contribute to the predicted segmentation maps. Wehave evaluated the MSRF-Net segmentation model using fourpublicly available biomedical datasets. Results demonstratethat the proposed MSRF-Net outperforms the state-of-the-art(SOTA) segmentation methods on all standard computer visionevaluation main contributions of this work are as following:1) We propose a novel architecture: MSRF-Net, whichis based on a DSDF block that comprises of residualdense connections.