Neuro-levelset system based segmentation in dynamic susceptibility contrast enhanced and diffusion weighted magnetic resonance images

Neuro-levelset system based segmentation in dynamic susceptibility contrast enhanced and diffusion weighted magnetic resonance images

Pattern Recognition 45 (2012) 3501–3511 Contents lists available at SciVerse ScienceDirect Pattern Recognition journal homepage: www.elsevier.com/lo...

1MB Sizes 0 Downloads 9 Views

Pattern Recognition 45 (2012) 3501–3511

Contents lists available at SciVerse ScienceDirect

Pattern Recognition journal homepage: www.elsevier.com/locate/pr

Neuro-levelset system based segmentation in dynamic susceptibility contrast enhanced and diffusion weighted magnetic resonance images Vijayakumar Chinnadurai a,n, Gharpure Damayanti Chandrashekhar b a b

Department of Radio diagnosis and Imaging, Armed Forces Medical College, Pune, Maharashtra 411040, India Department of Electronic Science, University of Pune, Pune, Maharashtra, India

a r t i c l e i n f o

abstract

Article history: Received 17 June 2011 Received in revised form 24 November 2011 Accepted 29 February 2012 Available online 9 March 2012

In this study, neuro-levelset method is proposed and evaluated for segmentation and grading of brain tumors on reconstructed images of dynamic susceptibility contrast (DSC) and diffusion weighted (DW) magnetic resonance images. The proposed neuro-levelset method comprises of two independent phases of processing. At first, reconstructed images have been independently processed by three different artificial neural network systems such as multilayer perceptron (MLP), self-organizing map (SOM), and radial basis function (RBF). The images used for these tasks were the cerebral blood volume (CBV), time to peak (TTP), percentage of base at peak (PBP) and apparent diffusion coefficient (ADC) images. This processing step ensued in formation of segmentation images of brain tumors. Further, in the second phase, these coarse segmented images of each artificial neural network system have been independently subjected as speed images to levelset method in order to optimize the segmentation performance. This has resulted in construction of three distinct neuro-levelset methods such as MLP-levelset, SOM-levelset and RBF-levelset method. Proposed neuro-levelset methods performed better in segmenting tumor, edema, necrosis, CSF and normal tissues as compared to independent artificial neural network systems. Among three neuro-levelset methods, RBF-levelset system has performed well with average sensitivity and specificity values of 91.4372.94% and 94.4371.90%, respectively. Crown Copyright & 2012 Published by Elsevier Ltd. All rights reserved.

Keywords: Neuro-levelset method Artificial neural networks Radial basis function Self-organizing map Dynamic contrast susceptibility magnetic resonance images Diffusion weighted images

1. Introduction Magnetic resonance imaging engages a prominent task in diagnosis and grading of brain tumors, and it is now one of the essential modalities for adequate clinical management of many tumor types. However, there are yet many challenges and difficulties in differentiating brain tumors and their pathologies. Inability of distinguishing edema and necrosis from the tumor is one of its primary limitations. In order to circumvent these inadequacies, many advanced techniques have been developed over the years. Some of the most important developments are evolution of newer image segmentation techniques and image acquisition techniques. Dynamic susceptibility contrast (DSC) studies and diffusion-weighted MRI (DWI) studies are recent advances in image acquisition techniques, which depict tumor morphology and relationships of malignant lesions to neighboring structures better. MR diffusion study [1] is sensitive to the molecular diffusion of water, has been well substantiated as a reliable non-invasive

n

Corresponding author. Tel.: þ91 2026332783. E-mail address: [email protected] (V. Chinnadurai).

approach in distinguishing necrotic spaces associated with malignant brain tumor from the benign ones. Contemporary advancements in dynamic susceptibility contrast (DSC) techniques [2,3] have allowed the formation of cerebral blood volume (CBV), time to peak (TTP), percentage of base at peak (PBP) images, which lead to the qualitative and quantitative inspection of tumor vascularity. These images have eased in the assessment of tumor grade and in choosing the right site of biopsy. DWI and DSC in non-enhancing brain tumors offer clinically relevant physiological data, which is otherwise not realizable by conventional T1 and T2 MRI. Non-enhancing brain tumor with lower ADC values in the solid regions and higher CBV ratios in both solid portion and peritumoral regions of tumors are significantly correlated with anaplasia. Therefore, DWI and DSC have been unified for providing better feature information in the automatic image segmentation task of brain tumor in order to predict tumor grading in this study. However, diffusion anisotropy based techniques such as diffusion tensor imaging and fractional anisotropy have not been included due to its least influence in predicting the grades of the brain tumor. Brain tumor segmentation [4] and grading in DSC and diffusion weighted (DW) MR Images are, nevertheless, a challenging issue due to the complicated appearance of tumors in these images and their irregular sizes and boundary. Some brain tumors also distort other structures and appear together with edema that hurdles

0031-3203/$ - see front matter Crown Copyright & 2012 Published by Elsevier Ltd. All rights reserved. doi:10.1016/j.patcog.2012.02.038

3502

V. Chinnadurai, G. Damayanti Chandrashekhar / Pattern Recognition 45 (2012) 3501–3511

intensity property of the neighboring region. In most of the clinical circumstances, segmentation of brain tumor alone does not serve the purpose. Recognizing the grade of the brain tumors along with segmentation plays an important role in surgical, treatment planning. Many researchers have explored various segmentation techniques to overcome these challenges associated with segmentation and grading of brain tumors. Georgiadis et al. [5,6] have analyzed brain MR images with support vector machines with 3D texture analysis and non-linear transformation of textural features. However, their methods were solely limited by contrast information provided by conventional T1 post contrast images. On the other hand, Lee et al. [7] have explored many features information provided by different image acquisition techniques to study brain tumors with k–NN algorithm and genetic algorithm. Nevertheless, their study concentrates primarily high grade tumor such as glioblastoma multiforme and did not attempt to classify other grades of tumors. Corso et al. [8] presented an approach based on bayesian formulation for incorporating soft model assignments into the calculation of affinities. Wels et al. [9] have carried out studies to perform segmentation of brain tumors using pseudo-conditional random fields and discriminative model-constrained graph cuts methods, respectively. Emblem et al. [10] have attempted to utilize perfusion weighted images to grade brain tumor using conventional histogram method. Liu et al. [11] have implemented fuzzy connectedness for segmenting glioblastoma. Kai Xie et al. [12] have developed a semi-automated method which performs brain tumor and edema segmentation. Solomon et al. [13] introduced an automated method using probabilistic reasoning over both space and time to segment brain tumors from 4D spatio-temporal MRI data. Chi-Hoon Lee et al. [14] implemented PCRFs for brain tumor segmentation. Further, Wang et al. [15] developed an approach called FVF active contour model. Emblem et al. [16] used knowledge-based fuzzy c-means clustering on large data set of low and high grade tumors. Nie et al. [17] proposed hidden Markov random field based approach and the study was evaluated on images of 15 patients. Even though all these approaches could carry out the brain tumor segmentation, they were either developed for segmenting specific grades of tumors or were executed on very limited patient population. In addition, their segmentation performances in terms of sensitivity and specificity were also not up to standard that is anticipated in order to perform these techniques in clinical practice every day. Artificial neural network systems have substantiated to be powerful image segmentation techniques [18], which is capable of segmenting the complex regions. This is primary due to its ability to incorporating multiple constraints and finding optimal combinations of constraints [19] for segmentation and hence features need not be treated as independent. In addition, artificial neural networks are highly parallel and regular structures, which make them, especially amenable to high-performance parallel architectures and hardware implementations. On the other hand, level sets have brought the tremendous impact to MR Image segmentation due to its capability of topology preservation [20,21] and fast shape recovery. Baillard et al. [22] presented a level set based strategy for the segmentation of brain from volumetric images, which integrate 3D segmentation and 3D registration processes. Jonasson et al. [23] presented a levelset method for segmenting white matter tracts in high angular resolution diffusion weighted images. Ciofolo et al. [24] proposed a technique to segment 3D structures with competitive level sets driven by fuzzy control. Taheri et al. [25] have introduced a threshold-based scheme that uses level sets for 3D tumor segmentation (TLS). Thus, in this study, a novel attempt has been put forward for simultaneous assessment of grading and segmentation of brain

tumors and their pathology regions using neuro-levelset method, which incorporates the artificial neural network system and levelset method.

2. Methodology In this study, the proposed neuro-levelset method has been carried out in two stages. In the first stage, the reconstructed images of diffusion weighted (DW) [26,27] and dynamic susceptibility contrast (DSC) MR images [27–29] have been analyzed by artificial neural network systems. This is carried out by constituting the feature vectors from reconstructed images and processes them to the artificial neural network as the input for training. These trained neural systems were subsequently allowed to process the images which were not used for training and produced coarse segmented images. The images used for these tasks were the cerebral blood volume (CBV) image, time to peak (TTP) image, percentage of base at peak (PBP) image and apparent diffusion coefficient (ADC) images. CBV, TTP and PBP images were reconstructed from of DSC images and ADC image was reconstructed from DW images. These images have a definite advantage over conventional T1 and T2 weighted MR images due to their ability to get cellular and vascular information of the tumors. Three different artificial neural network systems such as multi layer perceptron (MLP), self-organizing maps (SOM), and radial basis functions (RBF) has been utilized. These neural systems have been chosen due to their ability to generate any kind of nonlinear functional approximation [30,31]. Then, in the second stage, coarse segmented image of the artificial neural network system has been independently subjected to the levelset method to optimize the segmentation quality. The following sections briefly explain the methodology implemented. Overall the schematic block diagram of the methodology implemented is shown in Fig. 1. In this study, World Health Organization (WHO) classification [32] of brain tumor has been followed for grading the tumor. Out of four grades [33], Grade I, Grade II tumors are more often called as low grade tumors and moderate grade tumors, respectively, and Grade III and IV are normally known as high grade tumors. 2.1. Image acquisition The proposed method has been evaluated in the MR images of 40 patients (12 low grades þ17 moderate grades þ11 high grades) who were diagnosed as having glioma in the histopathology investigation. Table 1 shows acquisition parameters of both DSC and DW images. DW images were acquired with magnetic field gradients applied in three orthogonal directions with three different b values such as 0 s/mm2, 500 s/mm2 and 1000 s/mm2 by using echo-planar imaging. For every b value, a separate diffusion image set has been acquired. Analysis of diffusion changes was performed by calculating apparent diffusion coefficient (ADC) image from diffusion images. To perform DSC imaging, a Gd -DTPA contrast agent was administered intravenously to patients through the venous catheter by constant rate infusion. Total dose of contrast applied were 0.1 mmol Gd-DTPA/kg body weight. This contrast injection was followed by 20 ml saline injection at the same flow rate. This administration of contrast has induced perfusion phenomenon in the brain, which has been imaged using 1.5 T MR Machine. For this imaging, a gradient-echo based echo-planar imaging sequence was used with the flip angle of 901. Totally, eight slices were repeatedly acquired during perfusion phase within the brain. Every slice of anatomy has been acquired repeatedly with time interval of 10–20 s, which is finally totaled to 40 images for every slice.

V. Chinnadurai, G. Damayanti Chandrashekhar / Pattern Recognition 45 (2012) 3501–3511

3503

Fig. 1. Overview of neuro-levelset systems implemented.

Table 1 Acquisition parameters of DSC and diffusion weighted images. Type of Image/ Parameters

DSC images

Diffusion weighted

Slice thickness (in mm) Inter slice gap Field of view (FOV) (in mm) Matrix size Repetition time (TR) (in ms) Echo time (TE) (in ms) Pulse sequence Contrast used Number of slices

5 0 230  230 256  256 1440 47 EPI Gd—DTPA 8

5 0 230  230 256  256 3800 107 EPI Nil 8

2.2. Reconstruction of CBV, TTP, PBP and ADC The reconstructed images of both DSC and DW images are illustrated in Fig. 2. At first, the CBV image was derived on a pixelby-pixel basis from DSC images. In order to carry out this calculation, the base line measure for signal intensity (SI0) of every pixel were calculated by choosing a number of baseline points before the starting point of first-pass circulation and averaging them. Then, on a pixel-by-pixel basis, signal intensity (SIt) of every location on DSC images was traced and CBV images were generated by numeric integration of relative concentration of contrast media through each voxel. The PBP image is reconstructed by calculating relative change in signal intensity of voxel of DSC image to basic image prior to contrast administration. Further, TTP image is reconstructed by determining time needed

to reach minimum perfusion signal of every voxel in DSC images. Analysis of diffusion changes is carried out with sets of diffusion weighted images acquired with various b values and ADC images are computed. 2.3. Preprocessing MR images acquired at different MR scanners/hospitals usually delineate the similar anatomical regions with the dissimilar gray level (pixel intensity). The preprocessing operation nullifies these variations and brings pixel intensities to common standard levels. This operation consists of registration, standardization and cranium removal operations. Registration of reconstructed images has been carried out to get pixel to pixel correlation by bringing them all to a common matrix size of 256  256 with field of view of 23 cm  23 cm. This has been carried out by using affine transform registration method [34] that was restricted to rigidbody motion (rotation and translation), combined with isometric scaling. Subsequently, all the images have been standardized to have normalized pixel values to overcome the variation in the pixel values. Since both DSC and diffusion weighted images were acquired with same slice orientation and thickness, standardization technique has been used to bring all the images to the common histogram scale. In order to perform that, intensity standardization method [35] was applied on registered image data. Then, the cranium has been removed in standardized images by employing the threshold method. The threshold method is applied specifically at a peripheral side of images in order to avoid

3504

V. Chinnadurai, G. Damayanti Chandrashekhar / Pattern Recognition 45 (2012) 3501–3511

Fig. 2. Reconstructed images of moderate grade tumor and high grade tumor. Moderate grade tumor: (a) CBV, (b) PBP, (c) TTP, (d) ADC. High grade tumor: (e) CBV, (f) PBP, (g) TTP, (h) ADC.

Fig. 3. Preprocessed images of moderate grade tumor and high grade tumor. Moderate grade tumor: (a) CBV, (b) PBP, (c) TTP, (d) ADC. High grade tumor: (e) CBV, (f) PBP, (g) TTP, (h) ADC.

removal of regions having pixel values of a cranium in the intracranial region. These preprocessed MR images are illustrated in Fig. 3. 2.4. Feature vector formation The preprocessed images that consist of CBV, PBP, TTP and ADC images are subsequently used to organize the feature vector. In this study, bm ¼{fCBV,fPBP,fTTP,fADC} is the feature vectors collected where fCBV,fPBP,fTTP,fADC are individual pixel values collected from the same locations of preprocessed CBV, PBP, TTP and ADC images. In this operation, 700 training feature vectors were collected from seven different regions such as high grade tumor, moderate grade tumor, low grade tumor, necrosis, edema, CSF and normal tissue (whiteþgray matter). These feature vectors were collected from various patient image data and carried out by experienced radiologists. The collected feature vectors were subsequently cross examined by a different radiologist to conform to its reliability.

2.5. Neuro-levelset methods The organized training feature vectors were eventually then used to train the proposed neuro-levelset methods to perform segmentation and grading of brain tumors. As it is mentioned earlier, three distinct artificial neural network systems (MLP, SOM and RBF) were separately used to form speed images to the levelset method. This leads to the formation of three different types of neuro-levelset methods such as MLP-levelset, SOMlevelset and RBF-levelset methods. The formations of speed images such as F MLP , F SOM and F RBF by the neural networks were carried out at first in the stage one, and further they were subjected to the levelset in the second stage for optimization of segmentation. The following section briefly explains the implementation of these algorithms. 2.5.1. Multi layer perceptron At first, back propagation based multilayer perceptron [36,37] is used as a neural block to supply speed image to levelset

V. Chinnadurai, G. Damayanti Chandrashekhar / Pattern Recognition 45 (2012) 3501–3511

method, which is connected next to it. Generally, multilayer perceptron consists of three layers in which the middle layer called hidden layer whose units correspond to the basis function Fk . Usually, hidden units implement the sigmoid function:

Fðbm Þ ¼

1 1 þexpðbm Þ

ð1Þ

Connection weights of both layers, wjk, are trained in a supervised manner by gradient descent methods, the output of MLP system will be, F MLP ¼

d X

wjk Fk ðbm Þ þ wjo

ð2Þ

k¼1

In large sample case, multilayer perceptron estimate posterior probabilities, thus building a link between multilayer networks and statistical classifiers. In the trained MLP network, the output contains one node and hidden layer has been optimized with six hidden nodes with activation functions formed by hyperbolic tangent function. This choice has allowed to have the output whose numerical values would vary between ‘‘ 1’’ and ‘‘ þ1’’ continuously. In order to identify seven classes within those numerical values, every block of 0.28 between  1 and þ1 has been assigned a particular class. Training of MLP was carried out until the error reaches the value of 0.0000001 which, in fact, took 9000 iterations to attain that value. Further, trained MLP were used for generating coarse segmented images based on output values (F MLP ). 2.5.2. Self-organizing map In parallel to the previous method, SOM-levelset method was also implemented by training SOM for generating coarse segmented images and then used them as speed image to levelset technique. The SOM [38] consists of a regular grid of units. Each unit contains a model vector wij A Rq where q is dimensionality of data. The sequential SOM algorithm iterates two steps. The best matching model vector w0ij is sought using the equation w0ij ¼ arg minij :bm wij :

ð3Þ

where :bm wij : is the distance between the input feature vector bm and the model vector wij. When the best matching model vector has been found, the model vectors are updated with

3505

However, feature vector corresponding to necrosis was found to be distributed in various parts of SOM. Mostly, it was found to be occupying the region between CSF and low grade tumor. This may be probably due to similar DSC and diffusion appearance of CSF, low grade tumor and necrosis. After the training, SOM segmentation method based on U-matrix has been implemented. 2.5.3. RBF-levelset method An RBF neural network [40] can be considered as a mapping: Rq -Ro . Let bm A Rq and C j A Rq ð1r j r nÞ be the node centers, which are prototype of input vectors. Then the activity Rj ðbm Þ of the jth node is the Euclidean norm of the difference between the input feature vector bm and the node center C j and is given by: ! ð:bm C j :Þ2 Rj ðbm Þ ¼ exp  ð5Þ 2

sj

where sj is the width of jth node. The kth output Y k ðbm Þ of an RBF network F RBF ¼

n X

Rj ðbm Þ  wðk,jÞ,

k ¼ 1,2,    ,o

ð6Þ

j¼0

where wðk,jÞ is the weight of jth receptive fields to the kth output. During RBF learning process, every input feature vector bm, the associated output F RBF as well as error ek ðbm Þ can be computed. The computation of ek(bm) is ek ðbm Þ ¼ dk ðbm ÞF RBF Þ

ð7Þ

where dkbm is the desired output vector for the input feature vector bm. The goal of RBF network learning is to minimize the following error function, called training error: E ¼

a o X 1 X e2 ðbm Þ 2m¼1k¼1 k

ð8Þ

The determination of hidden node centers Cj and node width

sj is based on the fuzzy c-means (FCM) algorithm [41], while the connection weights are obtained using linear regression. This approach has a number of advantages compared to the standard k-means clustering technique, which is the most popular algorithm for selecting the centers of an RBF network. Trained RBF system was subsequently used for generating the coarse segmented image F RBF at the output node.

b

wij ðt þ 1Þ ¼ wij ðtÞ þ aðtÞhij ðtÞðbm wij ðtÞÞ wij ðt þ 1Þ ¼ wij ðtÞ,

otherwise

ð4Þ b

In the above expression, aðtÞ is the learning rate and hij ðtÞ is the neighborhood function centered on best matching unit. Typically, SOMs are visualized using the U-matrix. Subsequently, the segmentation method based in the U-matrix proposed by Vijayakumar et al. [39] is implemented to form the speed image F SOM for levelset method. In the trained SOM system, a hexagonal grid of dimension 40  20 was found to give optimum clustering. During initial b training phase, neighborhood function (hij ðtÞ) is allowed to cover entire hexagonal grid. This ordering phase was maintained for 5000 iterations to ensure sufficient preordering in SOM grid. After this ordering phase, value of sðtÞ has been reduced uniformly over b next 5000 iterations. This fine tuning phase allowed hij ðtÞ concentrating grid points nearer to best matching unit. As it is evident in Fig. 4, U-matrix of SOM, cluster constituted clearly differentiating best matching units corresponding to different regions such as high grade tumor, moderate grade tumor, low grade tumor, edema, necrosis and normal tissue. This clear differentiation is an indication of SOM ability in identifying different classes from DSC and diffusion image information.

2.5.4. Levelset implementation The coarse segmented images of three neural systems such as F MLP , F SOM and F RBF has been used as the speed image independently with the proposed neuro-levelset method. In the levelset method, a contour is propagated and allowed to recognize the region that requires to be segmented. In levelset method [42–45], given a closed (N  1) dimensional hyper surface gðt ¼ 0Þ, the objective is to produce an euclidian formulation for the motion of gðtÞ propagating along its normal direction with speed Fo. Main idea of levelset methodology is to embed this propagating interface as zero level set of a function c : RN  R-R defined by

cðx,tÞ ¼ dist ðx, gÞ

ð9Þ

where dist ðx, gÞ represents signed distance from point x to front gðtÞ. By definition, points inside initial front are assigned negative distance. The condition that moving front is the zero levelset of the function c is expressed by the equation

gðtÞ ¼ fx9cðx,tÞ ¼ 0g

ð10Þ

The propagation of c is calculated by the equation

ct þF o 9rcðx,tÞ9 ¼ 0 given cðx,t ¼ 0Þ

ð11Þ

3506

V. Chinnadurai, G. Damayanti Chandrashekhar / Pattern Recognition 45 (2012) 3501–3511

Fig. 4. U-matrix visualization of SOM clusters with illustration of best matching units of training pattern vectors. (a–c) Moderate tumor; (d–f) Edema; (g–i) Necrosis.

where ct is the derivative of c with respect to t and 9rc9 is the norm of spatial gradient of c. Equation (11) is Hamilton–Jacobi levelset equation. Evolution of levelset function is determined by speed function Fo. Speed component is formed by combining curvature speed component, which is inversely proportional to the curvature of zero levelset-curve, with the propagation speed component, F, which is the intensity value of the speed image, as follows:

2.5.5. Evaluation In this study, in order to compare ground truth and segmentation results provided by image processing techniques, evaluation parameters [46,47] such as sensitivity, specificity, segmentation accuracy (SA) and correspondence ratio (CR) have been evaluated. These parameters were determined from four different criterion parameters such as true-positive (TP), true-negative (TN), falsepositive (FP) and false-negative (FN) as follows,

ct þFð1eKÞ9rc9 ¼ 0

Sensitivity ¼ TPVF ¼

ð12Þ

where e controls the relative importance of the curvature component for levelset propagation and the curvature K is calculated as the divergence of the unit normal vector. F MLP , F SOM and F RBF has been used as the speed component F. Throughout levelset contour evolution procedure, an initial contour has been commenced manually as a seed inside the regions of low grade tumors, moderate grade tumor, high grade tumor, necrosis, edema, normal tissue and CSF. Contours were subsequently endured to grow on the speed image with e been set to be one. Fig. 5 illustrates the evolution of contour on top of tumors and their pathologies. Time taken by contour to fit every region was limited by size of the pathology. The region like normal tissue and edema took more iterations than regions such as necrosis and CSF. Finally, segmentation performance of levelset approach has been subjected to evaluation methods.

TP  100%, TP þ FN

Specificity ¼ 1FPVF ¼

SA ¼

TN  100% TN þ FP

TP þ TN  100%, TP þ TN þ FN þFP

CR ¼

TP0:5  FP TP þ FN

ð13Þ

where TPVF, FPVF are true positive volume fraction and false positive volume fraction, respectively. An ideal segmentation is one, which achieves 100% sensitivity, specificity and segmentation accuracy. In case of correspondence ratio (CR), the value ‘‘0’’ indicates that the lesion is completely missing and the value ‘‘1’’ means the lesion is fully identified. CR compares isolated tumor with ground truth tumor in terms of correspondence in size and location based on importance of FPs and FNs. Segmentation accuracy (SA) has been chosen for its ability in providing quantitative assessment of segmentation technique by determining

V. Chinnadurai, G. Damayanti Chandrashekhar / Pattern Recognition 45 (2012) 3501–3511

3507

Fig. 6. Manually segmented images. (a) Moderate grade tumor. (b) High grade tumor. Color Coding: Dark Red: High grade Tumor; Light Red: Moderate Grade Tumor; Green: Edema; White: CSF; Yellow: Normal Tissue; Blue: Necrosis. (For interpretation of the references to color in this figure legend, the reader is referred to the web version of this article.)

Table 2 Sensitivity and specificity values of neural systems: I. Sensitive values. II. Specificity values. Fig. 5. Evolution of the contour in levelset method. Regions

percentage of correctly classified pixels with respect to total number of pixels. Thus, higher SA value means better identification of lesions. In order to assess reliability of ground truth (for manual tracing in pixel level) provided by radiologists, inter operator reliability parameter called interoperator variance (or Jaccard distance) is calculated as

High grade tumor Moderate grade tumor Low grade tumor Necrosis Normal tissue CSF Edema

MLP

SOM

RBF

I

II

I

II

I

II

81 84 83 75 79 80 88

89 82 85 88 85 91 89

85 89 87 79 81 82 96

90 97 95 94 88 95 90

88 90 89 85 86 80 96

89 95 96 92 89 93 93

Interoperator varianceðIVÞ ¼ ð1 Jaccard similarity coefficientÞ  100%

ð14Þ When the value of IV is 0%, then it shows perfect reliability, whereas a value of 100% shows no reliability. Jaccard similarity coefficient represents total amount of tissue that is common to both M1and M2 as a fraction of total amount of tissue in union of M1 and M2.   M1 \ M2 ð15Þ Jaccard similarity coefficient ¼ M1 [ M2 and M1, M2 are manual segmentation provided by two different radiologists.

3. Results As it is mentioned earlier, the proposed methodology has been assessed on forty patients (12 low grades þ17 moderate grades þ11 high grades) who were diagnosed as having glioma in the histopathology investigation. The assessment of the segmentation performance has been carried out in two stages. At first, the initial coarse segmentation images of three different artificial neural networks such as MLP, SOM and RBF have been evaluated. Then, the final segmentation images of every neuro-levelset method has been evaluated each other, and relative improvement achieved using levelset method were assessed quantitatively. Throughout the evaluation, segmentation results of every algorithm have been equated against manual segmentation provided by the expert radiologist. The manual segmentation images have been carried out by two experienced radiologist and mean of interoperator variance (IV) of those manually traced images was found to be 0.8270.18%. Both the radiologists carried out the manual segmentation task without consulting each other. Fig. 6 illustrates the manually segmentation images.

Table 3 Segmentation accuracy of MLP, SOM, RBF. Segmentation accuracy

MLP

SOM

RBF

Mean Maximum Minimum Standard deviation

86.20 88.89 82.22 2.91

92.71 97.00 88.00 3.35

92.43 96.00 89.00 2.70

3.1. Coarse segmentation performance At first, coarse segmented images generated by the trained MLP, SOM and RBF methods were evaluated in detail. Table 2 shows sensitivity and specificity values of artificial neural network systems implemented. Among the three neural systems, performance of radial basis function is better than other two systems. Sensitivity of radial basis function was high except in case of CSF. On the other hand, self-organizing map provided better specificity for moderate and high grade tumors, which has more clinical importance. Mean sensitivity and specificity values of RBF and SOM systems were 87.71 74.92%, 92.43 72.70% and 85.5775.77%, 92.72 73.35%, respectively. At the same time, MLP system performed with moderate mean sensitivity and specificity values of 81.4374.12%, 8773.11%, respectively. As it is evident from Table 3, segmentation accuracy of SOM and RBF were very close to each other. However, in comparison with RBF, standard deviation of SOM was high despite having a better mean value. Mean segmentation accuracy of SOM and RBF were 92.7173.35% and 92.4372.7%. Mean segmentation accuracy of MLP system was found to be 86.2072.91% which was far less than other systems. Correspondence ratio of neural systems has shown that RBF achieves better correspondence with manually traced images

3508

V. Chinnadurai, G. Damayanti Chandrashekhar / Pattern Recognition 45 (2012) 3501–3511

of the radiologist, and their comparative performance is shown in Table 4. RBF achieves the better correspondence ratio with mean value of 0.817 0.047 while SOM observed to be having the mean value of 0.8070.055. On the other hand, MLP system was having mean correspondence value of 0.7170.04. 3.2. Performance of neuro-levelset systems Further, these coarse segmented images generated by MLP, SOM and RBF methods were subjected to the levelset method as the speed image. During implementation of levelset method, a seed contour was initiated inside every pathological region and was allowed to grow slowly. The number of iterations taken by contour to identify the region was primarily decided by size of the region. Region like necrosis took very few iterations and normal tissue took more iterations. Fig. 7 illustrates the coarse segmentation images and final segmentation images generated by artificial neural network systems and three neuro – levelset methods, respectively. Sensitivity and specificity of the neuro-levelset systems are shown in Table 5. The RBF-levelset has provided better improvements with sensitivity and specificity of 91.437 2.94% and 94.43 71.90%, respectively. Sensitivity and specificity of SOM-levelset and MLP-levelset was 89.8671.07%, 92.1472.73% and 89.5773.10%, 90.8673.24%, respectively. As shown in Table 6, segmentation accuracy of MLP-levelset, SOM-levelset and RBF-levelset techniques were 90.5973.10%, 92.1472.73% and 94.4371.90%, respectively. Correspondence ratio has also shown the similar trend which is shown in Table 7. The CR values of the MLP-levelset, SOM-levelset and RBF-levelset techniques were 0.8370.041, 0.8470.016 and 0.8470.028, respectively. Figs. 8 and 9 illustrate improvements of sensitivity and specificity of neuro-levelset methods. Although overall performances of neural systems seem to be improved by implementation of levelset method, there are some shortcomings in performances, especially

Table 4 Correspondence ratio of MLP, SOM, RBF.

in architectures like SOM-levelset. Sensitivities of all the neural systems have been substantially improved by the levelset method. However, it has found that specificities of self-organizing maps and multi layer perceptron were not improved in conjunction with levelset method. In case of MLP-levelset, as it is evident from the chart, sensitivity of every region has been improved with levelset method. Primarily, necrosis and high grade tumor observed to have very high improvement of 17.33% and 11.11% in comparison with coarse segmented image of multi layer perceptron. Similar trend has been observed in specificity of MLP-levelset system. Performance of Table 5 Sensitivity and specificity of neuro-levelset methods: I. Sensitivity, II. Specificity. Regions

High grade tumor Moderate grade tumor Low grade tumor Necrosis Normal tissue CSF Edema

MLP-levelset

SOM-levelset

RBF-levelset

I

II

I

II

I

II

90 91 90 88 85 88 95

91 97 91 91 90 86 90

90 91 91 90 88 89 90

88 96 92 92 90 95 92

91 92 93 89 89 89 97

95 97 94 95 92 96 92

Table 6 Segmentation accuracy of neuro-levelset methods. Segmentation accuracy

MLP-levelset

SOM-levelset

RBF-levelset

Mean Maximum Minimum Standard deviation

90.59 96.34 86.42 3.10

92.14 96.00 88.00 2.73

94.43 97.00 92.00 1.90

Table 7 Correspondence ratio of neuro-levelset methods.

Correspondence ratio

MLP

SOM

RBF

Correspondence ratio

MLP-levelset

SOM-levelset

RBF-levelset

Mean Maximum Minimum Standard deviation

0.71 0.79 0.65 0.04

0.8 0.86 0.75 0.055

0.81 0.87 0.77 0.047

Mean Maximum Minimum Standard deviation

0.83 0.86 0.78 0.041

0.84 0.89 0.80 0.016

0.84 0.90 0.80 0.028

Fig. 7. Segmented images of moderate grade tumor and high grade tumor. (a) MLP; (b) SOM; (c) RBF; (d) MLP—Levelset; (e) SOM—Levelset; (f) RBF—Levelset. High grade tumor: (g) MLP; (h) SOM; (i) RBF; (j) MLP—Levelset; (k) SOM—Levelset; (l) RBF—Levelset. Color Coding: Dark Red: High grade Tumor; Light Red: Moderate Grade Tumor; Green: Edema; White: CSF; Yellow: Normal Tissue; Blue: Necrosis. (For interpretation of the references to color in this figure legend, the reader is referred to the web version of this article.)

V. Chinnadurai, G. Damayanti Chandrashekhar / Pattern Recognition 45 (2012) 3501–3511

3509

Fig. 8. Improvement of sensitivity performance of neural systems with levelset.

Fig. 9. Improvement of specificity performance of neural systems with levelset.

Fig. 10. Mean sensitivity comparison chart of segmentation techniques.

all other regions found to be improved between 7.59% and 10%. Moderate grade tumor has been significantly improved with 18.29% while all other regions found to be improved with moderate values varying between 1.12 and 7.06. However, interestingly, CSF specificity values were decreased by 5.49%. This may be due to over segmentation achieved by contour that was allowed to evolve during levelset implementation. On the other hand, results achieved by SOM –levelset showed mixed performances. There is an improvement in sensitivity of SOM- levelset in segmenting all regions except edema. In case of edema,  6.25% of the region which was found to be edema with self-organizing map alone has not been recognized by SOM-levelset. Similarly, specificity of SOM-levelset has been seriously affected by levelset method. Other than marginal betterment in case of edema and normal tissue, most of the regions face major deterioration in its performance, which is clearly seen from its specificity values poor variations. In comparison with another two architectures, sensitivity and specificity of RBF-levelset has achieved better improvements. In particular, CSF has observed high sensitivity improvement with 11.25% and all other regions have achieved moderate sensitivity improvement, which was varying between 1.04% and 4.71%. Overall improvement of specificity of RBF-levelset has been good

except low grade tumor and edema. The specificity of CSF, normal tissue, necrosis, moderate grade tumor and high grade tumor has found to be improved between 2.11% and 6.75%.

4. Discussion Mean value of validation criterions such as sensitivity, specificity computed across every region of image segmentation techniques implemented to have been shown in Figs. 10 and 11. As it is clearly manifest from comparative plots, overall performance of RBF-levelset is better as compared to all other image segmentation techniques used. Although self-organizing maps based techniques performs similar to radial basis function based method, its performance declines strongly in combination with levelset. This could be due to over segmentation achieved by levelset. Present work outperforms most of the study who has been carried out in order to perform brain tumor segmentation as shown in Table 8. Almost every study except Emblem et al. [10,16] has validated their results on very small patient population. As patient population increases, same algorithms may provide different performance results. However, our study has been carried

3510

V. Chinnadurai, G. Damayanti Chandrashekhar / Pattern Recognition 45 (2012) 3501–3511

Fig. 11. Mean specificity comparison chart of segmentation techniques.

Table 8 Comparative chart of various brain tumor segmentation work and the present study. Research work

Technique

Cases

Validation

Validity value

Images

Liu et al. [11]

Fuzzy connectedness

10

Semi-automated method

10

Solomon et al. [13] Chi-Hoon Lee et al. [14] Lee et al. [7]

Hidden Markov PCRF k–NN algorithm

10

Emblem et al. [10] Wang et al. [15] Emblem et al. [16] Nie et al. [17] Taheri et al. [25] Present study

Manual histogram method Fluid vector flow Knowledge-based fuzzy c-means HMRF Adaptive scheme RBF-levelset

53

96.72% 96.55% 79.12–93.25% 0.74–0.91 87% 7 7% 71.11 78 7 18% 79 7 6% 90% 61 7 5% 83% (LG), 69% (HG) 75%–76% (HG), 70%–75% (LG) 87.5 7 3.9%. 91.43 7 2.94% 94.43 7 1.90% 0.84 7 0.028 94.43 7 1.90%

FLAIR, T1, & T1 Enhan

Kai Xie et al. [12]

Sensitivity Specificity PM (sensitivity) CR Sensitivity JC Sensitivity Specificity Sensitivity TM Sensitivity JC JM Sensitivity Specificity CR SA

26

50 15 16 40

out on 40 cases and evaluation criterion has been observed to be better than other studies. Although every study has performed different validation criteria, parameters such as FPVF, PM, FNVF and PPV can be easily compared with sensitivity and specificity. This study primarily used sensitivity and specificity as evaluation parameters instead of Jaccard similarity coefficients due to their ability in providing under and over segmentation of lesions [48]. While comparing performance of these studies, it is evident that methodologies presented to perform better than those presented in literature so far. Among various algorithms used, RBF-levelset observed to be performing better in segmenting brain tumor, edema, necrosis, CSF and normal tissue with mean sensitivity and specificity values of 91.4372.94%, 94.4371.90%, respectively. Correspondence ratio of RBF-levelset has found to be 0.847 0.028. It also achieved better segmentation accuracy of 94.437 1.90%. Although overall performance of RBF-levelset found to be better than other neuro-levelset systems, the marginal poor performance in case of low grade tumor and edema derails performance of RBF-levelset slightly.

5. Conclusion A comprehensive methodology based on neuro-levelset systems has been evaluated for brain tumor segmentation and grading using reconstructed images of DSC and DW images. Three different neural systems such as MLP, SOM and RBF have been integrated independently with the levelset system where neural systems were trained to provide speed images to levelset system. Among the three neuro-levelset systems, overall performance of RBF-levelset is better for the segmentation and grading of brain

T1 Enhan Synthetic images, T1, T1 Enhan T1, T2, and T1 Enhan T1 Enhan, perfusion, MRS, diffusion CBV T1 T1,T2,T1 Enhan T1,T2, FLAIR T1, & T1 Enhan ADC PBP,CBV,TTP

tumors and their associated pathologies. Evaluation of the segmented images with the manually segmented images reveals that the results of RBF-levelset system produces mean sensitivity and specificity values of 91.43 72.94%, 94.43 71.90%, respectively. Similarly, correspondence ratio and segmentation accuracy of RBF-levelset were 0.8470.028 and 94.43 71.90%. In conclusion, methodology proposed has shown promising results and can combine information provided by CBV, PBP, TTP and ADC images to segment and grade brain tumors and their pathologies such as edema, necrosis. The result also showed that neuro-levelset method is capable of providing improvised results as compared with other algorithms for the tumor segmentation and their grading.

Reference [1] M.D. Jenkinson, D.G. Du Plessis, C. Walker, T.S. Smith, Advanced MRI in the management of adult gliomas, British Journal of Neurosurgery 21 (6) (2007) 550–561. [2] S.J. Price, The role of advanced MR imaging in understanding brain tumour pathology, British Journal of Neurosurgery 21 (6) (2007) 562–575. [3] H.S. Kim, S.Y. Kim, A prospective study on the added value of pulsed arterial spin-labeling and apparent diffusion coefficients in the grading of gliomas, American Journal of Neuroradiology 28 (9) (2007) 1693–1699. [4] Axel Wismueller, Segmentation with neural networks. Handbook of Medical Image Processing and Analysis, 2nd edn., , 2009 113–143. [5] P. Georgiadis, D. Cavouras, I. Kalatzis, et al., Enhancing the discrimination accuracy between metastases, gliomas and meningiomas on brain MRI by volumetric textural features and ensemble pattern recognition methods, Magnetic Resonance Imaging 27 (1) (2009) 120–130. [6] P. Georgiadis, D. Cavouras, I. Kalatzis, et al., Improving brain tumor characterization on MRI by probabilistic neural networks and non-linear transformation of textural features, Computational Methods Programs Biomedicine 89 (1) (2008) 24–32.

V. Chinnadurai, G. Damayanti Chandrashekhar / Pattern Recognition 45 (2012) 3501–3511

[7] M.C. Lee, S.J. Nelson, Supervised pattern recognition for the prediction of contrast-enhancement appearance in brain tumors from multivariate magnetic resonance imaging and spectroscopy, Artificial Intelligence Medicine 43 (1) (2008) 61–74. [8] J.J. Corso, E. Sharon, S. Dube, S. El-Saden, U. Sinha, A. Yuille, Efficient multilevel brain tumor segmentation with integrated Bayesian model classification, IEEE Transactions on Medical Imaging 27 (5) (2008) 629–640. [9] M. Wels, G. Carneiro, A. Aplas, M. Huber, J. Hornegger, D. Comaniciu, A discriminative model-constrained graph cuts approach to fully automated pediatric brain tumor segmentation in 3-D MRI, Medical Image Computing and Computer Assisted Intervention 11 (1) (2008) 67–75. [10] K.E. Emblem, B. Nedregaard, T Nome, et al., A. Glioma grading by using histogram analysis of blood volume heterogeneity from MR-derived cerebral blood volume maps, Radiology 247 (3) (2008) 808–811. [11] J. Liu, J.K. Udupa, D. Odhner, D. Hackney, G. Moonis, A system for brain tumor volume estimation via MR imaging and fuzzy connectedness, Computerized Medical Imaging and Graphics 29 (1) (2005) 21–34. [12] Kai Xie, et al., Semi-automated brain tumor and edema segmentation using MRI, European Journal of Radiology 56 (2005) 12–19. [13] J. Solomon, J.A. Butman, A. Sood, Segmentation of brain tumors in 4D MR images using the hidden Markov model, Computational Methods Programs Biomedicine 84 (2-3) (2006) 76–85. [14] Chi-Hoon Lee, Wang Murtha, Brown Greiner, Segmenting brain tumors using pseudo-conditional random fields, Medical Image Computing and Computer Assisted Intervention 11 (1) (2008) 359–366. [15] T. Wang, I. Cheng, A. Basu, Fluid vector flow and applications in brain tumor segmentation, IEEE Transactions on Biomedical Engineering 56 (3) (2009) 781–789. [16] K.E. Emblem, Nedregaard Hald, Due Bjornerud, Automatic glioma characterization from dynamic susceptibility contrast imaging: brain tumor segmentation using knowledge-based fuzzy clustering, Journal of Magnetic Resonance Imaging 30 (1) (2009) 1–10. [17] J. Nie, Z. Xue, T. Liu, G.S. Young, K. Setayesh, L. Guo, S.T. Wong, Automated brain tumor segmentation using spatial accuracy-weighted hidden Markov random field, Computerized Medical Imaging and Graphics 33 (6) (2009) 431–441. [18] S. Powell, V.A. Magnotta, H. Johnson, V.K. Jammalamadaka, R. Pierson, N.C. Andreasen, Registration and machine learning-based automated segmentation of subcortical and cerebellar brain structures, Neuroimage 39 (1) (2008) 238–247. [19] J.M. Buhmann, T. Lange, U. Ramacher, Image segmentation by networks of spiking neurons, Neural Computing 17 (5) (2005) 1010–1031. [20] J.S. Suri, K. Liu, S. Singh, S.N. Laxminarayan, X. Zeng, L. Reden, Shape recovery algorithms using level sets in 2-D/3-D medical imagery: a state-of-the-art review, IEEE Transactions on Information Technology in Biomedicine 6 (1) (2002) 8–28. [21] A.H. Zhuang, D.J. Valentino, A.W. Toga, Skull-stripping magnetic resonance brain images using a model-based level set, Neuroimage 32 (1) (2006) 79–92. [22] C. Baillard, P. Hellier, C. Barillot, Segmentation of brain 3D MR images using level sets and dense registration, Medical Image Analysis 5 (3) (2001) 185–194. [23] L. Jonasson, P. Hagmann, X. Bresson, J.P. Thiran, J. Van Wedeen, Representing diffusion MRI in 5D for segmentation of white matter tracts with a levelset method, Information Processing Medical Imaging 19 (2005) 311–320. [24] C. Ciofolo, C. Barillot, Brain segmentation with competitive level sets and fuzzy control, Information Processing Medical Imaging 19 (2005) 333–344. [25] Taheri, et al., Level-set segmentation of brain tumors using a threshold-based speed function, Image and Vision Computing 28 (2010) 26–37. [26] Stephan E. Maier, Hatsuho Mamata, Diffusion Imaging of Brain Tumors: Handbook of Neuro-Oncology NeuroImaging, Academic Press, 2008 pp. 239–247.

3511

[27] C. Calli, O. Kitis, N. Yunten, T. Yurtseven, S. Islekel, T. Akalin, Perfusion and diffusion MR imaging in enhancing malignant cerebral tumors, European Journal of Radiology 58 (3) (2006) 394–403. [28] N. Fayed, J. Da´vila, J. Medrano, S. Olmos, Malignancy assessment of brain tumours with magnetic resonance spectroscopy and dynamic susceptibility contrast MRI, European Journal of Radiology 67 (3) (2008) 427–433. [29] Sorensen AG Huisman, Perfusion-weighted magnetic resonance imaging of the brain: techniques and application in children, European Radiology 14 (1) (2004) 59–72. [30] Axel Wismueller, Segmentation with neural networks, Handbook of Medical Image Processing and Analysis (2009) 113–143. [31] Robert.Duin Dick De Ridder, Lucas. Petersen, Piet Van Vliet, Verbeek, nonlinear image processing using artificial neural network, Advances in Imaging and Electron Physics 126 (2003) 351–450. [32] D.N. Louis, H. Ohgaki, O.D. Wiestler, W.K. Cavenee (Eds.), WHO Classification of Tumours of the Central Nervous System, IARC, Lyon, France, 2007. [33] S.W. Coons, P.C. Johnson, B.W. Scheithaur, A.J. Yates, D.K. Pearl, Improving diagnostic accuracy and interobserver concordance in the classification and grading of primary gliomas, Cancer 79 (7) (1997) 1381–1393. [34] Ruttimann Philippe Thevenaz, A. Michael Unser, Pyramid approach to subpixel registration based on intensity, IEEE on Transaction on Medical Imaging 7 (1) (1998) 27–41. [35] L.G. Nyul, J.K. Udupa, On standardizing the MR image intensity scale, Magnetic Resonance in Medicine 42 (1999) 1072–1081. [36] A. Kowalczyk, Estimates of storage capacity of multilayer perceptron with threshold logic hidden units, Neural Networks 10 (8) (1997) 1417–1433. [37] Nunes Ludwig, Novel maximum-margin training algorithms for supervised neural networks, IEEE Transactions on Neural Networks 21 (6) (2010) 972–984. [38] D. Brugger, M. Bogdan, W. Rosenstiel, Automatic cluster detection in Kohonen’s SOM, IEEE Transactions on Neural Networks 19 (3) (2008) 442–459. [39] C. Vijayakumar, G. Damayanti, R. Pant, C.M. Sreedhar, Segmentation and grading of brain tumors on apparent diffusion coefficient images using selforganizing maps, Computerized Medical Imaging and Graphics 31 (7) (2007) 473–484. [40] F. Schwenker, H.A. Kestler, G. Palm, Three learning phases for radial-basisfunction networks, Neural Networks 14 (4-5) (2001) 439–458. [41] W. Chen, M.L. Giger, U. Bick, A fuzzy c-means (FCM)-based approach for computerized segmentation of breast lesions in dynamic contrast-enhanced MR images, Academic Radiology 13 (1) (2006) 63–72. [42] J.A. Sethian, A fast marching levelset method for monotonically advancing fronts, Proceedings of the National academy of Sciences 93 (4) (1996) 1591–1595. [43] R. Malladi, J.A. Sethian, Image processing via level set curvature flow, Proceedings of the National academy of Sciences 92 (15) (1995) 7046–7050. [44] Malladi Deschamps, Ravve, fast evolution of image manifolds and application to filtering and segmentation in 3D medical images, IEEE Transactions on Visualization and Computer Graphics 10 (5) (2004) 525–535. [45] R. Malladi, J.A. Sethian, A unified approach to noise removal, image enhancement, and shape recovery, IEEE Transactions on Image Processing 5 (11) (1996) 1554–1568. [46] Ferenc A. Neculai Archip, Simon K. Jolesz, A. Warfield, Validation framework for brain tumor segmentation, Academic Radiology 14 (2007) 1242–1251. [47] Y. Zhang, A survey on evaluation methods for image segmentation, Pattern Recognition 29 (1996) 1335–1346. [48] J.K. Udupa, et al., A framework for evaluating image segmentation algorithms, Computerized Medical Imaging and Graphics 30 (2006) 75–87.

C. Vijayakumar was born in Coimbatore, India, in 1979. He received his ‘‘Doctorate in Electronics Science’’ degree from Pune University, Pune, India in 2011. In 2002, he joined Defense Research and Development organization (DRDO) as Scientist and currently deputed to Department of Radiodiagnosis and Imaging, Armed Forces Medical College, Pune, India. His research interests are artificial neural networks, levelset, image segmentation, image vision, multiresolution analysis.

Gharpure Damayanti received her M.Sc. and Ph.D. degrees from Pune University in 1984 and 1992, respectively. She joined the Department of Electronic Science, University of Pune, India in 1988. Since then she has worked on a number of research projects and consultancy projects related to machine vision applications, design of embedded E-nose, hardware implementation of Automatic trajectory tracking system, etc. Her current research interests include applications of neural networks for pattern recognition, image analysis and segmentation, odor analysis and embedded system design.