Detection of common defects on oranges using hyperspectral reflectance imaging
Jiangbo Li,Xiuqin Rao,Yibin Ying
DOI: https://doi.org/10.1016/j.compag.2011.05.010
IF: 8.3
2011-01-01
Computers and Electronics in Agriculture
Abstract:To detect various common defects on oranges, a hyperspectral imaging system has been built for acquiring reflectance images from orange samples in the spectral region between 400 and 1000 nm. Oranges with insect damage, wind scarring, thrips scarring, scale infestation, canker spot, copper burn, phytotoxicity, heterochromatic stripe, and normal surface were studied. Hyperspectral images of samples were evaluated using principal component analysis (PCA) with the goal of selecting several wavelengths that could potentially be used in an in-line multispectral imaging system. The third principal component images using six wavelengths (630, 691, 769, 786, 810 and 875 nm) in the visible spectral (VIS) and near-infrared (NIR) regions, or the second principal component images using two wavelengths (691 and 769 nm) in VIS region gave better identification results under investigation. However, the stem-ends were easily confused with defective areas. In order to solve this problem, representative regions of interest (ROIs) reflectance spectra of samples with different types of skin conditions were visually analyzed. The researches revealed that a two-band ratio (R875/R691) image could be used to differentiate stem-ends from defects effectively. Finally, the detection algorithm of defects was developed based on PCA and band ratio coupled with a simple thresholding method. For the investigated independent test samples, accuracies of 91.5% and 93.7% with no false positives were achieved for both sets of selected wavelengths using proposed method, respectively. The disadvantage of this algorithm is that it could not discriminate between different types of defects. Keywords Hyperspectral imaging Principal component analysis Defect detection Oranges Ratio images 1 Introduction Citrus is one of the major fruit products in China. Its annual production is over 23 million tons. For a long time, the agriculture industry has attempted to automate fruit sorting in order to decrease production costs and increase the quality of the production. Fresh market fruits are graded into quality categories according to parameters such as size, color, shape and external defects ( Leemans and Destain, 2004 ). The first two quality criteria have been automated on current commercial graders, but fruits grading according to the presence of defects is yet very challenging, although the most advanced machines are also capable of detecting blemishes ( Aleixos et al., 2002; Leemans and Destain, 2004; Blasco et al., 2007a ). Since the color is the natural sense we use to make our first evaluation of the quality of fruits, most of the inspection systems use this information to detect skin defects of fruit ( Miller and Delwiche, 1991; Leemans et al., 1998; Mendoza and Aguilera, 2004; Blasco et al., 2007b ). In terms of citrus, Recce et al. (1998) identified orange defects based on gray value differences between defects and sound peel in R and G component images. A neural network classifier on rotation invariant transformations was used to recognize the radial color variation of the stem region. Diaz et al. (2004) observed abnormal color on table olive was related with low values of R and G co-ordinates. Defects were then segmented using a Bayesian model. Three different algorithms were applied to classify the olives, and results showed that a neural network with a hidden layer was able to classify the olives with an accuracy of over 90%. Leemans and Destain (2004) compared defect segmentation results of three models that were generated using the RGB, RGI and RGBI. The RGBI model, which was found to be the best, correctly recognized 100% of the stem-ends and 99.1% of defects based on Bayesian classification model. In a recent study, Blasco et al. (2007b) reported the application of near-infrared, ultraviolet and fluorescence computer vision systems to detect the external defects of citrus fruits. Although proposed approach could detect different peel conditions, a complex image processing algorithm affect the real-time application in packinghouse processing line. Kim et al. (2009) used 14 color texture features of grapefruit peel diseases to classify defects based on HIS model. López-García et al. (2010) developed a citrus fruits surface detection algorithm, which was based on multivariate image analysis strategy and principal component analysis approach. The success ratio for the detection of individual defect was 91.5%. The classification accuracy was acceptable. However, the complexity of the algorithm restrained the detection speed. From these studies, the researchers were more interested in detecting the defects based on color images and more complex algorithms. In agricultural production systems, the time available to evaluate individual object normally ranges from tenths to hundredths of a second ( Kim et al., 2002 ). In order to achieve high processing speeds, sometimes inspection system worked with low resolution images, or used more advanced digital signal processor. However, the low resolution images reduced the accuracy of the system, especially for detection of some very small defects, such as scale infestation on citrus surface, and the advanced hardware increased cost of system. Hyperspectral imaging method, which combines the features of imaging and VIS/NIR spectroscopy to simultaneously acquire spatial and spectral information, has attracted the interest of researchers as a powerful tool for detecting a variety of agricultural products. Examples include detecting bruises on apples ( Xing et al., 2005; ElMasry et al., 2008 ), bruises on pickling cucumbers ( Ariana et al., 2006 ), pork marbling ( Qiao et al., 2007 ), contaminant detection on poultry carcasses and cantaloupes ( Lawrence et al., 2006; Vargas et al., 2005 ), pits in tart cherries ( Qin and Lu, 2005 ), nematodes in cod fillets ( Heia et al., 2007 ), cracks in shell eggs ( Lawrence et al., 2008 ), and so on. In order to detect citrus surface defects, Gómez-Sanchis et al. (2004, 2008) developed multispectral imaging to detect infections caused by Penicillium digitatum in citrus fruits. To investigate the detection of canker on citrus fruit surface, Qin et al. (2008, 2009) utilized principal component analysis (PCA) together with a simple threshold classifier and spectral information divergence (SID) classification method to discriminate canker on grapefruit from other peel diseases using hyperspectral images with an accuracy of 92.7% and 95.2%, respectively. These research works provided good references and resources for dealing with various problems associated with detection of citrus surface defects. However, to our knowledge no attempts were made to differentiate different defect types such as insect damage, wind scarring, thrips scarring, scale infestation, canker spot, copper burn, phytotoxicity, heterochromatic stripe using hyperspectral imaging. In addition, hyperspectral imaging technology has not yet been directly implemented in an in-line system for automated quality detection because its time requirements for image acquisition and analysis are too great ( Mehl et al., 2002 ). Usually, the hyperspectral data can be used to determine optimal wavelengths for developing multispectral imaging system. The multispectral imaging approach in conjunction with selected wavelengths is favorable for rapid in-line assessments ( Kim et al., 2002, 2005; Liu et al., 2007 ). The main objective of the present study was to investigate the potential of using a hyperspectral imaging for detecting various common defects on orange surfaces. For this purpose, the research was conducted through (1) development of a hyperspectral imaging system with a spectral region from 400 to 1000 nm to detect various common defects on orange surfaces; (2) determination of effective wavelengths for skin defects detection based on principal component analysis method; (3) development of an algorithm to identify stem-ends from true defects; (4) development of a simple algorithm to isolate defected areas from sound surfaces. The ultimate purpose was to develop a faster and more efficient multispectral method for real-time inspection of orange defects. 2 Materials and methods 2.1 Orange samples The orange samples were hand picked from two different commercial orchards in Jiangxi during the harvest season of 2009 and 2010, respectively. A total of 460 samples were separated into non-defect and defect groups by visual inspection. The defects included insect damage, wind scarring, thrips scarring, scale infestation, canker spot, copper burn, heterochromatic stripe and phytotoxicity. The diseases on the fruit surfaces show different symptoms. Wind scar, which is caused by leaves, twigs, or thorns rubbing against the fruit, is a common physical injury on the fruit peel, and the scar tissue is generally gray ( Qin et al., 2009 ). Scale infestation, insect damage, and thrips scarring are caused by insect, which generate surface blemishes during the fruit growing season. The armor of the scale is 2–3 mm long, dark brown in appearance. Lesions of insect damage and thrips scarring are caused by some insect bites and the color of surface blemishes are gray and brown, respectively. Heterochromatic stripe fruit is characterized by non-uniform surface color. Those regions commonly appear as deep yellow stripe shape. Citrus canker is caused by bacteria, and it is featured with conspicuous dark lesions. Most circular in shape, canker lesions vary in number, and they are superficial (up to 1 mm deep) on the fruit peel ( Qin et al., 2009; Schubert et al., 2001 ). Number of the canker lesions on every orange surface in this study was approximately in the range of 5–27. Copper burn scar, as a non-infectious disease, is caused by high temperature from long time sun exposure. The affected areas exhibit tan lesions in appearance. Phytotoxicity is caused by residual pesticide and exhibit brown around fruit calyx ( Li et al., 2010 ). The area of infected region is bigger than one-seventh of fruit surface area in this study. Fig. 1 shows samples with various common peel defects and stem-end. Twenty samples of each type of skin defect and thirty samples of normal oranges were selected from the 2009 data set as training set and used to develop the algorithm. In addition, thirty samples of each peel type were selected from the 2010 data set as test set and used to evaluate the performance of algorithm for this study. All the samples were washed and treated with chlorine and sodium o-phenylphenate (SOPP) at Machine Vision Lab of College of Biosystems Engineering and Food Science, Zhejiang University, Hangzhou. These samples were then stored in an environmental control chamber maintained at 5 °C and they were removed from cold storage about 2 h before imaging to allow them to reach room temperature. 2.2 Hyperspectral imaging system A schematic diagram of developed hyperspectral imaging system with a spectral resolution of approximately 0.58 nm is shown in Fig. 2 . The system consists of five components: an imaging spectrograph (ImSpector V10E-QE, Spectral Imaging Ltd., Oulu, Finland) coupled with a standard C-mount zoom lens (V23-f/2.4, Specim Ltd., Finland), two 150 Watt (W) halogen lamps assemblies (SCHOTT DCR III, SCHOTT North America, Inc., USA) provided a uniform VIS–NIR illumination for the sample in the field of view of the optics, a Hamamastsu monochrome linear CCD camera (C8484-05G, Hamamatsu Photonics, Japan) with 1344 effective pixels, a sample transportation plate (PSA200-11-X, Zolix Instruments Co., Ltd., Beijing, China), and a computer (ACER, Inter®CoreTM2 4400@2.00 GHz, RAM 1.00G). The spectrograph has a prism-grating-prism (PGP) element, which is a holographic transmission grating. During measurements with the spectrograph-camera assembly, the system is well shielded from the environment to minimize interference from ambient light ( Wallays et al., 2009 ). The PGP disperses the incoming light corresponding to a pixel in the scanned line into its spectral components and projects the range from 400 to 1000 nm onto the CCD of the camera. The size of the acquired images is 1344 by 700 pixels with a resolution of 9.25 pixels/mm. The wavelength range between 550 nm and 900 nm was used in this investigation due to inefficiencies of the system at certain wavelength regions (e.g., low light output in the VIS < 550 nm, and low quantum efficiency of the CCD in the NIR > 900 nm). 2.3 Hyperspectral image acquisition The laboratory-based system was operated in a darkened inspection chamber where only the halogen light sources were used. In order to acquire more accurate data, some parameters need be adjusted before the images were acquired. In this study, the exposure time was adjusted to 200 ms and the speed of fruit movement was adjusted to 0.12 cm/s throughout the test. The object distance was set 42.5 cm. The average illumination intensity from samples surface was 27333 lux acquired through a lux gauge (TES1336A). During image acquisition, the orange samples were placed on a tray painted with flat black paint that were fixed on the positioning table ( Fig. 2 ), manually orienting the side of the fruit that contained the defects towards the camera. The camera and spectrograph were then used to scan the oranges line-by-line as the transportation plate moved the oranges through the field of view of the optical system. The line scan data were saved and processed later to create hyperspectral image cubes containing spatial and spectral data. The spectral images were acquired based on Spectral Cube_v2_75 software (Spectral Imaging Ltd., Finland). However, after images were acquired, we found that several defective areas located on the border of the fruit in acquired images due to human fatigue error. These images were also used in our study. Due to the uneven intensity of light source in different bands and the existence of dark current in CCD camera, some bands with less light intensity acquired the bigger noises ( Polder et al., 2003 ). Therefore, the hyperspectral images need to be calibrated with a white and a dark references. The dark reference was used to remove the dark current effect of the CCD detectors, which are thermally sensitive. The dark image (with 0% reflectance) was collected by turning off all light sources and covered the lens with a black cap. A Teflon white board with the 99% reflection efficiency (Spectralon, Labsphere Inc.) was used to obtain white reference image. The corrected image ( R ) was calculated using Eq. (1) ( Mehl et al., 2002; Xing et al., 2005; ElMasry et al., 2009 ): (1) R = R o - R d R r - R d where R o is the acquired original hyperspectral image, R r is the white reference image, R d is the dark image. 2.4 Data processing and analyzing All data processing and analyzing were performed using the Environment for Visualizing Images software program (ENVI 4.6, Research System Inc., Boulder, CO, USA) and Matlab 2008a (The MathWorks Inc., Natick, USA) with the image processing toolbox. This research uses the ENVI software package for the application of PCA to the hyperspectral images of oranges. In the process of creating the PCA images, a correlation matrix of the image is calculated. This correlation matrix is then used to compute the eigenvalues. The eigenvalues are equivalent to the variance of each principal component (PC) image. These PC images are ordered in the decreasing degree of variance sizes, where first PC accounts for the largest variance ( Liu et al., 2007 ). The more details on this method can be found in Malinowski and Howery (1980) . In this study, the PCA was used to reduce spectral dimensionality of the hyperspectral reflectance images, and to aid in visualizing the hyperspectral data to determine the several dominant wavebands responsible for discriminating defects from normal orange surfaces. Before applying the PCA and band ratio, a binary mask was created to produce an image containing only the fruit, avoiding any interference from the background that could reduce discrimination efficiency. Image at 750 nm was used for this task because it appeared the best contrast between the fruit surface and the background and can be segmented easily by setting a simple threshold value. Subsequently, individual principal component (PC) images were visually evaluated to determine PC images with (1) the least variation in normal orange surfaces and (2) the largest contrast between defective areas and sample surfaces. Each PC image is a linear sum of the original images at individual wavelengths multiplied by corresponding (spectral) weighing coefficients. Several wavelengths with high (local maxima) and low (local minimum) weighing coefficients from the PC image were selected as the dominant wavelengths ( Vargas et al., 2005 ). Principal components images using only the selected dominant wavelengths (multispectral images) were re-calculated. Because band ratio images can effectively enhance the contrast between different regions and produce more uniform responses across the orange surface ( Vargas et al., 2005 ), the two-band ratio method was also used in this study. The two-band ratio was performed as Eq. (2): (2) Q t / k = R t R k where Q t / k represents a quotient of spectral reflectances, and R t and R k are reflectance intensities at t nm and k nm, respectively. Finally, the multispectral PC images and two-band ratio images were combined and subjected to a simple thresholding method to segment the defective areas from the normal areas. In addition, morphological filtering was used during defects segmentation with an aim of removing undesired small size pixels (noises) in the binary images. One step morphological opening operation based on a rectangle structuring element with a 3 × 3 kernel size in MATLAB morphological filter tools were used in this study and defined as erosion of the image by structuring element, followed by a dilation of the result by same structuring element. Structuring element with a 3 × 3 kernel size was used mainly considering effectively removing the noises and retaining some small defects like scale infestation as many as possible in defect binary image. 3 Results and discussion 3.1 Hyperspectral reflectance spectra The representative regions of interest (ROIs) reflectance spectra of orange samples studied in the wavelength range between 550 and 900 nm are shown in Fig. 3 . These spectra were extracted from the hyperspectral image data of training set and were an average of twenty spectra (one per orange) for each type of peel condition, except for stem-end and sound orange which were used to obtain the mean spectra from fifteen spectra (one per orange) per type. Each spectrum was obtained from a rectangular ROI, varying in size from 80 to 100 pixels. The reflectance of spectra depicted in the VIS region was lower than in the NIR region over the entire spectral region. The spectra of sound peel showed higher reflectance comparing to the defective peels and stem-ends in the spectral region between 600 and 775 nm. Therefore, the bands from VIS region could be more adequate for defects detection than NIR region bands. However, the spectra shown in Fig. 3 do not account for the spatial variations (decreases) in intensities from the center portions toward the edges. The single band image was also attempted to segment different types of defects studied by using a simple thresholding. The results showed that it was hardly to get the target due to variations in the gray level within the defective area and the surrounding surface, which was in agreement with results achieved by Benneden and Peterson (2005) . On the one hand, it was reasonable to observe that the spectral features of heterochromatic stripe fruit were similar to sound fruit surfaces, which made the detection of this defect very difficult. On the other hand, it was also practically very difficult to identify heterochromatic stripe area from sound skins based on RGB images due to the color similarity ( Fig. 1 ). In addition, one common feature also observed from stem-end and phytotoxicity in the spectra is absorption of chlorophyll a at approximately 689 nm. The spectra of most defective surfaces have maxima located approximately at 810 and 875 nm, respectively. On the basis of these features, band ratio algorithms could be promoting to identify stem-ends from other defects. 3.2 PCA in the VIS to NIR region of the spectrum The first three images (denoted by PC-1 to PC-3) obtained from PCA for the hyperspectral reflectance images of oranges samples using all 599 wavelengths in the region from 550 to 900 nm are shown in Fig. 4 . The PCA provides a means to reduce the high spectral dimensionality of image data. Features such as wind scarring and heterochromatic stripe which are not readily visible in the individual images are more apparent in these images. In the first principal component (PC-1) images, intensity decreases from the center to the edges of oranges, and they do not provide more unique features than the original untransformed hyperspectral images. The PC-2 images demonstrate more information of defects on fruit surfaces. The defective regions could be clearly identified in the PC-2, especially for insect damage, wind scarring, thrips scarring, copper burn, phytotoxicity and heterochromatic stripe. In the PC-3 images, only the insect damage and heterochromatic stripe could be more effectively identified comparing to PC-2. Starting from PC-4 (figures not shown), the transformed images no longer possess meaningful information, and they are not useful for oranges surface defects detection. Based on visual assessment, the PC-2 images provide the better peel defects discrimination. However, the noises are also observed in some sample surfaces, such as insect damage, canker spot and stem-end. The main reason may be due to too many bands used to perform PCA. 3.3 PCA in the VIS region of the spectrum The gray levels variations between defective and sound skin regions were mainly affected by the spectra from VIS region. The spectra in the NIR region were usually not sensitive to the variations. Thus, PCA on the full wavelength region (VIS to NIR) may weaken the contrast of different regions on fruit surface. In general, the band region from 380 to 780 nm is known as VIS region ( Hernández-Andrés et al., 2001; Skoglund et al., 2004 ). In addition, the image information below 600 nm shows poor features in our study. The poor features may be brought because chemical components in the orange peel are not sensitive to this waveband region. Therefore, the hyperspectral images in the wavelength range of 600–780 nm were used to perform PCA for further analysis. Fig. 5 illustrates representative PC-1, PC-2 and PC-3 images obtained from the PCA of the 600–780 nm hyperspectral reflectance image data. The PC-1 images reflected a weighted sum and showed effects similar to those observed in the entire spectral region ( Fig. 4 ). Subsequent PC-2 images depicted the best contrast between skin defects and normal orange surfaces for different types of samples. PC-3 images did not give any useful information for defects identification. Compared to PC-2 images in Fig. 4 , the PC-2 images shown in Fig. 5 were more effective to identify various peel defects. It was also noticed that some obvious noises remained on orange surfaces, such as normal surface with stem-end. These noises were easy to be misclassified as defects (false positive). In addition, using too many wavelengths (309 spectral channels with a spectral resolution of approximately 0.58 nm) was also not effective to develop multispectral system for defects detection. 3.4 Selection of optimal spectral wavebands The weighing coefficients for the PC-2 obtained by using images across the entire spectral region are shown in Fig. 6 a for wind scarring, insect damage, scale infestation and thrips scarring, and 6b for canker spot, copper burn, heterochromatic stripe, phytotoxicity and stem-end, respectively. The weighing coefficients for the PC-2 obtained by using images from VIS region are shown in Fig. 6 c and d, respectively. The peaks and valleys indicated the dominant wavelengths. Therefore, six wavebands from 550 to 900 nm were chosen, which were centered at around 630, 691, 769, 786, 810 and 875 nm, respectively. In addition, two wavelengths at around 691 and 769 nm from VIS region were also singled out for further analysis. 3.5 PCA on selected optimal wavebands The principal components analysis was first carried out on the six optimal wavelengths (630, 691, 769, 786, 810 and 875 nm) instead of the full wavelength range. The first five PC images are shown in Fig. 7 . A visual inspection of the five PC images revealed the major features such as defected regions and stem-ends became more evident in these transformed images except for PC-1 and PC-5. The useful image features were enhanced from PC-2 to PC-4. Since each PC image shown in Fig. 7 is a linear sum of the original images at six optimal wavelengths multiplied by corresponding weighing coefficients, the intensity value of defective areas in the PC images changed with different PC images. As shown in Fig. 7 , the defected regions of fruit surfaces appear dark in PC-2 and PC-3 images, and turn bright in PC-4 images. The surface defects in PC-5 images are not as evident as those in PC-2, PC-3 and PC-4 images because of the relatively low contrasts between the defective regions and the sound skins. The resultant multispectral PC images (i.e., PC-1, PC-2 and PC-3 in Fig. 7 ) obtained from the multispectral PCA also gave similar characteristics in appearance to those (i.e., PC-1, PC-2 and PC-3 in Fig. 4 ) obtained on the full wavelength region. However, the multispectral PC images were more effective to identify defects due to decreasing noises from too many bands. In addition, less spectral bands were desired to develop robust and rapid multispectral imaging systems suitable for skin defects detection. The resultant PC-2, PC-3 and PC-4 images shown in Fig. 7 clearly demonstrated that they all may be used to segment defects. However, compared to PC-3 images, the PC-2 and PC-4 were more sensitive to illumination variations. PC-2, for example, the lighter region (center portions) of original images appeared lower intensities than other part after performing the PCA. The sensibility suggested that the PC-2 and PC-4 may be inadequate for very reliable detection of various skin defects. Based on the inspection and the comparisons above, the PC-3 images showed great potential for discriminating various common orange peel defects from sound skins. Therefore, PC-3 images were chosen for performing the images classification. Fig. 8 illustrates the second principal component images (PC-2) obtained by PCA of hyperspectral reflectance images using two selected wavebands (691 and 769 nm) images in the VIS region. Because the PC-1 for two wavebands (691 and 769 nm) exhibited similar results in appearance to those of the corresponding PC-1 images shown in Figs. 4, 5 and 7 , only PC-2 images are shown in Fig. 8 . Based on visual assessment, the PC-2 ( Fig. 8 ) appeared to also provide the best orange peel defects detection regardless of defect types and illumination variations (Note that a dome lamp can be effective to prevent the effect from illumination variations, which is not the aim of this work). Therefore, PC-2 ( Fig. 8 ) images were also chosen for further analysis. 3.6 Band ratio images for stem-ends identification In the PC images (i.e., PC-2, PC-3 and PC-4 in Fig. 7 or PC-2 in Fig. 8 ), it was very easy to observe that the characteristics of stem-ends resembled those of the peel defects. Therefore, the stem-ends could be misclassified as defects (false positive). Based on the spectral responses in Fig. 3 , the two-band ratio using 689 and 810 nm pair or 689 and 875 nm pair could be potential to detect stem-ends because of the reflectance spectra of stem-ends region exhibiting local minimum value at 689 nm, and local maximum values at 810 and 875 nm. To develop robust and rapid multispectral imaging systems suitable for skin defects detection, less spectral bands were desired. On the one hand, the wavelength at 689 nm was replaced by a wavelength at 691 nm. On the other hand, two-band ratio images based on wavelengths at 691 and 769 nm were also observed. The ratio images were obtained and shown in Fig. 9 . Compared to the resultant images, the R769/R691 and R810/R691 ratio images were ineffective in producing images for distinguishing the stem-end from most surface defects. On the contrary, the R875/R691 ratio images showed clear stem-end region (white area in Fig. 9 ), which suggested much promise in the detection of stem-ends because of most peel defects regions exhibiting darker features. In the study, it was found that sometimes canker spots and phytotoxicity regions could give similar intensity value to stem-end in R875/R691 ratio images due to the two types of defects usually appearing slightly green skin characteristic. However, in most situations, the stem-ends were significantly lighter than canker spots and phytotoxicity region in the R875/R691 ratio images due to greener color characteristic for stem-ends region. 3.7 Defects detection algorithm Fig. 10 demonstrates major procedures for multispectral images processing and detection for thrips scarring identification using an orange sample with stem-end and thrips scarring scar on the fruit. Based on analysis in Section 3.5 and 3.6, two wavebands images at 691 nm and 769 nm and two wavebands images at 691 nm and 875 nm are effective for detection of defects and identification of stem-end, respectively. Therefore, three wavebands (691, 769 and 875 nm) images were used in Fig. 10 . Firstly, a mask template was created using a single-band image at 750 nm, and three wavebands (691, 769 and 875 nm) images were masked using the template to exclude the background that could affect the results. Then, on the one hand, the images at 691 and 875 nm were first used to produce ratio image (R875/R691). Afterwards, a global threshold value of 0.8 was applied to the ratio image to separate stem-end from the fruit surface. On the other hand, the PCA based on two images at 691 and 769 nm was also performed to generate two PC images (PC-1 and PC-2). Subsequently, the addition operation was performed between stem-end binary image and PC-2 image to increase stem-end gray value (stem-end gray compensation). Finally, the compensated PC-2 image was subjected to a simple thresholding method with a global threshold value of 0.23 to separate the defective skins from the normal fruit surfaces. Note that flow charts of the key steps for defects detection algorithm based on PCA from six (630, 691, 769, 786, 810 and 875 nm) and two (691 and 769 nm) optimal wavelengths are similar. The only difference is if the PC images are generated by six wavebands, the addition operation was performed using stem-end binary image and resultant PC-3 image. In some cases, there are some very minor defects/blemishes (noises) on normal orange surfaces. Because these noises may carry the same spectral signature as defects detected in this work, some normal orange could be misidentified as defected oranges. In order to overcome this problem, one step morphological opening operation (Erosion and Dilation) and two steps dilation operation for stem-end binary images and single one step morphological opening operation for defect binary images was used to remove these noises ( Fig. 10 ). Note that two steps dilation operation for stem-end binary image is used in order to compensate the stem-end as completely as possible in addition operation. The resultant binary image with segmented defects is shown at the left bottom image in Fig. 10 . Two global threshold values of 0.8 and 0.23 used in the algorithm was chosen based on spatial profile plots for stem-end and defects. Fig. 11 shows R875/R691 ratio image and compensated PC-2 image in Fig. 10 . The plots adjacent to these two images illustrate spatial variations in intensity values (percent reflectance) along the white lines in the images, respectively. The dotted horizontal lines in the profile plots represent threshold lines for stem-end and thrips scarring segmentation, respectively. Note that although only one representative orange with stem-end and defect are shown in Fig. 11 , selection of threshold values are generally based on the results observed in all training samples. In terms of stem-ends identification, canker spots and phytotoxicity for R875/R691 images sometimes could be misclassified as stem-ends if only intensity information was used. However, the area of region infected by phytotoxicity is usually significantly bigger than stem-end area and the number of canker spots on the orange surface is more than one. Thus, the size and number of marked region for stem-end binary image, as two characteristics, can be used to further identify stem-end. Firstly, the stem-end binary image ( Fig. 10 ) was obtained and marked. If only one region in the stem-end binary image was marked and its size was less than 278 pixels (Note that a stem-end region vary in size from 164 to 275 pixels in this research), this region was classified as stem-end. Then, the addition operation for stem-end gray value compensation was performed before defects segmentation. Otherwise, any other case, the addition operation was not performed and corresponding PC images (PC-3 for six bands and PC-2 for two bands) directly were used to segment defects. By doing this, although there was a very low percentage of false detections for samples with canker spots and phytotoxicity (Note that the false detections are acceptable due to only occurring in defective fruit), the classification procedure above-mentioned can be simplified. An example of analysis performed by the proposed approach for each peel type is shown in Fig. 12 . The illustrated nine samples were randomly selected from test set. In order to obtain more distinct contrast, RGB images of fruit were showed in the first row, and the defective regions were marked manually using a graphical tool. Second row shows the two-band ratio images (R875/R691). Third and fourth rows correspond to PC-2 images obtained using the two selected wavebands (691 and 769 nm) in the VIS region and PC-3 images obtained using six selected wavebands (630, 691, 769, 786, 810 and 875 nm) in the full wavelength region, respectively. Fifth row corresponds to binary images obtained using developed algorithm based on two-band ratio (R875/R691) images and PC-2 images. Sixth row corresponds to binary images obtained using developed algorithm based on two-band ratio (R875/R691) images and PC-3 images. It can be noticed that defects positions in the RGB images are different with those in the PC and ratio images because the fruit was imaged using the different cameras (hyperspectral imaging cameras and RGB cameras) in different inspection chambers, so moved from one image to other but presenting the same defects images for each type. In the binary images, the normal fruit surfaces and stem-end were converted to zero, and the remaining white regions represented defects isolated from the normal fruit surfaces, showing the effectiveness of the proposed classification algorithm for defects detection. 3.8 Identification results The algorithms for multispectral image processing and classification described above were evaluated using 270 independent samples. The test results for different types of defects are shown in Table 1 . A total of 270 fruit samples were divided into two classes: ‘Defected’ class including 240 samples with eight types of skin defects, 30 samples for each type, and ‘Normal’ class including 15 samples with stem-end and 15 samples without stem-end. All samples were evaluated using two detection methods: one was two-band ratio (R875/R691) and PCA using two wavelengths (691 and 769 nm), the other was two-band ratio (R875/R691) and PCA using six wavelengths (630, 691, 769, 786, 810 and 875 nm). As shown in Table 1 , the overall detection accuracy for the tested samples was 93.7% and 91.5% with no false positives (0 out of 30 normal oranges) based on two methods, respectively. In terms of the first detection method, all ‘Scale infestation’, ‘Copper burn’ and ‘Normal’ samples were correctly identified under T 1 = 0.8, T 2 = 0.23. Note that the T 1 and T 2 represent the threshold values applied to two-band ratio images and PC images (see Fig. 11 ), respectively. Seventeen samples were undetected, including two ‘Insect damage’ samples, three ‘Wind scarring’ samples, two ‘Thrips scarring’ samples, two ‘Canker spot’ samples, one ‘Phytotoxicity’ sample, and seven ‘Heterochromatic stripe’ samples. The reflectance properties for heterochromatic stripe was close to those of normal skin (see Fig. 3 ), especially for the lightly heterochromatic stripe. That probably is the reason for the lowest identification rate for this case. However, if identification accuracy is not required to be very high, fruit with heterochromatic stripe is usually considered as sound fruit. For undetected wind scarring, canker spot, one insect damage, and phytotoxicity samples, the lightly defects presented on orange surfaces. After PCA was performed, the contrast between defective areas and sound skins is lower. In this study, we also attempted to increase threshold value T 2 from 0.23 to 0.39 in order to improve detection rate of defective oranges. However, although the defects detection rate was improved from 92.9% to 95.4%, it was also found in this investigation that false positive rate was increased from 0% to 3.3% along with increasing the threshold value, showing the selection of threshold values is very important in defects detection. The cases of incorrect detection of thrips scarring and other insect damage are different: what happened here is that these damages located on the border of the fruit in the detected image, indicating that the identification algorithm could be affected by the position of defected regions on fruit surface in the detected image. Therefore, the accuracy for the identification using the simple global thresholding method may be decreased for detecting the orange samples when defects located on the border of the fruit. In general, two strategies could be potential to resolve this problem. One method proposed in many literatures was performed by simply not inspecting the borders of the fruit in the images. Not inspecting the borders, however, does not imply losing efficacy, since in the in-line automatic computer vision systems for fruit inspection, the fruits rotate, thus allowing the system to inspect most of the fruit surfaces by acquiring different images while they pass below the camera ( Blasco et al., 2007a ). The other method was that local threshold value was applied instead of global threshold method. In order to use this principle, the fruit detected was first segmented into two parts, border region (circular region) and remaining region (middle region). Then, different threshold values were applied to two regions. 4 Conclusions In this investigation, hyperspectral reflectance images were evaluated for detecting various common defects on the orange surface in the wavelength range between 550 and 900 nm. This investigation illustrated that high spectral dimensionality of reflectance images data were reduced to few optimal wavelengths to form multispectral images by using the PCA method. We identified six wavelengths (630, 691, 769, 786, 810 and 875 nm) and, alternatively, three wavelengths (691 and 769 and 875 nm) could potentially be implemented in multispectral imaging systems for detection of orange peel defects. The two-band ratio and PCA coupled with a simple threshold method achieved the best 93.7% orange surface defects identification accuracy and no false positives. It should also be pointed out that simple two-band ratio (R875/R691) algorithm could be more effective to identify stem-ends from skin defects compared to pattern recognition algorithms, which increase the computational complexity. Although the results shown in this study were further manifestation that multispectral reflectance techniques based on band ratio and PCA method for detection of various common defects on orange surfaces in practical applications may be feasible. However, because of the limited sample size for each type defect and skin defect types, it could not indicate that wavelengths at 691, 769 and 875 nm were the best wavelengths to develop algorithm for all defect types of citrus fruit. Therefore, further research will focus on (1) increasing the sample size and collecting samples with other types of skin defects, such as anthracnose and sooty mold, to make the algorithm more robust; (2) applying this procedure to other citrus fruit such as tangerine, pomelo and lemon; and (3) developing an efficient multispectral imaging system, including developing a better classification algorithm in conjunction with morphological information of defects. In addition, multispectral fluorescence imaging techniques will be also tried as aid-system for detection of surface defects in citrus. Acknowledgements The authors gratefully acknowledge the financial support provided by National Natural Science Foundation of China (No. 30825027 ). References Aleixos et al., 2002 N. Aleixos J. Blasco F. Navarrón E. Moltó Multispectral inspection of citrus in real-time using machine vision and digital signal processors Computers and Electronics in Agriculture 33 2 2002 121 137 Ariana et al., 2006 D. Ariana R. Lu D.E. Guyer Hyperspectral reflectance imaging for detection of bruises on pickling cucumbers Computers and Electronics in Agriculture 53 1 2006 60 70 Benneden and Peterson, 2005 B.S. Benneden D.L. Peterson Performance of a system for apple surface defect identification in near-infrared images Biosystems Engineering 90 4 2005 419 431 Blasco et al., 2007a J. Blasco N. Aleixos E. Moltó Computer vision detection of peel defects in citrus by means of a region oriented segmentation algorithm Journal of Food Engineering 81 3 2007 535 543 Blasco et al., 2007b J. Blasco N. Aleixos J. Gómez E. Moltó Citrus sorting by identification of the most common defects using multispectral computer vision Journal of Food Engineering 83 3 2007 384 393 Diaz et al., 2004 R. Diaz L. Gil C. Serrano M. Blasco E. Moltó J. Blasco Comparison of three algorithms in the classification of table olives by means of computer vision Journal of Food Engineering 61 1 2004 101 107 ElMasry et al., 2008 G. ElMasry N. Wang C. Vigneault J. Qiao A. ElSayed Early detection of apple bruises on different background colors using hyperspectral imaging LWT – Food Science and Technology 41 2 2008 337 345 ElMasry et al., 2009 G. ElMasry N. Wang C. Vigneault Detecting chilling injury in Red Delicious apple using hyperspectral imaging and neural networks Postharvest Biology and Technology 52 1 2009 1 8 Gómez-Sanchis et al., 2004 Gómez-Sanchis, J., Moltó, E., Gomez-Chova, L., Aleixos, N., Camps-Valls, G., Juste, F., 2004. Hyperspectral computer vision system for the detection of Penicillium digitatum in citrus packing lines. In: 2004 CIGR International Conference, Beijing, China, 11–14 October, 2004. Gómez-Sanchis et al., 2008 J. Gómez-Sanchis L. Gómez-Chova N. Aleixos G. Camps-Valls C. Montesinos-Herrero E. Moltó J. Blasco Hyperspectral system for early detection of rottenness caused by Penicillium digitatum in mandarins Journal of Food Engineering 89 1 2008 80 86 Heia et al., 2007 K. Heia A. Sivertsen S. Stormo E. Elvevoll J. Wold H. Nilsen Detection of nematodes in Cod ( Gadus morhua ) fillets by imaging spectroscopy Journal of Food Engineering 72 1 2007 E011 E015 Hernández-Andrés et al., 2001 J. Hernández-Andrés J. Romero J.L. Nieves Color and spectral analysis of daylight in southern Europe Journal of the Optical Society of America a-Opticals Image Science and Vision 18 6 2001 1325 1335 Kim et al., 2002 M.S. Kim A.M. Lefcourt K. Chao Y.R. Chen I. Kim D.E. Chan Multispectral detection of fecal contamination on apples based on hyperspectral imagery: part I. Application of visible and near-infrared reflectance imaging Transactions of the ASAE 45 6 2002 2027 2037 Kim et al., 2005 M.S. Kim A.M. Lefcourt Y.R. Chen T. Yang Automated detection of fecal contamination of apples based on multispectral fluorescence image fusion Journal of Food Engineering 71 1 2005 85 91 Kim et al., 2009 D.G. Kim T.F. Burks J.W. Qin D.M. Bulanonm Classification of grapefruit peel diseases using color texture feature analysis International Journal of Agricultural and Biological Engineering 2 3 2009 41 50 Lawrence et al., 2006 K.C. Lawrence W.R. Windham B. Park G.W. Heitschmidt D.P. Smith P. Feldner Partial least squares regression of hyperspectral images for contaminant detection on poultry carcasses Journal of Near Infrared Spectroscopy 14 4 2006 223 230 Lawrence et al., 2008 K.C. Lawrence S.C. Yoon G.W. Heitschmidt D.R. Jones B. Park Imaging system with modified-pressure chamber for crack detection in shell eggs Sensing and Instrumentation for Food Quality and Safety 2 3 2008 116 122 Leemans et al., 1998 V. Leemans H. Magein M.F. Destain Defects segmentation on ‘Golden Delicious’ apples by using colour machine vision Computers and Electronics in Agriculture 20 2 1998 117 130 Leemans and Destain, 2004 V. Leemans M.F. Destain A real-time grading method of apples based on features extracted from defects Journal of Food Engineering 61 1 2004 83 89 Li et al., 2010 J.B. Li X.Q. Rao J.X. Guo Y.B. Ying Hyperspectral reflectance imaging for detecting citrus canker based on dual-band ratio image classification method Proceedings of SPIE 7656 2010 1 10 Liu et al., 2007 Y.L. Liu Y.R. Chen M.S. Kim D.E. Chan A.M. Lefcourt Development of simple algorithms for the detection of fecal contaminants on apples from visible/near infrared hyperspectral reflectance imaging Journal of Food Engineering 81 2 2007 412 418 López-García et al., 2010 F. López-García G. Andreu-García J. Blasco N. Aleixos J. Valiente Automatic detection of skin defects in citrus fruits using a multivariate image analysis approach Computers and Electronics in Agriculture 71 2 2010 189 197 Malinowski and Howery, 1980 Malinowski, E.R., Howery, D.G., 1980. Factor Analysis in Chemistry. John Wiley & Sons, New York. Mehl et al., 2002 P.M. Mehl K. Chao M. Kim Y.R. Chen Detection of defects on selected apple cultivars using hyperspectral and multispectral image analysis Applied Engineering in Agriculture 18 2 2002 219 226 Mendoza and Aguilera, 2004 F. Mendoza J.M. Aguilera Application of image analysis for classification of ripening bananas Journal of Food Science 69 9 2004 E471 E477 Miller and Delwiche, 1991 B.K. Miller M.J. Delwiche Peach defect detection with machine vision Transactions of the ASAE 34 6 1991 2588 2597 Polder et al., 2003 G. Polder W.A.M. Gerie D.H. Van Calibration and characterization of imaging spectrographs Near Infrared Spectroscopy 11 3 2003 193 210 Qiao et al., 2007 J. Qiao M. Ngadi N. Wang C. Gariepy S. Prasher Pork quality and marbling level assessment using a hyperspectral imaging system Journal of Food Engineering 83 1 2007 10 16 Qin and Lu, 2005 J. Qin R. Lu Detection of pits in tart cherries by hyperspectral transmission imaging Transactions of the ASAE 48 5 2005 1963 1970 Qin et al., 2008 J.W. Qin T.F. Burks M.S. Kim K.L. Chao M.A. Ritenour Detecting citrus canker by hyperspectral reflectance imaging and PCA-based image classification method Proceedings of SPIE 6983 2008 1 11 Qin et al., 2009 J.W. Qin T.F. Burks M.A. Ritenour W.G. Bonn Detection of citrus canker using hyperspectral reflectance imaging with spectral information divergence Journal of Food Engineering 93 2 2009 183 191 Recce et al., 1998 M. Recce A. Plebe J. Taylor G. Tropiano Video grading of oranges in real time Artificial Intelligence Review 12 1 1998 117 136 Schubert et al., 2001 T.S. Schubert S.A. Rizvi X. Sun T.R. Gottwald J.H. Graham W.N. Dixon Meeting the challenge of eradicating citrus canker in Florida-Again Plant Disease 85 4 2001 340 356 Skoglund et al., 2004 A. Skoglund W. Kessler R.W. Kessler On-line spectroscopic measurements of wood chips before a continuous digester Chemometrics and Intelligent Laboratory Systems 70 2 2004 129 135 Vargas et al., 2005 A.M. Vargas M.S. Kim Y. Tao A.M. Lefcourt Y.R. Chen Y. Luo Y. Song R. Buchanan Defection of fecal contamination on cantaloupes using hyperspectral fluorescence imagery Journal of Food Science 70 8 2005 471 476 Wallays et al., 2009 C. Wallays B. Missotten J.D. Baerdemaeker W. Saeys Hyperspectral waveband selection for on-line measurement of grain cleanness Biosystems Engineering 104 1 2009 1 7 Xing et al., 2005 J. Xing C. Bravo P. Jancsók H. Ramon J. De Baerdemaeker Detecting bruises on ‘Golden Delicious’ apples using hyperspectral imaging with multiple wavebands Biosystems Engineering 90 1 2005 27 36