A Classification Method for Mango Leaf Diseases and Pests by Integrating SKNet and MobilenetV3
-
摘要:
目的 针对芒果叶片病虫害缺少数据集和识别准确率低的问题,提高芒果叶病虫害分类准确率。 方法 提出使用去噪扩散模型进行病虫害数据增强,同时联合SKNet与MobilenetV3模型的芒果叶片病虫害分类方法。首先使用去噪扩散模型对数据集进行扩充,再采用多尺度结构相似性指标对生成的病虫害图像与拍摄的病虫害图像之间的相似程度进行评估,接着对DDIM与DCGAN网络训练和生成效果进行比对。在MobilenetV3模型中,将SE注意力模块替换为SKNet模块进行构建网络模型。 结果 使用DDIM生成的所有类型的病虫害图像与拍摄的病虫害图像的MS-SSIM指标均大于0.63,且都高于DCGAN。相较于其他注意力模块,联合SKNet与MobilenetV3的分类效果最佳,在98%以上。对添加CA、CBAM、ECA注意力模块进行平滑类激活图可视化,对比其他注意力模块,使用SKNet注意力分布区域更为集中在病虫害叶片上。 结论 该方法在病虫害叶片检测上具有良好的应用前景,能提升病虫害识别效率与精度,减少检测成本,同时可应用于移动式或者嵌入式设备。 -
关键词:
- 芒果叶片 /
- 扩散概率模型 /
- Mobilenet /
- Selective Kernel Networks
Abstract: :Objective To provide reference for the classification of mango leaf diseases, addressing the issues of lack of datasets and low recognition accuracy. Method This article proposes the use of denoising diffusion models for disease data augmentation, and innovatively proposes a mango leaf disease classification method that combines SKNet and MobilenetV3 models. Firstly, a denoising diffusion model was used to expand the dataset, and then a multi-scale structural similarity index was used to evaluate the similarity between the generated disease images and the captured disease images. Then, the training and generation effects of DDIM and DCGAN networks were compared. In the Mobilenet model, replace the SE attention module with the SKNet module to construct the network model. Results The MS-SSIM index of all types of disease images generated and captured using DDIM is greater than 0.63 and higher than DCGAN. Compared to other attention modules, the combination of SKNet and MobilenetV3 has the best classification performance, at over 98%. Visualize the smooth class activation diagram of adding CA, CBAM, and ECA attention modules. Compared with other attention modules, the attention distribution area of SKNet is more concentrated on diseased leaves. Conclusion This method has good application prospects in the detection of diseased leaves, which can improve the efficiency and accuracy of disease identification, reduce detection costs, and can be applied to mobile or embedded devices. -
Key words:
- mango leaf /
- diffusion probability model /
- Mobilenet /
- Selective Kernel Networks
-
表 1 MobilenetV3模型结构
Table 1. MobilenetV3 model structure
输入尺寸
Input shape操作算子
Operation扩展尺寸
Expand size输出通道
Output channelSE模块
SE module激活函数
Activation function步长
Stride2242 conv2d,3×3 — 16 — HS 2 1122×16 bneck,3×3 16 16 √ RE 2 562×16 bneck,3×3 72 24 — RE 2 282×24 bneck,3×3 88 24 — RE 1 282×24 bneck,5×5 96 40 √ HS 2 142×40 bneck,5×5 240 40 √ HS 1 142×40 bneck,5×5 240 40 √ HS 1 142×40 bneck,5×5 120 48 √ HS 1 142×48 bneck,5×5 144 48 √ HS 1 142×48 bneck,5×5 288 96 √ HS 2 72×96 bneck,5×5 576 96 √ HS 1 72×96 bneck,5×5 576 96 √ HS 1 72×96 conv2d,1×1 — 576 √ HS 1 72×576 pool,7×7 — — — — 1 12×576 conv2d1×1,NBN — 1280 — HS 1 12×1024 conv2d1×1,NBN — 7 — — 1 注:conv2d—二维卷积;bneck—瓶颈模块;pool—池化层;NBN—不使用批量归一化;HS—硬切线激活函数;RE—修正线性单元;√—使用SE模块。
Note: conv2d—2D Convolution; bneck—Bottleneck Module; pool—Pooling Layer; NBN—No Batch Normalization; HS—Hard Swish Activation Function; RE—ReLU (Rectified Linear Unit); √—Uses SE Module表 2 图片生成前后数据量比较
Table 2. Comparison of data volume before and after image generation
类型
Type原始图数量
original dataset增强后图像数量
Augmented dataset炭疽病 Colletotrichum gloeosporioides 320 1800 细菌性溃疡
Xanthomonas campestris pv. mangiferaeindicae324 1800 切叶象甲 Myllocerus viridanus 320 1800 枯萎病 Fusarium oxysporum 320 1800 癭蚊 Erosomyia mangiferae 544 1800 白粉病 Oidium mangiferae 320 1800 煤污病 Capnodium mangiferae 320 1800 表 3 DDIM与DCGAN网络模型训练指标对比
Table 3. DDIM vs. DCGAN Network Model Training Metrics Comparison
指标 Index DDIM DCGAN 模型大小 Model size/MB 117 89 训练时间 Training time /h 48 36 收敛速度(训练轮次) Convergence speed (epochs) 19 57 训练中损失函数的标准偏差 Standard deviation 0.05 0.15 总耗时(秒) Total time (seconds) 392.6 457.2 表 4 生成的病虫害图像与拍摄的病虫害图像MS-SSIM值
Table 4. MS-SSIM values of generated defect images and captured defect images
图像类型
Image typeMS-SSIM
(DDIM)MS-SSIM
(DCGAN)炭疽病 Colletotrichum gloeosporioides 0.6312 0.5992 细菌性溃疡
Xanthomonas campestris pv. mangiferaeindicae0.7298 0.6912 切叶象甲 Myllocerus viridanus 0.6754 0.6413 枯萎病 Fusarium oxysporum 0.7123 0.6805 癭蚊 Erosomyia mangiferae 0.7459 0.6990 白粉病 Oidium mangiferae 0.7211 0.6853 煤污病 Capnodium mangiferae 0.6671 0.6396 表 5 引入注意力模块的MobilenetV3模型实验对比
Table 5. Experimental comparison of MobilenetV3 model with attention module introduction
算法
Algorithm准确率
Accuracy/%参数量
Params/M乘加运算数
MACs/GMobilenetV3(SE) 97.24 2.54 0.06 MobilenetV3+CA 95.75 2.18 0.06 MobilenetV3+CBAM 97.69 2.59 0.09 MobilenetV3+ECA 97.51 2.08 0.06 MobilenetV3+SKNet 98.21 2.58 0.06 -
[1] 杨芝霓. 芒果主要采后病害的病原鉴定及炭疽病的生物防治[D]. 南宁: 广西大学.YANG Z N. Pathogen identification of mango post havest disease and biological control of anthracnose[D]. Nanning: Guangxi University. (in Chinese) [2] 秦丰, 刘东霞, 孙炳达, 等. 基于图像处理技术的四种苜蓿叶部病害的识别 [J]. 中国农业大学学报, 2016, 21(10):65−75.QIN F, LIU D X, SUN B D, et al. Recognition of four different alfalfa leaf diseases based on image processing technology [J]. Journal of China Agricultural University, 2016, 21(10): 65−75. (in Chinese) [3] 柴阿丽, 李宝聚, 石延霞, 等. 基于计算机视觉技术的番茄叶部病害识别 [J]. 园艺学报, 2010, 37(9):1423−1430.CHAI A L, LI B J, SHI Y X, et al. Recognition of tomato foliage disease based on computer vision technology [J]. Acta Horticulturae Sinica, 2010, 37(9): 1423−1430. (in Chinese) [4] SHI B Z, ZHOU X L, QIN Z K, et al. Corn ear quality recognition based on DCGAN data enhancement and transfer learning[C]//The 4th International Conference on Electronics, Communications and Control Engineering. Seoul Republic of Korea. ACM, 2021. [5] AZIZI S, KORNBLITH S, SAHARIA C, et al. Synthetic data from diffusion models improves imagenet classification [J]. arXiv preprint arXiv:, 2304, 08466: 2023. [6] 哈马友吉, 任万春, 张秤, 等. 基于轻量级网络MobileNet V2的二极管玻壳缺陷识别 [J]. 传感器与微系统, 2022, 41(4):153−155,160.HA M, REN W C, ZHANG C, et al. Defect recognition of diode glass shells based on lightweight network MobileNet V2 [J]. Transducer and Microsystem Technologies, 2022, 41(4): 153−155,160. (in Chinese) [7] 王志强, 于雪莹, 杨晓婧, 等. 基于WGAN和MCA-MobileNet的番茄叶片病害识别 [J]. 农业机械学报, 2023, 54(5):244−252.WANG Z Q, YU X Y, YANG X J, et al. Tomato leaf diseases recognition based on WGAN and MCA-MobileNet [J]. Transactions of the Chinese Society for Agricultural Machinery, 2023, 54(5): 244−252. (in Chinese) [8] LI X, WANG W H, HU X L, et al. Selective kernel networks[J]. IEEE, 2020. DOI: 10.1109/CVPR.2019.00060. [9] AHMED S I, IBRAHIM M, NADIM M, et al. MangoLeafBD: A comprehensive image dataset to classify diseased and healthy mango leaves [J]. Data in Brief, 2023, 47: 108941. doi: 10.1016/j.dib.2023.108941 [10] ZHANGQ, TAOM, CHENY. gDDIM: Generalizeddenoisingdiffusionimplicitmodels [J]. arXiv preprint arXiv:, 2206, 05564: 2022. [11] 闫志浩, 周长兵, 李小翠. 生成扩散模型研究综述 [J]. 计算机科学, 2024, 51(1):273−283.YAN Z H, ZHOU C B, LI X C. Survey on generative diffusion model [J]. Computer Science, 2024, 51(1): 273−283. (in Chinese) [12] WANG Z, SIMONCELLI E P, BOVIK A C. Multiscale structural similarity for image quality assessment[C]//The Thrity-Seventh Asilomar Conference on Signals, Systems & Computers, 2003. Pacific Grove, CA, USA. IEEE, 2003: 1398-1402. [13] HOWARD A G, ZHU M L, CHEN B, et al. MobileNets: Efficient convolutional neural networks for mobile vision applications [J]. arXiv preprint arXiv:, 1704, 04861: 2017. [14] SANDLER M, HOWARD A, ZHU M L, et al. MobileNetV2: inverted residuals and linear bottlenecks[C]//2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition. June 18-23, 2018. Salt Lake City, UT. IEEE, 2018: 4510-4520. [15] HOWARD A, SANDLER M, CHEN B, et al. Searching for MobileNetV3[C]//2019 IEEE/CVF International Conference on Computer Vision (ICCV). October 27-November 2, 2019. Seoul, Korea (South). IEEE, 2019: 1314-1324. [16] 刘勇国, 高攀, 兰荻, 等. ECA-SKNet: 玉米单倍体种子的卷积神经网络识别模型 [J]. 电子科技大学学报, 2023, 52(6):866−871.LIU Y G, GAO P, LAN D, et al. ECA-SKNet: Convolutional neural network identification model for corn haploid seeds [J]. Journal of University of Electronic Science and Technology of China, 2023, 52(6): 866−871. (in Chinese) [17] 何翔. 基于DCGANs的半片光伏组件电致发光图像增强技术 [J]. 应用光学, 2023, 44(2):314−322. doi: 10.5768/JAO202344.0202003HE X. Electroluminescence image enhancement technology of half-cut photovoltaic module based on DCGANs [J]. Journal of Applied Optics, 2023, 44(2): 314−322. (in Chinese) doi: 10.5768/JAO202344.0202003 [18] HOU Q B, ZHOU D Q, FENG J S. Coordinate attention for efficient mobile network design[C]//2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). June 20-25, 2021. Nashville, TN, USA. IEEE, 2021: 13713-13722. [19] WANG Q L, WU B G, ZHU P F, et al. ECA-net: Efficient channel attention for deep convolutional neural networks[C]//2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). June 13-19, 2020. Seattle, WA, USA. IEEE, 2020: 11534-11542. [20] WOO S, PARK J, LEE J Y, et al. CBAM: convolutional block attention module[M]//Computer Vision – ECCV 2018. Cham: Springer International Publishing, 2018: 3-19.