Journal of Software:2020.31(9):2785-2801

(浙江工业大学 信息工程学院, 浙江 杭州 310023)
Black-box Adversarial Attack Against Road Sign Recognition Model via PSO
CHEN Jin-Yin,CHEN Zhi-Qing,ZHENG Hai-Bin,SHEN Shi-Jing,SU Meng-Meng
(School of Information Engineering, Zhejiang University of Technology, Hangzhou 310023, China)
Chart / table
Similar Articles
Article :Browse 1101   Download 800
Received:July 03, 2019    Revised:August 18, 2019
> 中文摘要: 随着深度学习在计算机视觉领域的广泛应用,人脸认证、车牌识别、路牌识别等也随之呈现商业化应用趋势.因此,针对深度学习模型的安全性研究至关重要.已有的研究发现:深度学习模型易受精心制作的包含微小扰动的对抗样本攻击,输出完全错误的识别结果.针对深度模型的对抗攻击是致命的,但同时也能帮助研究人员发现模型漏洞,并采取进一步改进措施.基于该思想,针对自动驾驶场景中的基于深度学习的路牌识别模型,提出一种基于粒子群优化的黑盒物理攻击方法(black-box physical attack via PSO,简称BPA-PSO).BPA-PSO在未知模型结构的前提下,不仅可以实现对深度模型的黑盒攻击,还能使得实际物理场景中的路牌识别模型失效.通过在电子空间的数字图像场景、物理空间的实验室及户外路况等场景下的大量实验,验证了所提出的BPA-PSO算法的攻击有效性,可发现模型漏洞,进一步提高深度学习的应用安全性.最后,对BPA-PSO算法存在的问题进行分析,对未来的研究可能面临的挑战进行了展望.
Abstract:With the wider application of deep learning in the field of computer vision, face authentication, license plate recognition, and road sign recognition have also presented commercial application trends. Therefore, research on the security of deep learning models is of great importance. Previous studies have found that deep learning models are vulnerable to carefully crafted adversarial examples that contains small perturbations, leading completely incorrect recognition results. Adversarial attacks against deep learning models are fatal, but they can also help researchers find vulnerabilities of models and make further improvements. Motivated by that, this study proposes a black box physical attack method based on particle swarm optimization (BPA-PSO) for deep learning road sign recognition model in scenario of autonomous vehicles. Under the premise of unknown model structure, BPA-PSO can not only realize the black box attack on deep learning models, but also invalidate the road sign recognition models in the physical scenario. The attack effectiveness of BPA-PSO algorithm is verified through a large number of experiments in the digital images of electronic space, laboratory environment, and outdoor road conditions. Besides, the abilities of discovering models' vulnerabilities and further improving the application security of deep learning are also demonstrated. Finally, the problems existing in the BPA-PSO algorithm are analyzed and possible challenges of future research are proposed.
文章编号:     中图分类号:TP18    文献标志码:
基金项目:浙江省自然科学基金(LY19F020025);国家重点研发计划(2018AAA0100800);宁波市“科技创新2025”重大专项(2018B10063);浙江省认知医疗工程技术研究中心(2018KFJJ07) 浙江省自然科学基金(LY19F020025);国家重点研发计划(2018AAA0100800);宁波市“科技创新2025”重大专项(2018B10063);浙江省认知医疗工程技术研究中心(2018KFJJ07)
Foundation items:Zhejiang Provincial Natural Science Foundation of China (LY19F020025); National Key Research and Development Program of China (2018AAA0100800); Major Special Funding for "Science and Technology Innovation 2025" in Ningbo (2018B10063); Engineering Research Center of Cognitive Healthcare of Zhejiang Province (2018KFJJ07)
Reference text:


CHEN Jin-Yin,CHEN Zhi-Qing,ZHENG Hai-Bin,SHEN Shi-Jing,SU Meng-Meng.Black-box Adversarial Attack Against Road Sign Recognition Model via PSO.Journal of Software,2020,31(9):2785-2801