| 引用本文: | 钱亚冠,马骏,何念念,王滨,顾钊铨,凌祥,Wassim Swaileh.面向边缘智能的两阶段对抗知识迁移方法.软件学报,2022,33(12):4504-4516 |
| |
|
| 本文已被：浏览 1605次 下载 3214次 |
码上扫一扫！ |
|
|
| 面向边缘智能的两阶段对抗知识迁移方法 |
|
钱亚冠1, 马骏1, 何念念1, 王滨2, 顾钊铨3, 凌祥4, Wassim Swaileh5
|
|
1.浙江科技学院 大数据学院, 浙江 杭州 310023;2.杭州海康威视网络与信息安全实验室, 浙江 杭州 310052;3.广州大学 网络空间先进技术研究院, 广东 广州 510006;4.浙江大学 计算机科学与技术学院, 浙江 杭州 310058;5.CY Cergy Paris University, ETIS Research Laboratory, Paris 95032
|
|
| 摘要: |
| 对抗样本的出现，对深度学习的鲁棒性提出了挑战.随着边缘智能的兴起，如何在计算资源有限的边缘设备上部署鲁棒的精简深度学习模型，是一个有待解决的问题.由于精简模型无法通过常规的对抗训练获得良好的鲁棒性，提出两阶段对抗知识迁移的方法，先将对抗知识从数据向模型迁移，然后将复杂模型获得的对抗知识向精简模型迁移.对抗知识以对抗样本的数据形式蕴含，或以模型决策边界的形式蕴含.具体而言，利用云平台上的GPU集群对复杂模型进行对抗训练，实现对抗知识从数据向模型迁移；利用改进的蒸馏技术将对抗知识进一步从复杂模型向精简模型的迁移，最后提升边缘设备上精简模型的鲁棒性.在MNIST，CIFAR-10和CIFAR-100这3个数据集上进行验证，实验结果表明：提出的这种两阶段对抗知识迁移方法可以有效地提升精简模型的性能和鲁棒性，同时加快训练过程的收敛性. |
| 关键词: 对抗样本 对抗训练 知识迁移 知识蒸馏 |
| DOI：10.13328/j.cnki.jos.006352 |
| 分类号:TP182 |
| 基金项目:浙江省自然科学基金（LY17F020011）；国家重点研发计划（2018YFB2100400）；国家自然科学基金（61902082）； |
|
| Two-stage Adversarial Knowledge Transfer for Edge Intelligence |
|
QIAN Ya-Guan1, MA Jun1, HE Nian-Nian1, WANG Bin2, GU Zhao-Quan3, LING Xiang4, Wassim Swaileh5
|
|
1.School of Big Data Science, Zhejiang University of Science and Technology, Hangzhou 310023, China;2.Network and Information Security Laboratory of Hangzhou Hikvision Digital Technology Co. Ltd., Hangzhou 310052, China;3.Cyberspace Institute of Advanced Technology (CIAT), Guangzhou University, Guangzhou 510006, China;4.College of Computer Science and Technology, Zhejiang University, Hangzhou 310058, China;5.CY Cergy Paris University, ETIS Research Laboratory, Paris 95032, France
|
| Abstract: |
| The emergence of adversarial examples brings challenges to the robustness of deep learning. With the development of edge intelligence, how to train a robust and compact deep learning mode on edge devices with limited computing resources is also a challenging problem. Since compact models cannot obtain sufficient robustness through conventional adversarial training, a method called two-stage adversarial knowledge transfer is proposed. The method transfers adversarial knowledge from data to models and complex models to compact models. The so-called adversarial knowledge has two forms, one is contained in data with the form of adversarial examples, and the other is contained in models with the form of decision boundary. The GPU clusters of cloud center is first leveraged to train the complex model with adversarial examples to realize the transfer of adversarial knowledge from data to models, and then an improved distillation approach is leveraged to realize the further transfer of adversarial knowledge from complex models to compact models on edge nodes. The experiments over MNIST and CIFAR-10 show that this two-stage adversarial knowledge transfers can efficiently improve the robustness and convergence of compact models. |
| Key words: adversarial examples adversarial training knowledge transfer knowledge distillation |
