| 摘要: |
| 日志数据记录了系统的运行状态、用户行为及错误信息. 基于日志的异常检测可快速识别潜在的安全风险或性能瓶颈, 提升运维效率, 助力故障诊断. 然而, 现有的日志异常检测方法仍面临诸多挑战, 如无法有效适应系统升级引起的日志模式变化, 缺乏高效反馈机制导致难以持续保持检测性能等. 为此, 提出一种日志异常检测框架PCLog, 采用强化学习方法中的近端策略优化(proximal policy optimization, PPO)算法进行模型训练. 该方案将检测模型视为智能体, 日志对应的语义向量视为状态, 事件视为动作, 通过最大化正常序列的累计奖励来学习系统的正常行为模式从而实现异常检测. 此外, 当检测性能下降时, PCLog可通过收集错误预测的样本作为专家示范数据, 并结合模仿学习中的行为克隆方法, 最大化专家数据的对数似然, 从而使模型更有效地逼近专家行为, 实现模型的自适应修正, 有效减少误报率, 提升系统长期运行的可靠性. 在HDFS、BGL与OpenStack这3大公开日志数据集上的实验结果表明, PCLog相较于现有方法表现更优, 具备较强的动态日志模式适应能力. |
| 关键词: 日志异常检测 近端策略优化 行为克隆 |
| DOI：10.13328/j.cnki.jos.007585 |
| 分类号:TP311 |
| 基金项目:湖北省重点研发基金(2025BEB012) |
|
| PCLog: Adaptive Log Anomaly Detection Based on Proximal Policy Optimization and Behavior Cloning |
|
ZHOU Jun-Wei, HU Miao, WANG Chun-Long, DU Ya-Juan, TAN Cheng
|
|
School of Computer Science and Artificial Intelligence, Wuhan University of Technology, Wuhan 430070, China
|
| Abstract: |
| Log data record system operating status, user behavior, and error information. Log-based anomaly detection enables the rapid identification of potential security risks or performance bottlenecks, thereby enhancing operational efficiency and facilitating fault diagnosis. However, existing log anomaly detection methods still face several challenges, including the inability to effectively adapt to log pattern changes caused by system updates and the lack of efficient feedback mechanisms to consistently maintain detection performance. To address these issues, this study proposes a log anomaly detection framework, PCLog, which adopts the proximal policy optimization (PPO) algorithm from reinforcement learning for model training. In the proposed framework, the detection model is formulated as an agent, semantic vectors of logs are regarded as states, and events are treated as actions. By maximizing the cumulative rewards of normal sequences, normal system behavior patterns are learned to enable anomaly detection. In addition, when detection performance decreases, PCLog collects mispredicted samples as expert demonstration data and integrates behavior cloning from imitation learning to maximize the log-likelihood of expert data. This mechanism enables the model to more effectively approximate expert behavior, achieve adaptive self-correction, reduce false positives, and enhance long-term reliability. Experimental results on three public log datasets, HDFS, BGL, and OpenStack, show that PCLog outperforms existing methods and exhibits high adaptability to dynamic log patterns. |
| Key words: log anomaly detection proximal policy optimization (PPO) behavior cloning |
