Log data record system operating status, user behavior, and error information. Log-based anomaly detection enables the rapid identification of potential security risks or performance bottlenecks, thereby enhancing operational efficiency and facilitating fault diagnosis. However, existing log anomaly detection methods still face several challenges, including the inability to effectively adapt to log pattern changes caused by system updates and the lack of efficient feedback mechanisms to consistently maintain detection performance. To address these issues, this study proposes a log anomaly detection framework, PCLog, which adopts the proximal policy optimization (PPO) algorithm from reinforcement learning for model training. In the proposed framework, the detection model is formulated as an agent, semantic vectors of logs are regarded as states, and events are treated as actions. By maximizing the cumulative rewards of normal sequences, normal system behavior patterns are learned to enable anomaly detection. In addition, when detection performance decreases, PCLog collects mispredicted samples as expert demonstration data and integrates behavior cloning from imitation learning to maximize the log-likelihood of expert data. This mechanism enables the model to more effectively approximate expert behavior, achieve adaptive self-correction, reduce false positives, and enhance long-term reliability. Experimental results on three public log datasets, HDFS, BGL, and OpenStack, show that PCLog outperforms existing methods and exhibits high adaptability to dynamic log patterns.