Kernel method is an effective and popular trick in machine learning, and small world network is a common phenomenon which exists widely in social fields. In this paper, by introducing them into Hattori et al’s multi-module associative memory for many-to-many associations ((MMA)²), a unified framework of small world structure inspired many-to-many kernel associative memory models (SWSI-M²KAMs) is proposed. The SWSI-M²KAMs not only can store patterns online without more iteration steps, but also extend the range of the processed intelligent information. More importantly, the SWSI-M²KAMs framework can develop more new many-to-many associative memory models by selecting different kernel functions and reduce models’ configuration complexity by using the sparse small world architecture. Finally, computer simulations demonstrate that the constructed models have good performance on many-to-many associative memory.