Journal of Software:2016.27(9):2303-2319

(东北大学 计算机科学与工程学院, 辽宁 沈阳 110819)
Entity Resolution Oriented Clustering Algorithm
SUN Chen-Chen,SHEN De-Rong,KOU Yue,NIE Tie-Zheng,YU Ge
(College of Computer Science and Engineering, Northeastern University, Shenyang 110819, China)
Chart / table
Similar Articles
Article :Browse 1476   Download 1841
Received:September 24, 2015    Revised:January 12, 2016
> 中文摘要: 实体识别是数据质量的一个重要方面,对于大数据处理不可或缺.已有的实体识别研究工作聚焦于数据对象相似度算法、分块技术和监督的实体识别技术,而非监督的实体识别中匹配决定的问题很少被涉及.提出一种面向实体识别的聚类算法来弥补这个缺失.利用数据对象及其相似度构建带权重的数据对象相似图.聚类过程中,利用相似图上重启式随机游走来动态地计算类簇与结点的相似度.聚类的基本逻辑是,类簇迭代地吸收离它最近的结点.提出数据对象排序方法来优化聚类的顺序,提高聚类精确性;提出了优化的随机游走平稳概率分布计算方法,降低聚类算法开销.通过在真实数据集和生成数据集上的对比实验,验证了该算法的有效性.
Abstract:Entity resolution (ER) is a key aspect of data quality and is necessary for big data processing. Existing ER research focuses on data object similarity algorithms, blocking and supervised ER technologies, but pays little attention to matching decision problems in unsupervised ER. This paper proposes a clustering algorithm for ER to complement existing work. The algorithm builds a weighted similarity graph with data objects and their pairwise similarities. During clustering, the similarity between a cluster and a vertex is dynamically computed via random walk with restarts on the similarity graph. The basic logic behind clustering is that a cluster absorbs the nearest neighbor vertex iteratively. A data object ordering method is also proposed to optimize clustering order, promoting clustering accuracy. Further, an improved computation method of random walk's stationary probability distribution is proposed to reduce cost of the clustering algorithm. The evaluation on real datasets and synthetic datasets validates effectiveness of the proposed algorithm.
文章编号:     中图分类号:    文献标志码:
基金项目:国家自然科学基金(61472070,61402213);国家重点基础研究发展计划(973)(2012CB316201);教育部基本科研业务费项目(N110404010) 国家自然科学基金(61472070,61402213);国家重点基础研究发展计划(973)(2012CB316201);教育部基本科研业务费项目(N110404010)
Foundation items:National Natural Science Foundation of China (61472070, 61402213); National Basic Research Program of China (973) (2012CB316201); Fundamental Research Funds for the Central Universities (N110404010)
Reference text:


SUN Chen-Chen,SHEN De-Rong,KOU Yue,NIE Tie-Zheng,YU Ge.Entity Resolution Oriented Clustering Algorithm.Journal of Software,2016,27(9):2303-2319