| 本文已被：浏览 4127次 下载 6756次 |
码上扫一扫！ |
|
|
| DF还是IDF?主特征模型在Web信息检索中的使用 |
|
张敏1,2, 马少平1,2, 宋睿华1,2
|
|
1.清华大学,计算机科学与技术系,北京,100084;2.清华大学,智能技术与系统国家重点实验室,北京,100084
|
|
| 摘要: |
| Web信息检索的难点之一就是简短、模糊的用户查询与存在大量冗余和噪声的文档之间的不匹配.对Web文档信息特征进行分析,提出Web文档主特征词、主特征域和主特征空间的概念,在该空间上使用文档频度DF(document frequency)信息而非传统意义上的IDF(inverse document frequency)信息进行权值计算,并给出一个改进的相似度计算模型.使用该模型在10G和19G的两个大规模Web文档集合上进行了3组标准测试.比较实验表明,与传统IDF思想相比,在各项评价指标上,DF相关的主特征权值计算方法都能始终较大幅度地提高系统性能,最大达到18.6%的性能改善. |
| 关键词: Web信息检索 主特征模型 权值计算 文档频度 |
| DOI： |
| 分类号: |
| 基金项目:Supported by the National Natural Science Foundation of China under Grant Nos.60223004,60321002,60303005(国家自然科学基金) |
|
| DF or IDF? On the Use of Primary Feature Model for Web Information Retrieval |
|
ZHANG Min,MA Shao-Ping,SONG Rui-Hua
|
| Abstract: |
| In Web information retrieval (IR), input queries are too short and fuzzy to describe user request, which leads to the mismatch problem between user query and the documents full of redundancy and noise. This paper first studies the feature of web documents information and proposes the concepts of primary feature word, primary feature field and primary feature space (PFS). Then a new PFS query term weighting scheme is proposed, which takes document frequency (DF) into account instead of the traditional IDF factor. Finally, a combination strategy of term weighting is given. Using this PFS model, three groups of experiments have been performed on 10G and 19G large scale Web collections with TREC9, TREC10 and TREC11 standard tests of Web tracks. Comparative studies indicate that the new DF-related PFS term weighting improves the system performance consistently and effectively in terms of recall, top n precision and mean average precision. At most 18.6% improvement has been made. |
| Key words: Web information retrieval primary feature model term weighting document frequency |
