| 摘要: |
| 深网查询在Web上众多的应用，需要查询大量的数据源才能获得足够的数据，如多媒体数据搜索、团购网站信息聚合等.应用的成功，取决于查询多数据源的效率和效果.当前研究侧重查询与数据源的相关性而忽略数据源之间的重叠关系，使得不同数据源上相同结果的数据被重复查询，增加了查询开销及数据源的工作负载.为了提高深网查询的效率，提出一种元组水平的分层抽样方法来估计和利用查询在数据源上的统计数据，选择高相关、低重叠的数据源.该方法分为两个阶段：离线阶段，基于元组水平对数据源进行分层抽样，获得样本数据；在线阶段，基于样本数据迭代地估计查询在数据源上的覆盖率和重叠率，并采用一种启发式策略以高效地发现低重叠的数据源.实验结果表明，该方法能够显著提高重叠数据源选择的精度和效率. |
| 关键词: 数据源选择 分层抽样 数据源重叠率估计 回归 |
| DOI：10.13328/j.cnki.jos.005105 |
| 分类号: |
| 基金项目:国家自然科学基金（61232002，61202035）；湖北省科技支撑计划（2015BAA127） |
|
| Overlapping Deep Web Data Source Selection Based on Stratified Sampling |
|
JIANG Jun-Yan1,2, PENG Zhi-Yong2, WU Xiao-Ying2, PENG Cheng-Chen1, WANG Min1
|
|
1.Computer School, Wuhan University, Wuhan 430072, China;2.State Key Laboratory of Software Engineering (Wuhan University), Wuhan 430072, China
|
| Abstract: |
| Many Web applications, such as multimedia data integration and online business data aggregation, require deep Web querying to integrate information from many data sources on the Web. The success of such applications is largely determined by the efficiency and effectiveness of querying methods over relevant sources. Existing studies on multiple data source integration have focused on ranking the relevance of queries w.r.t data sources without considering the impact of overlap among the sources over data source selection, resulting in not only query processing overhead but also increased workloads on data sources. In order to improve query efficiency on overlapping data sources, this work proposes a tuple-level stratified sampling approach for overlapping data source selection. The approach has two stages: the offline stage and the online stage. In the offline stage, tuple-level stratified sampling is applied to obtain sample tuples. In the online stage, samples are used to estimate query coverage and overlap among multiple data sources. A heuristic method is also designed to discover data sources with low overlap. Experimental results show that the proposed approach is more efficient and effective than the state of the art methods for selecting overlapping data sources. |
| Key words: data source selection stratified sampling tata source overlap estimation regression |
