“天网”中的并行与分布处理*
赵江华 闫宏飞 王建勇 李晓明
北京大学计算机科学技术系网络与分布式系统实验室 北京 100871
{zhaojh,yhf,lxm}@net.cs.pku.edu.cn
摘 要 本文介绍了“天网”搜索引擎系统2.0版本中所采用的分布并行处理系统结构和关键技术，主要包括搜集子系统和查询子系统两个部分。通过对设计、模拟、实现和实际运行评测等相关过程的描述，展现了系统在性能、负载平衡和扩展性等方面的性质和特点。
关键词 WWW，搜索引擎，信息检索，分布式系统，并行处理
Parallel and Distributed Processing in WebGather
ZHAO Jianghua YAN Hongfei WANG Jianyong LI Xiaoming
Networking & Distributed Computer Systems Laboratory
Department of Computer Science and Technology, Peking University, Beijing 100871
{zhaojh, yhf, lxm}@net.cs.pku.edu.cn
Abstract This paper presents parallel and distributed architectures and key technologies employed in crawling subsystem and retrieving subsystem of WebGather search engine version 2.0. Characteristics of the system, in terms of performance, load balance and scalability, are shown through design, simulation, implementation, and actual running measurement.
Keywords WWW, search engine, information retrieving, distributed system, parallel processing
