目 录
绪 论 ........................................................................................................................................................1
1 概述 ......................................................................................................................................................2
1.1 研究目的和意义 ...........................................................................................................................2
1.2 网络爬虫的历史 ...........................................................................................................................2
1.3 研究现状和发展趋势 ...................................................................................................................3
2 网络爬虫相关技术 ..............................................................................................................................5
2.1 URL ................................................................................................................................................5
2.2 HTTP 协议 .....................................................................................................................................5
2.2.1 HTTP 协议简介 ......................................................................................................................5
2.2.2 HTTP 消息的结构 ..................................................................................................................6
2.2.3 请求方法 ................................................................................................................................7
2.2.4 HTTP 状态代码 ......................................................................................................................8
2.3 Java 多线程 ....................................................................................................................................8
2.3.1 定义 ........................................................................................................................................8
2.3.2 优点 ........................................................................................................................................8
2.3.3 不足 ........................................................................................................................................8
2.3.4 Java 多线程编程 .....................................................................................................................9
2.4 Java 网络编程 .............................................................................................................................11
2.4.1 TCP/IP ...................................................................................................................................11
2.4.2 网络编程步骤 ......................................................................................................................11
2.4.3 Java 网络编程技术 ...............................................................................................................13
2.5 正则表达式 .................................................................................................................................14
2.5.1 正则表达式特点 ..................................................................................................................14
2.5.2 符号功能 ...............................................................................................................................15
2.6 网页搜索策略 .............................................................................................................................15
2.6.1 广度优先搜索 .......................................................................................................................15
2.6.2 深度优先搜索 .......................................................................................................................16
2.6.3 最佳优先搜索 .......................................................................................................................16
3 需求分析和模型设计 ........................................................................................................................17
3.1 网络爬虫的定义 .........................................................................................................................17
3.2 网络爬虫的分类 ..........................................................................................................................17
