《Python 编程》大作业(论文)
学院(系)：
专 业：
学 生：
学 号：
班 级：
指导教师：
完成日期 2019 年 12 月
南阳理工学院《Python 编程》大作业（论文）
python 爬虫与数据可视化分析
Python crawler and data visualization analysis
Python 编程——爬虫与数据可视化分析
1
python 爬虫与数据可视化分析
软件工程
摘 要
网络爬虫将下载的网页和收集到的网页信息存储在本地数据库中以供搜索引擎使用，它
是一个专门从万维网上下载网页并分析网页的程序。随着网络的快速发展，人们对搜索引擎
的要求也越来越高，而网络爬虫的效率直接影响着搜索引擎的质量。 本课题研究的是通用
网络爬虫，它是从一个或若干个初始网页的链接开始进而得到一个链接队列。伴随着网页的
抓取又不断从抓取到的网页中抽取新链接放入到链接队列中，直到爬虫系统满足了停止条件。
该课题主要涉及到了 HTTP 和 SSL 协议，BeautifulSoup，pyecharts，jieba，Tkinter 的使用等
相关技术。 本说明书叙述的网络爬虫是以 python 实现的，加以 Tkinter 界面使用户更加方
面的操作。
[关键词] python；BeautifulSoup；Tkinter；pyecharts；jieba
Python 编程——爬虫与数据可视化分析
2
Python crawler and data visualization analysis
Software Engineering Major WangMingyang
Abstract: The web crawler stores the downloaded Web pages and the collected web
page information in the local database for the use of search engines. It is a special
program to download web pages from the world wide web and analyze them. With
the rapid development of the Internet, people have higher and higher requirements for
search engine, and the efficiency of web crawler directly affects the quality of search
engine. In this paper, we study the general web crawler, which starts from the links of
one or several initial web pages to get a link queue. With the capture of web pages,
new links are extracted from the captured web pages and put into the link queue until
the crawler system meets the stop condition. This topic mainly involves the HTTP and
SSL protocol, the use of beautiful soup, pyecharts, Jieba, Tkinter and other related
technologies. The web crawler described in this manual is implemented in Python,
and Tkinter interface is used to make users operate in more aspects.
Key word: python；BeautifulSoup；Tkinter；pyecharts；jieba
Python 编程——爬虫与数据可视化分析
3
目 录
1 引言.........................................................................................................................................4
1.1 选题背景及意义.............................................................................................4
1.2 开发目标.........................................................................................................4
2 技术简介.................................................................................................................................4
2.1 系统环境........................................................................................................4
2.2 Tkinter...........................................................................................................4
3 需求分析.................................................................................................................................5
3.1 系统的可行性分析........................................................................................5
3.2 功能性需求分析............................................................................................5
4 概要设计与详细设计.............................................................................................................6
4.1 概要设计........................................................................................................6
5 项目实现.................................................................................................................................8
5.1 查询商品页面.....................................................................................8
5.2 网页数据提取分析.......................................................................................10
5.3 数据分析与数据可视化............................................................................. ..11
5.4 数据可视化结果展示....................................................................................13
6 测试报告...............................................................................................................................15
6.1 测试目的............................................................................................... .........15
6.2 测试技术............................................................................................... .........15
6.3 测试结果............................................................................................... .........16
结束语.......................................................................................................................................16
参考文献...................................................................................................................................16
