目 录
第一章 绪论 ........................................................................................................................1
1.1 课题背景及意义 ...................................................................................................1
1.2 国内外研究现状 ...................................................................................................1
1.3 课题研究内容 .......................................................................................................2
第二章 开发技术与工具 ....................................................................................................3
2.1 基于 Scrapy 框架 python 的全国高考数据爬虫及数据分析简介...................3
2.2 Pycharm 简介........................................................................................................3
2.3 Scrapy 简介 ...........................................................................................................3
第三章 项目和数据库的设计 ............................................................................................5
3.1 项目设计 ...............................................................................................................5
3.2 数据库设计 ...........................................................................................................5
3.2.1 数据库设计 ................................................................................................5
3.2.2 数据表设计 ................................................................................................6
第四章 Scrapy 框架爬虫程序实现 ....................................................................................8
4.1 数据爬取的实现说明 ...........................................................................................8
4.2 数据爬取的实现过程 ...........................................................................................8
4.2.1 Scrapy 框架的创建 .................................................................................10
4.2.2 collegebacth 表数据获取实现...............................................................11
4.2.3 writtenscore 表数据获取实现 ...............................................................12
4.2.4 collegedata 表数据获取实现.................................................................14
4.2.5 majorscore 表数据获取实现 .................................................................15
4.2.6 College_number 表格数据获取实现.....................................................16
4.2.7 College_percentage 表格数据获取实现 ...............................................17
4.3 数据可视化与数据挖掘的实现 .........................................................................19
4.3.1 数据可视化与数据挖掘的实现说明 ....................................................19
4.3.1 数据可视化的实现 ................................................................................19
4.2.2 数据挖掘的实现 ....................................................................................23
第五章 数据分析报告 ......................................................................................................25
5.1 数据分析报告说明 ..............................................................................................25