Selenium 高级自动化(上海-悠悠、葵花)
目录
第 1 章 环境搭建 ............................................................................................................................. 9
1.1 环境搭建............................................................................................................................ 9
1.1.1 selenium 简介 .......................................................................................................... 9
1.1.2 python 安装 ........................................................................................................... 10
1.1.3 环境变量 ............................................................................................................... 11
1.1.4 安装 selenium ....................................................................................................... 12
1.1.5 验证 selenium ....................................................................................................... 13
1.1.6 浏览器................................................................................................................... 13
1.2 pip 降级 selenium3.0 ........................................................................................................ 14
1.2.1 遇到问题 ............................................................................................................... 14
1.2.2 解决方案 ............................................................................................................... 15
1.2.3 检查 pip 环境 ....................................................................................................... 15
1.2.4 pip 查看 selenium 版本号 ..................................................................................... 16
1.2.5 pip 降级 selenium .................................................................................................. 17
1.2.6 升级 pip 版本 ....................................................................................................... 18
1.3 解决 pip 使用异常........................................................................................................... 19
1.3.1 pip 出现异常 .......................................................................................................... 19
1.3.2 解决办法 ............................................................................................................... 20
1.3.3 配置环境变量 ....................................................................................................... 21
1.4 Chrome 浏览器(chromedriver) ................................................................................... 22
1.4.1 Chrome 遇到问题 .................................................................................................. 22
1.4.2 查看版本号 ........................................................................................................... 23
1.4.3 chromedriver .......................................................................................................... 24
1.4.4 各版本匹配表 ....................................................................................................... 25
1.5 Pycharm 使用.................................................................................................................... 26
1.5.1 pycharm 安装 ......................................................................................................... 26
1.5.2 新建工程 ............................................................................................................... 30
1.5.3 新建脚本 ............................................................................................................... 31
1.5.4 开始编程 ............................................................................................................... 32
第 2 章 webdriver .......................................................................................................................... 32
2.1 操作浏览器基本方法 ...................................................................................................... 32
2.1.1 打开网页 ............................................................................................................... 33
2.1.2 设置休眠 ............................................................................................................... 33
2.1.3 页面刷新 ............................................................................................................... 34
2.1.4 页面切换 ............................................................................................................... 34
2.1.5 设置窗口大小 ....................................................................................................... 34
2.1.6 截屏....................................................................................................................... 35
2.1.7 退出....................................................................................................................... 35
2.2 常用 8 种元素定位(Firebug 和 firepath) ................................................................... 36
2.2.1 环境准备 ............................................................................................................... 36
2.2.2 查看页面元素: ................................................................................................... 37
2.2.3 find_element_by_id() ............................................................................................. 38
Selenium 高级自动化(上海-悠悠、葵花)
2.2.4 find_element_by_name() ...................................................................................... 38
2.2.5 find_element_by_class_name() ............................................................................. 38
2.2.6 find_element_by_tag_name() ............................................................................... 39
2.2.7 find_element_by_link_text() .................................................................................. 39
2.2.8 find_element_by_partial_link_text() ..................................................................... 40
2.2.9 find_element_by_xpath()....................................................................................... 41
2.2.10 find_element_by_css_selector() .......................................................................... 42
2.3 xpath 定位 ........................................................................................................................ 43
2.3.1 xpath:属性定位 ..................................................................................................... 43
2.3.2 xpath:其它属性 ..................................................................................................... 44
2.3.3 xpath:标签 ............................................................................................................. 44
2.3.4 xpath:层级 ............................................................................................................. 45
2.3.5 xpath:索引 ............................................................................................................. 46
2.3.6 xpath:逻辑运算 ..................................................................................................... 47
2.3.7 xpath:模糊匹配 ..................................................................................................... 47
2.4 CSS 定位语法 .................................................................................................................... 48
2.4.1 css:属性定位 .......................................................................................................... 48
2.4.2 css:其它属性 .......................................................................................................... 49
2.4.3 css:标签 .................................................................................................................. 50
2.4.4 css:层级关系 .......................................................................................................... 50
2.4.5 css:索引 .................................................................................................................. 50
2.4.6 css:逻辑运算 .......................................................................................................... 51
2.4.7 css:模糊匹配 .......................................................................................................... 51
2.5 SeleniumBuilder 辅助定位元素 ....................................................................................... 52
2.5.1 安装 Selenium Builder .......................................................................................... 52
2.5.2 直接运用 ............................................................................................................... 53
2.5.3 实践案例 ............................................................................................................... 55
2.6 操作元素(键盘和鼠标事件) ...................................................................................... 57
2.6.1 简单操作 ............................................................................................................... 57
2.6.2 submit 提交表单.................................................................................................... 58
2.6.3 键盘操作 ............................................................................................................... 59
2.6.4 鼠标悬停事件 ....................................................................................................... 61
2.7 多窗口、句柄(handle)............................................................................................... 62
2.7.1 定位一组元素 ....................................................................................................... 62
2.7.2 获取当前窗口句柄 ............................................................................................... 63
2.7.3 获取所有句柄 ....................................................................................................... 64
2.7.4 切换句柄 ............................................................................................................... 64
2.7.5 关闭新窗口,切回主页 ....................................................................................... 65
2.7.6 批量操作 ............................................................................................................... 66
2.7.7 最终脚本 ............................................................................................................... 66
2.8 定位一组元素 find_elements ......................................................................................... 68
2.8.1 定位搜索结果 ....................................................................................................... 68
2.8.2 确认定位结果 ....................................................................................................... 70
2.8.3 随机函数 ............................................................................................................... 70
Selenium 高级自动化(上海-悠悠、葵花)
2.8.4 随机打开 url ......................................................................................................... 71
2.8.5 通过 click 点击打开 .............................................................................................. 72
2.9 iframe ................................................................................................................................ 73
2.9.1 frame 和 iframe 区别 ............................................................................................. 73
2.9.2 163 登录界面 ......................................................................................................... 73
2.9.3 切换 iframe ........................................................................................................... 74
2.9.4 如果 iframe 没有 id 怎么办? ............................................................................. 75
2.9.5 释放 iframe ........................................................................................................... 75
2.9.6 如何判断元素是否在 iframe 上? ...................................................................... 76
2.10 select 下拉框 .................................................................................................................. 78
2.10.1 认识 select .......................................................................................................... 78
2.10.2 二次定位 ............................................................................................................. 79
2.10.3 直接定位 ............................................................................................................. 79
2.10.4 Select 模块(index) ................................................................................................ 80
2.10.5 Select 模块(value) ................................................................................................ 81
2.10.6 Select 模块(text) .................................................................................................. 82
2.10.7 Select 模块其它方法 ........................................................................................... 82
2.10.8 参考代码: ......................................................................................................... 83
2.11 alert\confirm\prompt ..................................................................................................... 84
2.11.1 认识 alert\confirm\prompt ................................................................................ 84
2.11.2 alert 操作 ............................................................................................................. 85
2.11.3 confirm 操作 ........................................................................................................ 86
2.11.4 prompt 操作......................................................................................................... 87
2.11.5 select 遇到的坑 ................................................................................................... 88
2.11.6 最终代码 ............................................................................................................. 89
2.12 单选框和复选框(radiobox、checkbox) .................................................................. 90
2.12.1 认识单选框和复选框 ......................................................................................... 90
2.12.2 radio 和 checkbox 源码 ....................................................................................... 91
2.12.3 单选:radio ........................................................................................................ 92
2.12.4 复选框:checkbox ............................................................................................. 93
2.12.5 全部勾选: ......................................................................................................... 93
2.12.6 判断是否选中:is_selected() ............................................................................ 94
2.12.7 参考代码: ......................................................................................................... 95
2.13 table 定位 ....................................................................................................................... 96
2.13.1 认识 table ........................................................................................................... 96
2.13.2 table 特征 ............................................................................................................ 97
2.13.3 xpath 定位 table .................................................................................................. 97
2.13.4 打印表格内容 ..................................................................................................... 98
2.13.5 参考代码: ......................................................................................................... 99
2.14 加载 Firefox 配置 ........................................................................................................... 99
2.14.1 遇到问题 ............................................................................................................. 99
2.14.2 FirefoxProfile ...................................................................................................... 100
2.14.3 profile_directory ................................................................................................ 101
2.14.4 启动配置文件 ................................................................................................... 102
Selenium 高级自动化(上海-悠悠、葵花)
2.14.5 参考代码: ....................................................................................................... 102
2.15 富文本.......................................................................................................................... 103
2.15.1 加载配置 ........................................................................................................... 103
2.15.2 打开编辑界面 ................................................................................................... 103
2.15.3 iframe 切换 ........................................................................................................ 104
2.15.4 输入正文 ........................................................................................................... 105
2.15.5 参考代码: ....................................................................................................... 105
2.16 文件上传(send_keys) ............................................................................................. 106
2.16.1 识别上传按钮 ................................................................................................... 106
2.16.2 定位 iframe ....................................................................................................... 107
2.16.3 文件上传 ........................................................................................................... 107
2.17 获取元素属性 .............................................................................................................. 108
2.17.1 获取页面 title ................................................................................................... 108
2.17.2 获取元素的文本 ............................................................................................... 109
2.17.3 获取元素的标签 ............................................................................................... 110
2.17.4 获取元素的其它属性 ....................................................................................... 111
2.17.5 获取输入框内的文本值 ................................................................................... 111
2.17.6 获取浏览器名称 ............................................................................................... 112
2.17.7 参考代码 ........................................................................................................... 112
2.18 爬页面源码(page_source) ..................................................................................... 112
2.18.1 page_source ....................................................................................................... 113
2.18.2 re 非贪婪模式 ................................................................................................... 113
2.18.3 删选 url 地址出来 ............................................................................................ 114
2.18.4 参考代码 ........................................................................................................... 114
2.19 cookie 相关操作 ........................................................................................................... 115
2.19.1 获取 cookies:get_cookies() ............................................................................ 115
2.19.2 登录后的 cookies ............................................................................................. 115
2.19.3 获取指定 name 的 cookie:driver.get_cookie(name) .................................. 116
2.19.4 清除指定 cookie:delete_cookie() .................................................................. 116
2.19.5 清除所有 cookies:delete_all_cookies() ......................................................... 117
2.19.6 cookie 操作的几个方法 .................................................................................... 117
2.19.7 参考代码 ........................................................................................................... 118
2.20 绕过验证码(add_cookie) ....................................................................................... 119
2.20.1 fiddler 抓包 ........................................................................................................ 119
2.20.2 添加 cookie 方法:driver.add_cookie() ..................................................... 120
2.20.3 cookie 组成结构 ................................................................................................ 121
2.20.4 添加 cookie ....................................................................................................... 122
2.20.5 参考代码: ....................................................................................................... 123
2.21 JS 处理滚动条 ............................................................................................................... 124
2.21.1 JavaScript 简介 ................................................................................................... 124
2.21.2 控制滚动条高度 ............................................................................................... 125
2.21.3 横向滚动条 ........................................................................................................ 125
2.21.4 Chrome 浏览器 .................................................................................................. 126
2.21.5 元素聚焦 ........................................................................................................... 126