## 主动学习推荐系统
### 0. 概述
- 数据集:[Amazon商品数据集](http://jmcauley.ucsd.edu/data/amazon/)
- 编程环境:Python, Matlab, Markdown
### 1. 数据预处理
- 商品信息
- 提取数据集中的title和description信息
- 命令:`python item_information.py [file1, ..., file3]`
- 用户物品评分信息
- 提取用户-物品评分,划分train集和test集
- 将train集中的用户作为用户全集,以防止出现train集中有用户没有评分的情况
- 命令:`python user_information.py [file1, ..., file7]`
- 商品相似度生成
- title: 分词 + LDA主题模型(topic number = 15)
- description: 分词 + LDA主题模型(topic number = 15)
- 未使用price(缺失值太多)
- 未使用category(同类商品)
- 命令:`python item_similarity.py [topic number, file1, ..., file6]`
- 商品description和title相似度权重生成
- non linear regression
- Similarity(i1, i2) = weight1 * S_title(i1) + weight2 * S_description(i2)
- 命令:
- `python similarity_parameters.py [file1, ..., file7]`
- `fitnlm(path, param1, param2)`
- 用户相似度生成
- 评分相似度
- 命令:`python user_similarity.py [file1, ..., file3]`
- 用户聚类
- 用户聚类依靠用户相似度作为距离度量,使用K-medoids作为聚类算法
- 问题主要存在于:由于评分稀疏,很多用户之间距离为0
- 命令:`python user_clustering.py input_file number_of_clusters output_file`
- 建树前的准备工作
- 生成用户聚类对任一物品的平均评分,便于计算时直接调用
- 利用非线性回归拟合的参数生成相似度矩阵
- 命令:`python buildtree_preparation.py input_file init_ptitle init_pdescrip output_file`
### 2. 建树及预测
- 树的生成:
- 三叉树,对应不喜欢、一般般喜欢和喜欢三个节点
- 生成的节点信息用*self.tree*和*self.node_interval*两个变量保存
- 构建预测模型:
- 利用Spark的mllib包实现ALS Matrix Factorization
- 生成伪物品(每个节点)和用户对应的latent vector(对每一层都计算)
- 预测评分:
- 对每一个test商品,从树的根节点开始向下走,利用目标叶子节点的latent vector作为它的特征向量
- 利用特征向量和所有物品的特征向量的点积预测评分,计算RMSE(对每一层都计算)
- 命令:`python build_tree.py [input_file1, ..., input_file5] desired_depth`
### 3. 运行
- 利用*Python*脚本运行上述所有步骤:`python script.py`
- 代码开头数据集名称(*dataset*)需相应更改
### 4. 对比实验
- FDT (Factorized Deicision Tree)
- `python factorized_decision_tree.py dataset depth` (dataset是数据集的名字,depth决定了树的高度)
- **输入:** *I\*U* 的矩阵 => *new-user problem*
- **输入:** *U\*I* 的矩阵 => *new-item problem*
- CAL (Content-based Active Learning)
- `python content_based_active_learning.py dataset K` (dataset是数据集的名字,K决定了选择TopK的用户进行query)
- CBCF (Content-based Collaborative Filtering)
### 4. 当前问题
1. 对Amazon数据集来说树的第一层预测效果最好,分析原因可能如下:
- 数据集过于稀疏,导致每一用户基本只有一个评分,第一层作为伪物品作矩阵分解时评分满,效果好,越往下效果越差。
- 点的划分过于不均匀,使得伪物品选择不优秀。
- 解决方法:
- 使用平均划分法划分每一个节点
- 使用相似度拟合评分作为MF的输入,而非平均评分
2. 物品个数超过30万的Automotive集合上计算*item similarity*时出现*Memory Error*
- 解决方法:选择评分个数大于5个的物品和用户
没有合适的资源?快使用搜索试试~ 我知道了~
温馨提示
编程环境:Python, Matlab, Markdown 1. 数据预处理 商品信息 提取数据集中的title和description信息 命令:python item_information.py [file1, ..., file3] 用户物品评分信息 提取用户-物品评分,划分train集和test集 将train集中的用户作为用户全集,以防止出现train集中有用户没有评分的情况 命令:python user_information.py [file1, ..., file7] 商品相似度生成 title: 分词 + LDA主题模型(topic number = 15) description: 分词 + LDA主题模型(topic number = 15) 未使用price(缺失值太多) 未使用category(同类商品) 命令:python item_similarity.py [topic number, file1, ..., file6] 商品description和title相似度权重生成 non linear regression Similarity(i
资源推荐
资源详情
资源评论
![zip](https://img-home.csdnimg.cn/images/20210720083736.png)
![zip](https://img-home.csdnimg.cn/images/20210720083736.png)
![zip](https://img-home.csdnimg.cn/images/20210720083736.png)
![zip](https://img-home.csdnimg.cn/images/20210720083736.png)
![zip](https://img-home.csdnimg.cn/images/20210720083736.png)
![zip](https://img-home.csdnimg.cn/images/20210720083736.png)
![zip](https://img-home.csdnimg.cn/images/20210720083736.png)
![zip](https://img-home.csdnimg.cn/images/20210720083736.png)
![zip](https://img-home.csdnimg.cn/images/20210720083736.png)
![zip](https://img-home.csdnimg.cn/images/20210720083736.png)
![zip](https://img-home.csdnimg.cn/images/20210720083736.png)
![zip](https://img-home.csdnimg.cn/images/20210720083736.png)
![zip](https://img-home.csdnimg.cn/images/20210720083736.png)
![zip](https://img-home.csdnimg.cn/images/20210720083736.png)
![zip](https://img-home.csdnimg.cn/images/20210720083736.png)
![zip](https://img-home.csdnimg.cn/images/20210720083736.png)
![zip](https://img-home.csdnimg.cn/images/20210720083736.png)
![zip](https://img-home.csdnimg.cn/images/20210720083736.png)
![zip](https://img-home.csdnimg.cn/images/20210720083736.png)
![zip](https://img-home.csdnimg.cn/images/20210720083736.png)
![rar](https://img-home.csdnimg.cn/images/20210720083606.png)
![rar](https://img-home.csdnimg.cn/images/20210720083606.png)
![rar](https://img-home.csdnimg.cn/images/20210720083606.png)
![rar](https://img-home.csdnimg.cn/images/20210720083606.png)
![zip](https://img-home.csdnimg.cn/images/20210720083736.png)
![zip](https://img-home.csdnimg.cn/images/20210720083736.png)
![rar](https://img-home.csdnimg.cn/images/20210720083606.png)
收起资源包目录
![package](https://csdnimg.cn/release/downloadcmsfe/public/img/package.f3fc750b.png)
![folder](https://csdnimg.cn/release/downloadcmsfe/public/img/folder.005fa2e5.png)
![folder](https://csdnimg.cn/release/downloadcmsfe/public/img/folder.005fa2e5.png)
![file-type](https://csdnimg.cn/release/download/static_files/pc/images/minetype/UNKNOWN.png)
![file-type](https://csdnimg.cn/release/download/static_files/pc/images/minetype/UNKNOWN.png)
![folder](https://csdnimg.cn/release/downloadcmsfe/public/img/folder.005fa2e5.png)
![folder](https://csdnimg.cn/release/downloadcmsfe/public/img/folder.005fa2e5.png)
![file-type](https://csdnimg.cn/release/download/static_files/pc/images/minetype/UNKNOWN.png)
![folder](https://csdnimg.cn/release/downloadcmsfe/public/img/folder.005fa2e5.png)
![folder](https://csdnimg.cn/release/downloadcmsfe/public/img/folder.005fa2e5.png)
![folder](https://csdnimg.cn/release/downloadcmsfe/public/img/folder.005fa2e5.png)
![file-type](https://csdnimg.cn/release/download/static_files/pc/images/minetype/UNKNOWN.png)
![folder](https://csdnimg.cn/release/downloadcmsfe/public/img/folder.005fa2e5.png)
![folder](https://csdnimg.cn/release/downloadcmsfe/public/img/folder.005fa2e5.png)
![file-type](https://csdnimg.cn/release/download/static_files/pc/images/minetype/UNKNOWN.png)
![file-type](https://csdnimg.cn/release/download/static_files/pc/images/minetype/UNKNOWN.png)
![folder](https://csdnimg.cn/release/downloadcmsfe/public/img/folder.005fa2e5.png)
![file-type](https://csdnimg.cn/release/download/static_files/pc/images/minetype/UNKNOWN.png)
![file-type](https://csdnimg.cn/release/download/static_files/pc/images/minetype/UNKNOWN.png)
![folder](https://csdnimg.cn/release/downloadcmsfe/public/img/folder.005fa2e5.png)
![file-type](https://csdnimg.cn/release/download/static_files/pc/images/minetype/UNKNOWN.png)
![folder](https://csdnimg.cn/release/downloadcmsfe/public/img/folder.005fa2e5.png)
![file-type](https://csdnimg.cn/release/download/static_files/pc/images/minetype/UNKNOWN.png)
![folder](https://csdnimg.cn/release/downloadcmsfe/public/img/folder.005fa2e5.png)
![folder](https://csdnimg.cn/release/downloadcmsfe/public/img/folder.005fa2e5.png)
![file-type](https://csdnimg.cn/release/download/static_files/pc/images/minetype/UNKNOWN.png)
![folder](https://csdnimg.cn/release/downloadcmsfe/public/img/folder.005fa2e5.png)
![folder](https://csdnimg.cn/release/downloadcmsfe/public/img/folder.005fa2e5.png)
![file-type](https://csdnimg.cn/release/download/static_files/pc/images/minetype/UNKNOWN.png)
![folder](https://csdnimg.cn/release/downloadcmsfe/public/img/folder.005fa2e5.png)
![file-type](https://csdnimg.cn/release/download/static_files/pc/images/minetype/UNKNOWN.png)
![file-type](https://csdnimg.cn/release/download/static_files/pc/images/minetype/UNKNOWN.png)
![file-type](https://csdnimg.cn/release/download/static_files/pc/images/minetype/UNKNOWN.png)
![file-type](https://csdnimg.cn/release/download/static_files/pc/images/minetype/UNKNOWN.png)
![file-type](https://csdnimg.cn/release/download/static_files/pc/images/minetype/UNKNOWN.png)
![file-type](https://csdnimg.cn/release/download/static_files/pc/images/minetype/UNKNOWN.png)
![file-type](https://csdnimg.cn/release/download/static_files/pc/images/minetype/UNKNOWN.png)
![file-type](https://csdnimg.cn/release/download/static_files/pc/images/minetype/UNKNOWN.png)
![file-type](https://csdnimg.cn/release/download/static_files/pc/images/minetype/UNKNOWN.png)
![file-type](https://csdnimg.cn/release/download/static_files/pc/images/minetype/UNKNOWN.png)
![file-type](https://csdnimg.cn/release/download/static_files/pc/images/minetype/UNKNOWN.png)
![file-type](https://csdnimg.cn/release/download/static_files/pc/images/minetype/UNKNOWN.png)
![file-type](https://csdnimg.cn/release/download/static_files/pc/images/minetype/UNKNOWN.png)
![file-type](https://csdnimg.cn/release/download/static_files/pc/images/minetype/UNKNOWN.png)
![folder](https://csdnimg.cn/release/downloadcmsfe/public/img/folder.005fa2e5.png)
![folder](https://csdnimg.cn/release/downloadcmsfe/public/img/folder.005fa2e5.png)
![file-type](https://csdnimg.cn/release/download/static_files/pc/images/minetype/UNKNOWN.png)
![file-type](https://csdnimg.cn/release/download/static_files/pc/images/minetype/UNKNOWN.png)
![file-type](https://csdnimg.cn/release/download/static_files/pc/images/minetype/UNKNOWN.png)
![folder](https://csdnimg.cn/release/downloadcmsfe/public/img/folder.005fa2e5.png)
![file-type](https://csdnimg.cn/release/download/static_files/pc/images/minetype/UNKNOWN.png)
![file-type](https://csdnimg.cn/release/download/static_files/pc/images/minetype/UNKNOWN.png)
![file-type](https://csdnimg.cn/release/download/static_files/pc/images/minetype/UNKNOWN.png)
![file-type](https://csdnimg.cn/release/download/static_files/pc/images/minetype/UNKNOWN.png)
![folder](https://csdnimg.cn/release/downloadcmsfe/public/img/folder.005fa2e5.png)
![file-type](https://csdnimg.cn/release/download/static_files/pc/images/minetype/UNKNOWN.png)
![file-type](https://csdnimg.cn/release/download/static_files/pc/images/minetype/UNKNOWN.png)
![file-type](https://csdnimg.cn/release/download/static_files/pc/images/minetype/UNKNOWN.png)
![file-type](https://csdnimg.cn/release/download/static_files/pc/images/minetype/UNKNOWN.png)
![folder](https://csdnimg.cn/release/downloadcmsfe/public/img/folder.005fa2e5.png)
![file-type](https://csdnimg.cn/release/download/static_files/pc/images/minetype/UNKNOWN.png)
![file-type](https://csdnimg.cn/release/download/static_files/pc/images/minetype/UNKNOWN.png)
![file-type](https://csdnimg.cn/release/download/static_files/pc/images/minetype/UNKNOWN.png)
![file-type](https://csdnimg.cn/release/download/static_files/pc/images/minetype/UNKNOWN.png)
![file-type](https://csdnimg.cn/release/download/static_files/pc/images/minetype/UNKNOWN.png)
![file-type](https://csdnimg.cn/release/download/static_files/pc/images/minetype/UNKNOWN.png)
![file-type](https://csdnimg.cn/release/download/static_files/pc/images/minetype/UNKNOWN.png)
![file-type](https://csdnimg.cn/release/download/static_files/pc/images/minetype/TXT.png)
![file-type](https://csdnimg.cn/release/download/static_files/pc/images/minetype/UNKNOWN.png)
![file-type](https://csdnimg.cn/release/download/static_files/pc/images/minetype/UNKNOWN.png)
![file-type](https://csdnimg.cn/release/download/static_files/pc/images/minetype/UNKNOWN.png)
![file-type](https://csdnimg.cn/release/download/static_files/pc/images/minetype/UNKNOWN.png)
![file-type](https://csdnimg.cn/release/download/static_files/pc/images/minetype/UNKNOWN.png)
![file-type](https://csdnimg.cn/release/download/static_files/pc/images/minetype/UNKNOWN.png)
![file-type](https://csdnimg.cn/release/download/static_files/pc/images/minetype/UNKNOWN.png)
共 52 条
- 1
资源评论
![avatar-default](https://csdnimg.cn/release/downloadcmsfe/public/img/lazyLogo2.1882d7f4.png)
![avatar](https://profile-avatar.csdnimg.cn/8b9566d6b7ee402eb11b3ab031c6660f_fl1768317420.jpg!1)
FL1768317420
- 粉丝: 5054
- 资源: 5729
上传资源 快速赚钱
我的内容管理 展开
我的资源 快来上传第一个资源
我的收益
登录查看自己的收益我的积分 登录查看自己的积分
我的C币 登录后查看C币余额
我的收藏
我的下载
下载帮助
![voice](https://csdnimg.cn/release/downloadcmsfe/public/img/voice.245cc511.png)
![center-task](https://csdnimg.cn/release/downloadcmsfe/public/img/center-task.c2eda91a.png)
最新资源
- mysql-odbc-64-8.0.26
- 新建 Microsoft Word 文档 (2).docx
- 1652189183080063_42AD52629C553C7BC65694A425954A3F_1.apk
- 王都创世物语魔改版.apk
- Linear_regrassion_with_gradien_decent_example.ipynb
- 1719163625521533_1e90151d86c4cc2fadb97a0a82d0fc0e
- 虚拟现实课件-第五章 3物理建模
- 虚拟现实课件-第五章 2运动建模
- A1-ERP(资源协同)管理平台需求说明书.pdf
- 基于springboot+sureness的面向REST API资源无状态认证权限管理系统(源码+数据库+说明文档)
资源上传下载、课程学习等过程中有任何疑问或建议,欢迎提出宝贵意见哦~我们会及时处理!
点击此处反馈
![feedback](https://img-home.csdnimg.cn/images/20220527035711.png)
![feedback](https://img-home.csdnimg.cn/images/20220527035711.png)
![feedback-tip](https://img-home.csdnimg.cn/images/20220527035111.png)
安全验证
文档复制为VIP权益,开通VIP直接复制
![dialog-icon](https://csdnimg.cn/release/downloadcmsfe/public/img/green-success.6a4acb44.png)