## Learning CNN-LSTM Architectures for Image Caption Generation
This code contains a Tensorflow implementation of the CNN-LSTM architecture used to attain state-of-the-art performance on the MSCOCO dataset. We achieve a BLEU-4 score of 24.4 and CIDEr score of 81.7 compared to 27.7 and 85.5 by Google's implementation. Qualitative analysis of the generated captions indicate that the model is able to sensibly caption a wide variety of images from the MSCOCO dataset.
### Demo instructions
To try a demo of our best trained model, first ensure that Caffe is installed on your computer and that you have downloaded the GoogleNet model using these [instructions](http://www.marekrei.com/blog/transforming-images-to-feature-vectors/). You'll also need Tensorflow 0.8 installed. Then, run:
./download.sh
which will retrive all pickled data files (graciously shared by [Satoshi](http://t-satoshi.blogspot.com/2015/12/image-caption-generation-by-cnn-and-lstm.html) in his chainer implementation.) and the Tensorflow saved model created in this project needed to run the demo. This requires around 180MB of disk space. The 'caption_image.py' file contains all the code needed to load and use the saved model. To run the demo, do:
python caption_image.py -i <path_to_image>
We have included a demo pizza image at images/pizza.jpg to sanity check your installation. Running `python caption_image.py -i images/pizza.jpg` produces the caption "a pizza with cheese and cheese on a table". It's not perfect, but still pretty cool!
### Other files
`model.py` contains the `Model` class that contains the CNN-LSTM architecture (using Tensorflow's dynamic_rnn API) and various helper functions for generating captions. `evaluate_captions.py` is a helper script to generate aggregated JSON files that can then be used for hyperparameter tuning. `image_feature_cnn.py` contains the helper functions we use to load up the GoogleNet batch normalization CNN model and turn images into 1024 x 1 vectors.
没有合适的资源?快使用搜索试试~ 我知道了~
CNN-LSTM-Caption-Generator-master
共11个文件
py:5个
gitignore:3个
sh:1个
需积分: 39 33 下载量 102 浏览量
2018-01-11
11:24:55
上传
评论 1
收藏 323KB ZIP 举报
温馨提示
CNN-LSTM-Caption-Generator-master.zip CNN-LSTM-Caption-Generator-master.zip CNN-LSTM-Caption-Generator-master.zip
资源推荐
资源详情
资源评论
收起资源包目录
CNN-LSTM-Caption-Generator-master.zip (11个子文件)
CNN-LSTM-Caption-Generator-master
.gitignore 5B
download.sh 863B
best_model
.gitignore 13B
README.md 2KB
evaluate_captions.py 1KB
image_feature_cnn.py 4KB
utils.py 2KB
model.py 10KB
data_files
.gitignore 13B
images
pizza.jpg 313KB
caption_image.py 1KB
共 11 条
- 1
资源评论
barryji
- 粉丝: 1
- 资源: 6
上传资源 快速赚钱
- 我的内容管理 展开
- 我的资源 快来上传第一个资源
- 我的收益 登录查看自己的收益
- 我的积分 登录查看自己的积分
- 我的C币 登录后查看C币余额
- 我的收藏
- 我的下载
- 下载帮助
最新资源
- 第十四章 scikit-learn 库(jupyter版 人工智能编程基础)
- 天天视频(一款功能强大的影视剧应用 电.apk
- 一个xxs做的python小游戏,当对方血量比你低时可以一直攻击(什么时候改改?)
- 第十三章 Matplotlib库(jupyter版 人工智能编程基础)
- 基于yolov8的人体动作识别检测项目源码.zip
- 自然语言处理大作业-python的汉语词义自动消歧系统源码+报告.zip
- MT管理器_2.15.3-会员功能解锁.apk
- 基于机器学习的共享单车使用量分析与预测
- 基于Python开发的汉语词义自动消歧系统源码+超详细注释+实验报告(95分以上).zip
- 51单片机汇编语言音乐盒+电子琴,proteus仿真运行正常,开发板实物运行正常 代码都带有注释,方便理解和学习
资源上传下载、课程学习等过程中有任何疑问或建议,欢迎提出宝贵意见哦~我们会及时处理!
点击此处反馈
安全验证
文档复制为VIP权益,开通VIP直接复制
信息提交成功