# Video Summarization with LSTM
This repository provides the data and implementation for video summarization with LSTM, i.e. vsLSTM and dppLSTM in our paper:
**[Video Summarization with Long Short-term Memory](http://www-scf.usc.edu/~zhan355/ke_eccv2016.pdf)**
<br>
[Ke Zhang](http://www-scf.usc.edu/~zhan355/index.html)\*, Wei-Lun Chao\*, Fei Sha, and Kristen Grauman.
<br>
In Proceedings of the European Conference on Computer Vision (ECCV), 2016, Amsterdam, The Netherlands. (*Equal contribution) \[[pdf](http://www-scf.usc.edu/~zhan355/ke_eccv2016.pdf)\] \[[supp](http://www-scf.usc.edu/~zhan355/ke_eccv2016_supp.pdf)\]
If you find the codes or other related resources from this repository useful, please cite the following paper:
```
@inproceedings{zhang2016video,
title={Video summarization with long short-term memory},
author={Zhang, Ke and Chao, Wei-Lun and Sha, Fei and Grauman, Kristen},
booktitle={ECCV},
year={2016},
organization={Springer}
}
```
## Environment
- MAC OS X or Linux
- NVIDIA GPU with compute capability 3.5+
- Python 2.7+
- Theano 0.7+
- Matlab
## Data
Download the [data](https://www.dropbox.com/s/ynl4jsa2mxohs16/data.zip?dl=0) and unzip to *./data/*
Note that we down-sampled the original video by 2fps.
1) file name: in the format 'Data_$Dataset$_google_p5.h5', e.g. Data_SumMe_google_p5.h5, means the frame level feature of SumMe dataset.
2) the index of videos are stored as ‘idx’ in the file, in most cases it’s from 1 to n, where n is the number of videos in the dataset (except for Youtube dataset).
3) feature & ground-truth: the feature is indexed as ‘fea_i’ , the importance is indexed as ‘gt_1_i’ (real number, from the original dataset), and the keyframe we used is indexed as ‘gt_2_i’ (binary value transferred from the original dataset) for the i-th video in the dataset.
Original videos and annotations for each dataset are also available from the the authors' project page
* TVSum dataset: https://github.com/yalesong/tvsum
* SumMe dataset: https://people.ee.ethz.ch/~gyglim/vsum/#benchmark
* OVP and YouTube datasets: https://sites.google.com/site/vsummsite/
## Codes
### dppLSTM for video summarization
We have enclosed pre-trained models in the *./model* directory
download the model and run the following commands:
Download the pre-trained models and unzip it to *./models* and run the following commands:
```
cd ./codes
THEANO_FLAGS=device=gpu0,floatX=float32 python dppLSTM_main.py
```
This will automatically run summarization on the video data using pre-trained model, and save the results in *./res_LSTM/* as **dppLSTM_$DATASET$_2_inference.h5**
If you want to train the model on your own data, just uncomment Line 85 in *dppLSTM_main.py*
```
train(model_idx = model_idx, train_set = train_set, val_set = val_set, model_saved = model_file)
```
### Evaluation
For both SumMe and TVSum datasets, you can find the code for evaluation provided by the author:
* TVSum: https://github.com/yalesong/tvsum
* SumMe: https://people.ee.ethz.ch/~gyglim/vsum/#benchmark
We also provided the evaluation code with wrappers that help adapt to the datasets above
To run evaluation on the predicted summarization, start the matlab and run the following commands:
```
cd ./codes
dppLSTM_eval('../data/', '$DATASET$', '/dppLSTM_$DATASET$_2_inference.h5')
```
没有合适的资源?快使用搜索试试~ 我知道了~
温馨提示
【达摩老生出品,必属精品,亲测校正,质量保证】 资源名:Video-Summarization-with-LSTM-matlab 资源类型:matlab项目全套源码 源码说明: 全部项目源码都是经过测试校正后百分百成功运行的,如果您下载后不能运行可联系我进行指导或者更换。 适合人群:新手及有一定经验的开发人员
资源推荐
资源详情
资源评论
收起资源包目录
Video-Summarization-with-LSTM-matlab.zip (70个子文件)
Video-Summarization-with-LSTM-matlab
models
model_trained_TVSum 10.27MB
model_trained_SumMe 10.27MB
codes
dppLSTM_eval.m 5KB
get_shot_knapsack.m 1KB
tools
data_loader.py 7KB
__init__.py 21B
dppLSTM_main.py 3KB
layers
mlp.py 3KB
__init__.pyc 179B
summ_dppLSTM.py 8KB
__init__.py 21B
evalSumMe
summe_evaluateSummary.m 2KB
summe_evaluateSummary_m.m 2KB
GT
Jumps.mat 2KB
Valparaiso_Downhill.mat 4KB
Air_Force_One.mat 3KB
playing_ball.mat 3KB
Eiffel Tower.mat 3KB
Statue of Liberty.mat 3KB
Base jumping.mat 4KB
Car_railcrossing.mat 3KB
Bearpark_climbing.mat 3KB
Kids_playing_in_leaves.mat 3KB
Scuba.mat 3KB
car_over_camera.mat 3KB
Uncut_Evening_Flight.mat 4KB
untitled.pdf 4KB
Bike Polo.mat 3KB
Cooking.mat 2KB
St Maarten Landing.mat 2KB
Excavators river crossing.mat 5KB
Notre_Dame.mat 4KB
Paintball.mat 4KB
Fire Domino.mat 2KB
Cockpit_Landing.mat 5KB
paluma_jump.mat 2KB
Bus_in_Rock_Tunnel.mat 4KB
Playing_on_water_slide.mat 3KB
Saving dolphines.mat 4KB
summe_evaluateIOU.m 3KB
SumMe_data.mat 114KB
knapsack
license.txt 1KB
knapsack_demo.m 454B
html
knapsack_demo.html 3KB
knapsack.m 2KB
solve_knapsack.m 1022B
evalTVSum
evaluate_TVSum.m 3KB
script_evaluate_result.m 4KB
matlab
eval
script_evaluate_result.m 4KB
knapsack
license.txt 1KB
knapsack_demo.m 454B
html
knapsack_demo.html 3KB
knapsack.m 2KB
solve_knapsack.m 1KB
consistency
pairwise_f1.m 976B
script_report_cronbach.m 1KB
cronbach.m 1011B
ydata-tvsum50.mat 1.05MB
tvsum50.mat 1.05MB
knapsack
license.txt 1KB
knapsack_demo.m 454B
html
knapsack_demo.html 3KB
knapsack.m 2KB
solve_knapsack.m 1KB
optimizer
__init__.py 21B
adam_opt.py 5KB
Matlab实现无约束条件下普列姆(Prim)算法.docx 14KB
data
README.md 37B
LICENSE.md 163B
README.md 3KB
共 70 条
- 1
资源评论
阿里matlab建模师
- 粉丝: 3302
- 资源: 2784
上传资源 快速赚钱
- 我的内容管理 展开
- 我的资源 快来上传第一个资源
- 我的收益 登录查看自己的收益
- 我的积分 登录查看自己的积分
- 我的C币 登录后查看C币余额
- 我的收藏
- 我的下载
- 下载帮助
安全验证
文档复制为VIP权益,开通VIP直接复制
信息提交成功