# Informer: Beyond Efficient Transformer for Long Sequence Time-Series Forecasting (AAAI'21 Best Paper)
![Python 3.6](https://img.shields.io/badge/python-3.6-green.svg?style=plastic)
![PyTorch 1.2](https://img.shields.io/badge/PyTorch%20-%23EE4C2C.svg?style=plastic)
![cuDNN 7.3.1](https://img.shields.io/badge/cudnn-7.3.1-green.svg?style=plastic)
![License CC BY-NC-SA](https://img.shields.io/badge/license-CC_BY--NC--SA--green.svg?style=plastic)
This is the origin Pytorch implementation of Informer in the following paper:
[Informer: Beyond Efficient Transformer for Long Sequence Time-Series Forecasting](https://arxiv.org/abs/2012.07436). Special thanks to `Jieqi Peng`@[cookieminions](https://github.com/cookieminions) for building this repo.
:triangular_flag_on_post:**News**(Mar 27, 2023): We will release Informer V2 soon.
:triangular_flag_on_post:**News**(Feb 28, 2023): The Informer's [extension paper](https://www.sciencedirect.com/science/article/pii/S0004370223000322) is online on AIJ.
:triangular_flag_on_post:**News**(Mar 25, 2021): We update all experiment [results](#resultslink) with hyperparameter settings.
:triangular_flag_on_post:**News**(Feb 22, 2021): We provide [Colab Examples](#colablink) for friendly usage.
:triangular_flag_on_post:**News**(Feb 8, 2021): Our Informer paper has been awarded [AAAI'21 Best Paper \[Official\]](https://aaai.org/Conferences/AAAI-21/aaai-outstanding-and-distinguished-papers/)[\[Beihang\]](http://scse.buaa.edu.cn/info/1097/7443.htm)[\[Rutgers\]](https://www.business.rutgers.edu/news/hui-xiong-and-research-colleagues-receive-aaai-best-paper-award)! We will continue this line of research and update on this repo. Please star this repo and [cite](#citelink) our paper if you find our work is helpful for you.
<p align="center">
<img src=".\img\informer.png" height = "360" alt="" align=center />
<br><br>
<b>Figure 1.</b> The architecture of Informer.
</p>
## ProbSparse Attention
The self-attention scores form a long-tail distribution, where the "active" queries lie in the "head" scores and "lazy" queries lie in the "tail" area. We designed the ProbSparse Attention to select the "active" queries rather than the "lazy" queries. The ProbSparse Attention with Top-u queries forms a sparse Transformer by the probability distribution.
`Why not use Top-u keys?` The self-attention layer's output is the re-represent of input. It is formulated as a weighted combination of values w.r.t. the score of dot-product pairs. The top queries with full keys encourage a complete re-represent of leading components in the input, and it is equivalent to selecting the "head" scores among all the dot-product pairs. If we choose Top-u keys, the full keys just preserve the trivial sum of values within the "long tail" scores but wreck the leading components' re-represent.
<p align="center">
<img src=".\img\probsparse_intro.png" height = "320" alt="" align=center />
<br><br>[.gitignore](..%2F..%2F.gitignore)
<b>Figure 2.</b> The illustration of ProbSparse Attention.
</p>
## Requirements
- Python 3.6+
- matplotlib == 3.1.1
- numpy == 1.19.4
- pandas == 0.25.1
- scikit_learn == 0.21.3
- torch == 1.8.0
Dependencies can be installed using the following command:
```bash
pip install -r requirements.txt
```
## Data
The ETT dataset used in the paper can be downloaded in the repo [ETDataset](https://github.com/zhouhaoyi/ETDataset).
The required data files should be put into `data/ETT/` folder. A demo slice of the ETT data is illustrated in the following figure. Note that the input of each dataset is zero-mean normalized in this implementation.
<p align="center">
<img src="./img/data.png" height = "168" alt="" align=center />
<br><br>
<b>Figure 3.</b> An example of the ETT data.
</p>
The ECL data and Weather data can be downloaded here.
- [Google Drive](https://drive.google.com/drive/folders/1ohGYWWohJlOlb2gsGTeEq3Wii2egnEPR?usp=sharing)
- [BaiduPan](https://pan.baidu.com/s/1wyaGUisUICYHnfkZzWCwyA), password: 6gan
## Reproducibility
To easily reproduce the results you can follow the next steps:
1. Initialize the docker image using: `make init`.
2. Download the datasets using: `make dataset`.
3. Run each script in `scripts/` using `make run_module module="bash ETTh1.sh"` for each script.
4. Alternatively, run all the scripts at once:
```
for file in `ls scripts`; do make run_module module="bash scripts/$script"; done
```
## Usage
<span id="colablink">Colab Examples:</span> We provide google colabs to help reproduce and customize our repo, which includes `experiments(train and test)`, `prediction`, `visualization` and `custom data`.
[![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/drive/1_X7O2BkFLvqyCdZzDZvV2MB0aAvYALLC)
Commands for training and testing the model with *ProbSparse* self-attention on Dataset ETTh1, ETTh2 and ETTm1 respectively:
```bash
# ETTh1
python -u main_informer.py --model informer --data ETTh1 --attn prob --freq h
# ETTh2
python -u main_informer.py --model informer --data ETTh2 --attn prob --freq h
# ETTm1
python -u main_informer.py --model informer --data ETTm1 --attn prob --freq t
```
More parameter information please refer to `main_informer.py`.
We provide a more detailed and complete command description for training and testing the model:
```python
python -u main_informer.py --model <model> --data <data>
--root_path <root_path> --data_path <data_path> --features <features>
--target <target> --freq <freq> --checkpoints <checkpoints>
--seq_len <seq_len> --label_len <label_len> --pred_len <pred_len>
--enc_in <enc_in> --dec_in <dec_in> --c_out <c_out> --d_model <d_model>
--n_heads <n_heads> --e_layers <e_layers> --d_layers <d_layers>
--s_layers <s_layers> --d_ff <d_ff> --factor <factor> --padding <padding>
--distil --dropout <dropout> --attn <attn> --embed <embed> --activation <activation>
--output_attention --do_predict --mix --cols <cols> --itr <itr>
--num_workers <num_workers> --train_epochs <train_epochs>
--batch_size <batch_size> --patience <patience> --des <des>
--learning_rate <learning_rate> --loss <loss> --lradj <lradj>
--use_amp --inverse --use_gpu <use_gpu> --gpu <gpu> --use_multi_gpu --devices <devices>
```
The detailed descriptions about the arguments are as following:
| Parameter name | Description of parameter |
| --- | --- |
| model | The model of experiment. This can be set to `informer`, `informerstack`, `informerlight(TBD)` |
| data | The dataset name |
| root_path | The root path of the data file (defaults to `./data/ETT/`) |
| data_path | The data file name (defaults to `ETTh1.csv`) |
| features | The forecasting task (defaults to `M`). This can be set to `M`,`S`,`MS` (M : multivariate predict multivariate, S : univariate predict univariate, MS : multivariate predict univariate) |
| target | Target feature in S or MS task (defaults to `OT`) |
| freq | Freq for time features encoding (defaults to `h`). This can be set to `s`,`t`,`h`,`d`,`b`,`w`,`m` (s:secondly, t:minutely, h:hourly, d:daily, b:business days, w:weekly, m:monthly).You can also use more detailed freq like 15min or 3h |
| checkpoints | Location of model checkpoints (defaults to `./checkpoints/`) |
| seq_len | Input sequence length of Informer encoder (defaults to 96) |
| label_len | Start token length of Informer decoder (defaults to 48) |
| pred_len | Prediction sequence length (defaults to 24) |
| enc_in | Encoder input size (defaults to 7) |
| dec_in | Decoder input size (defaults to 7) |
| c_out | Output size (defaults to 7) |
| d_model | Dimension of model (defaults to 512) |
| n_heads | Num of heads (defaults to 8) |
| e_layers | Num of encoder layers (defaults to 2) |
| d_layers | Num of decoder layers (defaults to 1) |
| s_layers | Num of stack encoder layers (defaults to `3,2,1`) |
| d_ff | Dimension of fcn (defaults to 2048) |
| factor |
没有合适的资源?快使用搜索试试~ 我知道了~
基于ARIMA+Transformer+LSTM对心跳时间序列数据进行预测(源码+项目说明).zip
共1179个文件
hea:310个
dat:310个
atr:310个
1.该资源内容由用户上传,如若侵权请联系客服进行举报
2.虚拟产品一经售出概不退款(资源遇到问题,请及时私信上传者)
2.虚拟产品一经售出概不退款(资源遇到问题,请及时私信上传者)
版权申诉
5星 · 超过95%的资源 1 下载量 4 浏览量
2024-02-28
10:34:29
上传
评论
收藏 19.81MB ZIP 举报
温馨提示
基于ARIMA+Transformer+LSTM对心跳时间序列数据进行预测(源码+项目说明).zip 1、该资源内项目代码经过严格调试,下载即用确保可以运行! 2、该资源适合计算机相关专业(如计科、人工智能、大数据、数学、电子信息等)正在做课程设计、期末大作业和毕设项目的学生、或者相关技术学习者作为学习资料参考使用。 3、该资源包括全部源码,需要具备一定基础才能看懂并调试代码。 基于ARIMA+Transformer+LSTM对心跳时间序列数据进行预测(源码+项目说明).zip 1、该资源内项目代码经过严格调试,下载即用确保可以运行! 2、该资源适合计算机相关专业(如计科、人工智能、大数据、数学、电子信息等)正在做课程设计、期末大作业和毕设项目的学生、或者相关技术学习者作为学习资料参考使用。 3、该资源包括全部源码,需要具备一定基础才能看懂并调试代码。 基于ARIMA+Transformer+LSTM对心跳时间序列数据进行预测(源码+项目说明).zip 1、该资源内项目代码经过严格调试,下载即用确保可以运行! 2、该资源适合计算机相关专业(如计科、人工智能、大数据、数学、电子信息等
资源推荐
资源详情
资源评论
收起资源包目录
基于ARIMA+Transformer+LSTM对心跳时间序列数据进行预测(源码+项目说明).zip (1179个子文件)
ANNOTATORS 73B
rec_2.atr 80B
rec_1.atr 80B
rec_2.atr 80B
rec_1.atr 80B
rec_2.atr 80B
rec_1.atr 80B
rec_4.atr 80B
rec_3.atr 80B
rec_2.atr 80B
rec_1.atr 80B
rec_5.atr 80B
rec_3.atr 80B
rec_2.atr 80B
rec_1.atr 80B
rec_4.atr 80B
rec_3.atr 80B
rec_2.atr 80B
rec_6.atr 80B
rec_1.atr 80B
rec_5.atr 80B
rec_2.atr 80B
rec_1.atr 80B
rec_4.atr 80B
rec_3.atr 80B
rec_2.atr 80B
rec_1.atr 80B
rec_5.atr 80B
rec_4.atr 80B
rec_3.atr 80B
rec_2.atr 80B
rec_1.atr 80B
rec_5.atr 80B
rec_2.atr 80B
rec_1.atr 80B
rec_2.atr 80B
rec_1.atr 80B
rec_2.atr 80B
rec_1.atr 80B
rec_2.atr 80B
rec_1.atr 80B
rec_2.atr 80B
rec_1.atr 80B
rec_4.atr 80B
rec_3.atr 80B
rec_2.atr 80B
rec_1.atr 80B
rec_5.atr 80B
rec_2.atr 80B
rec_1.atr 80B
rec_2.atr 80B
rec_1.atr 80B
rec_2.atr 80B
rec_1.atr 80B
rec_3.atr 80B
rec_2.atr 80B
rec_1.atr 80B
rec_2.atr 80B
rec_1.atr 80B
rec_3.atr 80B
rec_2.atr 80B
rec_1.atr 80B
rec_2.atr 80B
rec_1.atr 80B
rec_2.atr 80B
rec_1.atr 80B
rec_2.atr 80B
rec_1.atr 80B
rec_4.atr 80B
rec_3.atr 80B
rec_2.atr 80B
rec_1.atr 80B
rec_5.atr 80B
rec_3.atr 80B
rec_2.atr 80B
rec_1.atr 80B
rec_4.atr 80B
rec_3.atr 80B
rec_2.atr 80B
rec_1.atr 80B
rec_5.atr 80B
rec_4.atr 80B
rec_3.atr 80B
rec_2.atr 80B
rec_1.atr 80B
rec_2.atr 80B
rec_1.atr 80B
rec_2.atr 80B
rec_1.atr 80B
rec_2.atr 80B
rec_1.atr 80B
rec_1.atr 80B
rec_2.atr 80B
rec_1.atr 80B
rec_4.atr 80B
rec_3.atr 80B
rec_2.atr 80B
rec_1.atr 80B
rec_3.atr 80B
rec_2.atr 80B
共 1179 条
- 1
- 2
- 3
- 4
- 5
- 6
- 12
资源评论
- 星河️2024-05-08资源不错,很实用,内容全面,介绍详细,很好用,谢谢分享。
辣椒种子
- 粉丝: 3402
- 资源: 5725
上传资源 快速赚钱
- 我的内容管理 展开
- 我的资源 快来上传第一个资源
- 我的收益 登录查看自己的收益
- 我的积分 登录查看自己的积分
- 我的C币 登录后查看C币余额
- 我的收藏
- 我的下载
- 下载帮助
最新资源
- IMG_0694.GIF
- 基于图像的三维模型重建C++源代码+文档说明(高分课程设计)
- 基于聚焦法的工件立体测量方案,根据数据进行三维重建 使用HALCON处理图像,MATLAB拟合数据+源代码+数据集+效果图
- 锄战三国村 修改:货币使用不减 v1.10(2) 原创 (中文).apk
- 基于python实现的单目双目视觉三维重建+源代码+图像图片(高分课程设计)
- 基于C+++OPENCV的全景图像拼接源码(课程设计)
- 基于Python+OpenCV对多张图片进行全景图像拼接,消除鬼影,消除裂缝+源代码+文档说明+界面截图(高分课程设计)
- 基于C++实现的全景图像拼接源码(课程设计)
- 基于SIFT特征点提取和RASIC算法实现全景图像拼接python源码+文档说明+界面截图+详细注释(95分以上课程大作业)
- 基于matlab实现眼部判别的疲劳检测系统+源代码+全部数据+文档说明+详细注释+使用说明+截图(高分课程设计)
资源上传下载、课程学习等过程中有任何疑问或建议,欢迎提出宝贵意见哦~我们会及时处理!
点击此处反馈
安全验证
文档复制为VIP权益,开通VIP直接复制
信息提交成功