Long-termForecastingwithTiDE-Time-seriesDenseEncoder_Transformer模型在时间序列预测领域的应用资源-CSDN文库

需积分: 5 93 浏览量 2023-10-02 22:01:04 上传评论收藏 740KB PDF 举报

资源推荐

资源详情

资源评论

Long-term Forecasting with

TiDE: Time-series Dense Encoder

Abhimanyu Das

, Weihao Kong

, Andrew Leach

, Rajat Sen

, and Rose Yu

2, 3

Google Research

Google Cloud

University of California, San Diego

April 18, 2023

Abstract

Recent work has shown that simple linear models can outperform several Transformer based

approaches in long term time-series forecasting. Motivated by this, we propose a Multi-layer

Perceptron (MLP) based encoder-decoder model,

me-series

ense

ncoder (TiDE), for long-

term time-series forecasting that enjoys the simplicity and speed of linear models while also

being able to handle covariates and non-linear dependencies. Theoretically, we prove that the

simplest linear analogue of our model can achieve near optimal error rate for linear dynamical

systems (LDS) under some assumptions. Empirically, we show that our method can match or

outperform prior approaches on popular long-term time-series forecasting benchmarks while

being 5-10x faster than the best Transformer based model.

1 Introduction

Long-term forecasting, which is to predict several steps into the future given a long context or

look-back, is one of the most fundamental problems in time series analysis, with broad applications

in energy, ﬁnance, and transportation. Deep learning models [

WXWL21

NNSK22

] have emerged

as the preferred approach for forecasting rich, multivariate, time series data, outperforming classical

statistical approaches such as ARIMA or GARCH [

BJRL15

]. In several forecasting competitions

such as the M5 competition [

MSA20

] and IARAI Traﬃc4cast contest [

KKJ

], almost all the

winning solutions are based on deep neural networks.

Various neural network architectures have been explored for forecasting, ranging from recurrent

neural networks to convolutional networks to graph-neural-networks. For sequence modeling

tasks in domains such as language, speech and vision, Transformers [

VSP

] have emerged as

the most successful deep learning architecture, even outperforming recurrent neural networks

(LSTMs)[

HS97

]. Subsequently, there has been a surge of Transformer-based forecasting papers

[

WXWL21

ZZP

ZMW

] in the time-series community that have claimed state-of-the-art

(SoTA) forecasting performance for long-horizon tasks. However recent work [ZCZX23] has shown

that these Tranformers-based architectures may not be as powerful as one might expect for time

Authors are listed in alphabetical order.

arXiv:2304.08424v1 [stat.ML] 17 Apr 2023

series forecasting, and can be easily outperformed by a simple linear model on forecasting bench-

marks. Such a linear model however has deﬁciencies since it is ill-suited for modeling non-linear

dependencies among the time-series sequence and the time-independent covariates. Indeed, a

very recent paper [

NNSK22

] proposed a new Transformer-based architecture that obtains SoTA

performance for deep neural networks on the standard multivariate forecasting benchmarks.

In this paper, we present a simple and eﬀective deep learning architecture for forecasting that

obtains superior performance when compared to existing SoTA neural network based models on the

long-term time series forecasting benchmarks. Our Multi-Layer Perceptron (MLP)-based model is

embarrassingly simple without any self-attention, recurrent or convolutional mechanism. Therefore,

it enjoys a linear computational scaling in terms of the context and horizon lengths unlike many

Transformer based solutions.

The main contributions of this work are as follows:

•

We propose the

me-series

ense

ncoder (TiDE) model architecture for long-term time

series forecasting. TiDE encodes the past of a time-series along with covariates using dense

MLPs and then decodes time-series along with future covariates, again using dense MLPs.

•

We analyze the simplest linear analogue of our model and prove that this linear model can

achieve near optimal error rate in linear dynamical systems (LDS) [

Kal63

] when the design

matrix of the LDS has maximum singular value bounded away from 1. We empirically verify

this on a simulated dataset where the linear model outperforms LSTMs and Transformers.

•

On popular real-world long-term forecasting benchmarks, our model achieves better or similar

performance compared to prior neural network based baselines (

10% lower Mean Squared

Error on the largest dataset). At the same time, TiDE is 5x faster in terms of inference and

more than 10x faster in training when compared to the best Transformer based model.

2 Related Work

In this section we will focus on the prior work on deep neural network models for long-term forecasting.

The comparison with classical methods such as ARIMA has been discussed in prior work [

WXWL21

]

and references there in. LongTrans [

LJX

] uses attention layer with LogSparse design to capture

local information with near linear space and computational complexity. Informer [

ZZP

] uses

the ProbSparse self-attention mechanism to achieve sub-quadratic dependency on the lenght of

the context. Autoformer [

WXWL21

] uses trend and seasonal decomposition with sub-quadratic

self attention mechanism. FEDFormer [

ZMW

] uses a frequency enchanced structure while

Pyraformer [

LYL

] uses pyramidal self-attention that has linear complexity and can attend to

diﬀerent granularities.

Recently, [

ZCZX23

] proposes DLinear, a simple linear model that surprisingly outperforms many of

the Transformer based models mentioned above. Dlinear learns a linear mapping from context to

horizon, pointing to deﬁciencies in sub-quadratic approximations to the self-attention mechanism.

Indeed, a very recent model, PatchTST [

NNSK22

] has shown that feeding contiguous patches of

time-series as tokens to the vanilla self-attention mechanism can beat the performance of DLinear

in long-term forecasting benchmarks.

3 Problem Setting

Before we describe the problem setting we will need to setup some general notation.

3.1 Notation

We will denote matrices by bold capital letters like

X ∈ R

N×T

. The slice notation

denotes the

set

{i, i

+ 1

, ···j}

and [

] :=

{

, ··· , n}

. The individual rows and columns are always treated as

column vectors unless otherwise speciﬁed. We can also use sets to select sub-matrices i.e

[

I, J

]

denotes the sub-matrix with rows in

and columns in

, j

] means selecting the

-th column

while

[

:] means the

-th row. The notation [

;

] will denote the concatenation of the two column

vectors and the same notation can be used for matrices along a dimension.

3.2 Multivariate Forecasting

In this section we ﬁrst abstract out the core problem in long-term multivariate forecasting. There

are

time-series in the dataset. The look-back of the

-th time-series will be denoted by

(i)

1:L

, while

the horizon is denoted by

(i)

L+1:L+H

. The task of the forecaster is to predict the horizon time-points

given access to the look-back.

In many forecasting scenarios, there might be dynamic and static covariates that are known in

advance. With slight abuse of notation, we will use

(i)

∈ R

to denote the

-dimensional dynamic

covariates of time-series

at time

. For instance, they can be global covariates (common to all

time-series) such as day of the week, holidays etc or speciﬁc to a time-series for instance the discount

of a particular product on a particular day in a demand forecasting use case. We can also have

static attributes of a time-series denoted by

(i)

such as features of a product in retail demand

forecasting that do not change with time. In many applications, these covariates are vital for

accurate forecasting and a good model architecture should have provisions to handle them.

The forecaster can be thought of as a function that maps the history

(i)

1:L

, the dynamic covariates

(i)

1:L+H

and the static attributes a

(i)

to an accurate prediction of the future, i.e.,

f :



(i)

1:L

i=1

(i)

1:L+H

i=1

(i)

i=1



−→

(i)

L+1:L+H

i=1

. (1)

The accuracy of the prediction will be measured by a metric that quantiﬁes their closeness to the

actual values. For instance, if the metric is Mean Squared Error (MSE), then the goodness of ﬁt is

measured by,

MSE



(i)

L+1:L+H

i=1

(i)

L+1:L+H

i=1



i=1



(i)

L+1:L+H

−

(i)

L+1:L+H



. (2)

4 Model

Recently, it has been observed that simple linear models [

ZCZX23

] can outperform Tranformer based

models in several long-term forecasting benchmarks. On the other hand, linear models will fall short

剩余17页未读，继续阅读

评论收藏

内容反馈

AIFEx

粉丝: 34
资源: 34

Long-term Forecasting with TiDE- Time-series Dense Encoder

最新资源

Long-term Forecasting with TiDE- Time-series Dense Encoder

Deep-Time-Series-Forecasting-with-Python-An-Intuitive-Introduction-to-Deep-Learning-for-Applied-Time-Series-Modeling.pdf.pdf

Short-Term Forecasting of Electric Loads_pdf_forecasting_short_s

Regularization methods for the short-term forecasting of the Ita

Code for Introduction to Time Series Forecasting with Python.zip

时间序列分析 预测与控制 Time Series Analysis -- Forecasting and Control, 3rd Edition

Deep Time Series Forecasting with Python.pdf

improve forecasting accuracy for short-term load series.

Introduction to time series+ Deep Time Series Forecasting with Python

Deep Learning for Time Series Forecasting - by Jason Brownlee

Time-series-forecasting-via-deep-reinforcement-learning

time-series-forecasting-with-python:以用例为中心的教程，使用python进行时间序列预测

SVM_Short-term-Load-Forecasting.zip

Optimization of Short-term Load Forecasting with Extrapolation Algorithm Based on Fractal Theory

Neural machine-based forecasting of chaotic dynamics.pdf

SVM Short-term-Load-Forecasting-配套文档及源码.rar

SVM_Short-term-Load-Forecasting.zip_load forecasting_svm优化_电力_负荷

Examining COVID-19 Forecasting using Spatio-Temporal Graph Neural Networks.pdf

Warp to the Future - Joint Forecasting of Features and Feature M

SVM_Short-term-Load-Forecasting2.zip_SVM 负荷预测_SVM预测优化_svm预测_某地负荷

Academic+Phrasebank+2021+Edition+_中英文对照.pdf

2018年中国研究生数学建模C题恐怖袭击获奖论文

基于python的超市管理系统的设计与实现毕业论文+项目文档源码

基于暗通道先验的单幅图像去雾（何凯明）论文翻译

1000套计算机毕业设计带源码

数模国赛word模板.zip

2023高教社数学建模C题 - 蔬菜类商品的自动定价与补货决策【数据处理详细代码】

2021年国赛A题（FAST主动反射面形状调节）论文+代码材料.zip

Python大作业（包含论文）——可打包的双人五子棋程序

软考 系统分析师论文 范文

最新资源

时间序列分析预测与控制 Time Series Analysis -- Forecasting and Control, 3rd Edition

软考系统分析师论文范文