Motivation
Joint Probability
The r.v.s X
1
, · · · , X
n
are a sequence of discrete r.v.s, then joint pmf
P(X
1
= x
1
, · · · , X
n
= x
n
) can be computed as
If X
1
, · · · , X
n
are mutually independent r.v.s,
Q
n
i=1
P(X
i
= x
i
);
P(X
1
= x
1
)
Q
n
i=2
P(X
i
= x
i
|X
i−1
= x
i−1
, · · · , X
1
= x
1
).
Remarks:
When r.v.s are dependent, the generative model is hard to
scalable to large scale applications;
We need to simplify the model in practise.
The first-order correlation in language model can be defined as:
P(X
i
= x
i
|X
i−1
= x
i−1
, · · · , X
1
= x
1
) = P(X
i
= x
i
|X
i−1
= x
i−1
).
For financial applications, the stock price can be modeled by
t−order correlation P(X
i
= x
i
|X
i−1
= x
i−1
, · · · , X
1
= x
1
) =
P(X
i
= x
i
|X
i−1
= x
i−1
, · · · , X
i−t
= x
i−t
).
MING GAO (DaSE@ECNU) Algorithm Foundations of Data Science and Engineering Oct. 20, 2020 3 / 40
Motivation
Joint Probability
The r.v.s X
1
, · · · , X
n
are a sequence of discrete r.v.s, then joint pmf
P(X
1
= x
1
, · · · , X
n
= x
n
) can be computed as
If X
1
, · · · , X
n
are mutually independent r.v.s,
Q
n
i=1
P(X
i
= x
i
);
P(X
1
= x
1
)
Q
n
i=2
P(X
i
= x
i
|X
i−1
= x
i−1
, · · · , X
1
= x
1
).
Remarks:
When r.v.s are dependent, the generative model is hard to
scalable to large scale applications;
We need to simplify the model in practise.
The first-order correlation in language model can be defined as:
P(X
i
= x
i
|X
i−1
= x
i−1
, · · · , X
1
= x
1
) = P(X
i
= x
i
|X
i−1
= x
i−1
).
For financial applications, the stock price can be modeled by
t−order correlation P(X
i
= x
i
|X
i−1
= x
i−1
, · · · , X
1
= x
1
) =
P(X
i
= x
i
|X
i−1
= x
i−1
, · · · , X
i−t
= x
i−t
).
MING GAO (DaSE@ECNU) Algorithm Foundations of Data Science and Engineering Oct. 20, 2020 3 / 40
Motivation
Joint Probability
The r.v.s X
1
, · · · , X
n
are a sequence of discrete r.v.s, then joint pmf
P(X
1
= x
1
, · · · , X
n
= x
n
) can be computed as
If X
1
, · · · , X
n
are mutually independent r.v.s,
Q
n
i=1
P(X
i
= x
i
);
P(X
1
= x
1
)
Q
n
i=2
P(X
i
= x
i
|X
i−1
= x
i−1
, · · · , X
1
= x
1
).
Remarks:
When r.v.s are dependent, the generative model is hard to
scalable to large scale applications;
We need to simplify the model in practise.
The first-order correlation in language model can be defined as:
P(X
i
= x
i
|X
i−1
= x
i−1
, · · · , X
1
= x
1
) = P(X
i
= x
i
|X
i−1
= x
i−1
).
For financial applications, the stock price can be modeled by
t−order correlation P(X
i
= x
i
|X
i−1
= x
i−1
, · · · , X
1
= x
1
) =
P(X
i
= x
i
|X
i−1
= x
i−1
, · · · , X
i−t
= x
i−t
).
MING GAO (DaSE@ECNU) Algorithm Foundations of Data Science and Engineering Oct. 20, 2020 3 / 40
