东南大学 崇志宏:VAE和GAN技术要点(部分内容)

所需积分/C币:30 2017-08-18 11:00:48 2.82MB PDF

变分自动编码和对抗生成网络在深度学习领域取得广泛的影响力,其背后思想和方法技巧可以在wake-sleep框架下进行统一和相互改进。本来希望能够这方面的工作做个总结,但是发现这是一个大的topic,涉及的文献源远流长,很难把握其中的要义,只要做成一个部分的ppt,需要继续努力完整这个讨论!
VAE Problem class Directed graphical model x, observed variable z latent variables(continuous) 8: model parameters pe(x, z): joint PDF Factorized, differentiable Hard case: intractable posterior distribution pe( zx) e.g. neural nets as components We want fast approximate posterior inference per datapoint After inference, learning params is easy 东南大学数据与智能实验室(D&nte|Lab) Latent variable generative model latent variable model: learn a mapping from some latent variable z to a complicated distribution on a p(a)= p(a, x)dx where p(a, a)=p(a a)p(z) p(z)=something simple p(a x)=f(2)? Can we learn to decouple the true explanatory factors underlying the data distribution? E.g. separate identity and expression in face images 口■■■■口 1 mage from: Ward, A D, Hamarneh, G. 3D Surface Parameterization Using Manifold Learning for Medial Shape Representation, Conference on /mage Processing, Proc. of SPE Medical lmaging, 2007 IFT6266: Representation(Deep) Learning- Aaron Courville 10 Latent variable generative model latent variable model: learn a mapping from some latent variable z to a complicated distribution on a p(a)= p(a, x)dx where p(a, a)=p(a a)p(z) p(z)=something simple p(a x)=f(2)? Can we learn to decouple the true explanatory factors underlying the data distribution? E. g. separate identity and expression in face images p(x2)=N(z),6) X=f(z, e) 口■■■■口 1 mage from: Ward, A D, Hamarneh, G. 3D Surface Parameterization Using Manifold Learning for Medial Shape Representation, Conference on /mage Processing, Proc. of SPE Medical lmaging, 2007 IFT6266: Representation(Deep) Learning- Aaron Courville 10 ariational autoencoder(VAe) approach e Leverage neural networks to learn a latent variable model p(a)= p(a, a)dz where p(a, 2)=p(a a)p(e) P(z)=something simple p(a z)=f(z) ■■ f(a IFT6266: Representation(Deep) Learning- Aaron Courville What∨ AE can do? MNIST Frey Face dataset 22↑666oo0000000aaa 2色600000 日aa2 225556000042 日222223355666日2 日斗A22223333555日552 斗斗33223333355555J了 qqqq日373333335555P 日qqqq33333333555了了P 194999833333338833 111938883338888了了尸 CO00 9999988888日日88 7qq9q9了冒888866后6655 7qqqq9g3;55666666s ⑦qqq999g55666666ds 44乎49999566666 79"9?77111 Pose IFT6266: Representation(Deep) Learning Aaron Courville The inference/learning challenge o Where does z come from?- The classic directed model dilemma Computing the posterior p(z a )is intractable o We need it to train the directed model z f(e IFT6266: Representation(Deep) Learning Aaron Courville 13 Auto-Encoding variational Bayes dea Learn neural net to approximate the posterior q(zx) with variational parameters(p one-shot approximate inference akin to the recognition model in Wake-Sleep Construct estimator of the variational lower bound which we can optimize jointly w.r. t. op jointly with 8 Stochastic gradient ascent D P Kingma UNTVERSTTEIT VAN AMSTERDAM ariational Lower Bound of the marg. lik C(u,0, q)=logp(u; 0)-DkL (q(h u)lp(h v: 0)) C(u, 0, q)=Ehug [logp(h, v)]+H(q) logpe(x)=Kl(qzxllpzlx)+c(e, x where L(6,中;x)=Batz1x)ogpe(x,z)-lgg(团zx) L(u,, 0, q=logp(u; 0- DkL(q(h vlp(hv; 0) log p(u; 0)-Eh hag log 9 (h v) p(b|) -log p(v; 0)-Ehag log (hI u) P(配,U;6 (U;6) -logp(u; 0)-Ehvg [log q(h I v)-logp(h, U; 0)+logp(v; 0)] =-Ehwa llog q(h u)-logp(h, v; 0I

...展开详情

评论 下载该资源后可以进行评论 1

weixin_38313113 还不错,GAN的某些原理讲的比较透彻,有见地
2017-11-21
回复
上传资源赚积分,得勋章
相关内容推荐