高斯分布估计的各向异性自适应方差缩放资源-CSDN文库

164 浏览量 2021-03-13 17:49:00 上传评论收藏 731KB PDF 举报

在研究领域，高斯分布估计的各向异性自适应方差缩放（Anisotropic Adaptive Variance Scaling，简称AAVS）技术被提出，旨在优化传统高斯分布估计算法（Gaussian Estimation of Distribution Algorithm，简称GEDA）的性能。该技术的核心在于其能够同时调整GEDA中的方差及主搜索方向，从而提高算法的全局优化效率，并使其能够有效对抗过早收敛问题。高斯分布估计算法（GEDA）是一种基于概率模型的进化算法，它通过建立关于解空间的概率分布模型来指导搜索。然而，传统的GEDA存在一个主要问题：变量的方差快速减小，导致主搜索方向趋向于与适应度函数改进方向垂直，从而降低了搜索效率，并使得算法容易发生早熟收敛。针对这一问题，论文提出了一种新的方法——各向异性自适应方差缩放（AAVS）。该技术在不同特征向量方向上根据捕获的景观特性进行方差的各向异性缩放。具体来说，它使用了一种基于简单拓扑的检测方法来捕捉目标问题的景观特征，并据此来调节GEDA的方差和主搜索方向。这有助于确保算法沿着适应度函数改进的方向进行搜索，而不是与其垂直，从而克服了传统GEDA搜索效率低和容易早熟收敛的问题。 AAVS-EDA算法还引入了一个辅助的全局监控器。如果在某一代中没有实现改进，则通过缩小所有方差来确保算法的收敛性。实验结果显示，在CEC2014测试集的30个基准函数上，AAVS-EDA的性能优于传统的GEDAs。与其他最先进的进化算法相比，AAVS-EDA也展现出了高效率和竞争性。为了更深入地理解AAVS技术及其对高斯分布估计算法的改进，我们需要了解以下几个关键知识点： 1. 高斯分布估计算法（GEDA）：是一种基于概率分布模型的优化算法，该算法通过构建一个描述解空间分布的概率模型，并利用这个模型来生成新的候选解，以此指导搜索过程。在每一代的演化过程中，通过选择、交叉、变异等遗传操作产生新个体，然后利用适应度函数对这些个体进行评估，最后根据适应度选择策略选择优良的个体进行下一代的迭代。 2. 过早收敛问题：这是进化算法中常见的问题，当群体中的个体过于相似或者适应度分布过于集中时，算法可能丢失多样性，导致陷入局部最优解而不是全局最优解。这会使算法的搜索能力显著下降，无法找到更优的解。 3. 方差与搜索方向：在高斯分布估计算法中，方差代表了搜索的分布范围，它对算法的全局搜索能力有直接影响。主搜索方向与优化问题的适应度函数改进方向的关系决定了算法搜索效率。传统GEDA在方差快速减小的过程中，容易造成搜索方向与适应度函数的改进方向不一致，从而降低搜索效率。 4. 各向异性自适应方差缩放（AAVS）：这是为解决传统GEDA中变量方差下降快和搜索方向偏离问题而设计的技术。它通过基于简单拓扑的检测方法来捕获适应度景观特性，并在不同特征向量方向上根据这些特性进行方差的各向异性缩放，以此来同时调整方差和搜索方向。这种缩放保证了搜索过程中的方向性与多样性，提高了算法的全局优化能力和对抗早熟收敛的能力。 5. 全局监控器：AAVS-EDA算法中的全局监控器的作用是为了确保算法在搜索过程中的收敛性。当算法在某一代中没有取得进展时，全局监控器会缩小所有变量的方差，以推动算法向着更加集中的方向搜索，从而确保算法不会陷入停滞状态，且始终朝着最优解方向收敛。 AAVS技术对于传统高斯分布估计算法的改进是一个重要贡献，它提高了算法的搜索效率，增加了多样性，有效避免了早熟收敛的问题，并在实验中展示了其优越的全局优化效率和竞争力。对于研究者和实际应用者而言，这一技术的发展和完善，将有助于提升基于概率模型的进化算法在解决复杂优化问题中的表现。

资源推荐

资源详情

资源评论

Knowledge-Based Systems 146 (2018) 142–151

Contents lists available at ScienceDirect

Knowle dge-Base d Systems

journal homepage: www.elsevier.com/locate/knosys

Anisotropic adaptive variance scaling for Gaussian estimation of

distribution algorithm

Zhigang Ren

a , ∗

, Yongsheng Liang

, Lin Wang

, Aimin Zhang

, Bei Pang

, Biying Li

Department of Automation Science and Technology, School of Electronic and Information Engineering, Xi’an Jiaotong University, No.28 Xianning West Road,

Xi’an, Shaanxi 710049, China

School of Information Science and Technology, Northwest University, Xi’an, China

a r t i c l e i n f o

Article history:

Received 12 September 2017

Revised 25 January 2018

Accepted 1 February 2018

Available online 2 February 2018

Keywords:

Gaussian estimation of distribution

algorithm

Premature convergence

Search direction

Anisotropic adaptive variance scaling

a b s t r a c t

Traditional Gaussian estimation of distribution algorithms (EDAs) are confronted with issues that the vari-

able variances decrease fast and the main search direction tends to become perpendicular to the improve-

ment direction of the ﬁtness function, which reduces the search eﬃciency of Gaussian EDAs (GEDAs) and

makes them subject to premature convergence. In this paper, a novel anisotropic adaptive variance scal-

ing (AAVS) technique is proposed to improve the performance of traditional GEDAs and a new GEDA

variant named AAVS-EDA is developed. The advantages of AAVS over the existing variance scaling strate-

gies lie in its ability for tuning the variances and main search direction of GEDA simultaneously, which

are achieved by anisotropically scaling the variances along different eigendirections based on correspond-

ing landscape characteristics captured by a simple topology-based detection method. Besides, AAVS-EDA

also adopts an auxiliary global monitor to ensure its convergence by shrinking all the variances if no

improvement is achieved in a generation. The evaluation results on 30 benchmark functions of CEC2014

test suite demonstrate that AAVS-EDA possesses stronger global optimization eﬃciency than traditional

GEDAs. The comparison with other state-of-the-art evolutionary algorithms also shows that AAVS-EDA is

eﬃcient and competitive.

Introduction

As a special class of evolutionary algorithm (EA) [1] , estima-

tion of distribution algorithm (EDA) [2–4] is characterized by the

way of generating new solutions, i.e., sampling solutions according

to a probability distribution, but not through crossover and muta-

tion operators as other kinds of EAs. The probability distribution

employed in each generation is generally estimated from the rel-

atively high-quality solutions obtained in previous generations. It

can capture the structure of the problem being solved to a cer-

tain extent and consequently guide the algorithm to more promis-

ing solution regions. During the past few decades, EDAs attracted

much research effort and achieved great success in both combina-

torial and continuous domains [5,6] . In this paper, EDAs for contin-

uous domain are studied.

Continuous EDAs usually adopt Gaussian probability model

[3,4] and histogram model [7] as the basic probability model. Ac-

cording to the way in representing variable dependencies, Gaussian

models for EDAs can be further categorized into three kinds. The

∗

Corresponding author.

E-mail address: renzg@mail.xjtu.edu.cn (Z. Ren).

simplest one is the univariate model which neglects all the variable

dependencies. A representative algorithm with this type of model

is the univariate marginal distribution algorithm (UMDA

) [3,4] . A

slightly more sophisticated model is the one that just considers

some important variable dependencies. To identify these depen-

dencies, Bayesian factorization is usually employed [3,8] . The mul-

tivariate model takes all the variable dependencies into account. A

representative algorithm of this type is estimation of multivariate

normal density algorithm (EMNA

) [4] .

Although possessing clear physical concept, traditional Gaus-

sian EDAs (GEDAs) often suffer from premature convergence [8,11–

15]

. Early improvement studies attributed this defect to the rapid

shrink of variances and developed many variance scaling strate-

gies. Ocenasek et al. [8] proposed a variance adaption operator for

the mixed Bayesian optimization algorithm [9] based on the well-

known 1/5-success-rule [10] . Yuan and Gallagher [11] claimed that

the performance of GEDA could be improved on certain problems

by compulsively keeping the variances at a value of at least 1. Pošík

[12] suggested enlarging variances by a constant factor. Grahl et al.

[13] proposed an adaptive variance scaling (AVS) strategy which in-

creases the variances when the best solution improves, otherwise

reduces them. Nevertheless, AVS does not directly tune variances

in each generation unless it identiﬁes that the algorithm is travers-

https://doi.org/10.1016/j.knosys.2018.02.001

Z. Ren et al. / Knowledge-Based Systems 146 (2018) 142–151 143

ing a slope. To achieve this, Grahl and his coworkers successively

developed two identiﬁcation strategies, i.e., the strategies based

on correlation triggering [13] and standard deviation ratio (SDR)

[14] . Cai et al. [15] developed a different type of variance scaling

method named cross entropy adaptive variance scaling, which cal-

culates the scaling factor by minimizing the cross entropy between

the current probability model and the predicted model for the next

generation.

Besides directly tuning variances, some researchers achieved

variance scaling by modifying the eigenvalues of the estimated co-

variance matrix. Wagner et al. [16] proposed an eigenspace EDA

(EEDA) which adjusts variances by replacing the minimum eigen-

value with the maximum one. Dong et al. [17] developed an eigen

decomposition framework for multivariate GEDA and claimed that

most variance scaling methods by then could be uniﬁed within

their framework by applying different eigenvalue tuning strategies.

Liu et al. [18] introduced principal component analysis into EDA

(PCA-EDA) and tried to avoid premature convergence by regulating

the maximum eigenvalue.

It is easily comprehensible that the eﬃciency of GEDA depends

not only on its search scope, but also on its search directions.

Unfortunately, it has been shown that, without ﬁne intervention,

the main search direction of GEDA tends to become perpendicu-

lar to the ﬁtness improvement direction [15,19] , which greatly re-

duces its search eﬃciency. To remedy this defect, some researchers

made beneﬁcial attempts by improving the estimation method for

the covariance matrix. The covariance matrix adaptation evolu-

tion strategy (CMA-ES) [20] , which can be considered as a spe-

cial EDA, employs a sophisticated covariance matrix estimation

method, where the rank- μ-update operator updates the covariance

matrix using the weighted high-quality solutions in the current

generation and the corresponding mean in the last generation. By

this means, the variance along the gradient direction can be in-

creased. Bosman et al. [19] proposed an anticipated mean shift

(AMS) operator which estimates the covariance matrix by shifting

part of selected solutions along the anticipated gradient direction

such that the main search direction can be corrected to a certain

extent. Bosman et al. integrated AVS, SDR and AMS together and

developed a powerful EDA variant known as AMaLGaM [19] . Ren

et al. [21] improved the original AMS operator by directly shifting

the mean of selected solutions and taking the shifted mean as the

center when estimating the covariance matrix. Liang et al. [22] re-

cently reported an enhanced GEDA, in which the inferior solutions

in current generation are repaired and utilized to estimate the co-

variance matrix such that the search directions could be adaptively

adjusted.

In addition to scaling variances and improving the covariance

matrix estimation method, extensive effort s have been made to en-

hance the performance of EDA. Xu [23] combined EDA with chaos

perturbation operator for the purpose of enhancing the population

diversity. Chen et al. [24] proposed a fast interactive EDA which ex-

tracts user’s preference on the decision variables from historical in-

formation to reduce the initial search space to a preferred subspace

such that the search process can be accelerated. Fang et al. [25] de-

veloped a mean shift strategy to speed up the convergence of EDA.

Zhou et al. [7] suggested combining EDA with cheap and expen-

sive local search. Auger and Hansen [26] developed a restart CMA-

ES with increasing population size (IPOP-CMAES). Although IPOP-

CMAES was developed a decade years ago, it was recently reported

that IPOP-CMAES is still competitive with many other state-of-the-

art EAs proposed in recent years [27] . Karshenas et al. [28] investi-

gated the effect of regularization method on the model learning

process of GEDA. Santana et al. [29] tried to improve EDA with

the help of new selection strategies. Instead of using Gaussian and

histogram model, Zhang and Zeng [30] , Qian et al. [31] and Pour-

MohammadBagher et al. [32] adopted particle ﬁlter, Copula the-

ory and probabilistic graphical model, respectively, to capture the

distribution of good solutions. Aiming at seeking multiple solu-

tions, the techniques of detecting promising areas [33] and nich-

ing [34,35] were introduced to EDA to enhance its performance on

multimodal problems. Moreover, EDAs have been integrated with

other EAs like particle swarm optimization (PSO) [36] and differ-

ential evolution (DE) [37] to fuse their advantages together. Theo-

retical researches have also been done to characterize the behav-

iors of EDA. Rastegar [38] analyzed the convergence probability of

two univariate EDAs and showed a suﬃcient condition for the con-

vergence. Echegoyen et al. [39] comprehensively studied the rela-

tionship between the behaviors of EDA and the solution space of

optimization problems.

To sum up, EDAs have been improved signiﬁcantly in the past

decades, but there are still some shortcomings. So far, most exist-

ing variance scaling strategies are able to adjust the search scope

of GEDA, but can hardly change its search directions. This is one

of the key issues that severely restrict the performance of GEDA,

but has not been fully recognized and studied. Consequently, few

related work was reported in recent years. CMA-ES and AMaLGaM

achieve great success by comprehensively regulating their search

scopes and directions. However, their algorithmic frameworks are

so complex that it is too diﬃcult for practitioners to understand

the mechanisms therein, not to mention setting corresponding pa-

rameters. As for some other EDA variants, their performance also

highly depends on the search eﬃciency of basic EDA, then it is

hopeful that they would be further improved if the eﬃciency of

EDA can be increased.

Taking enhancing EDA with some simple operations as the goal,

this paper proposes an anisotropic adaptive variance scaling (AAVS)

strategy and develops a novel GEDA variant named as AAVS-EDA.

Different from most existing variance scaling strategies which ad-

just all the variances by a same factor, AAVS ﬁrst captures the land-

scape of the problem being solved along each eigendirection with a

simple topology-based detection method, then adaptively tunes the

variances along different directions according to the correspond-

ing detection results. If a slope is detected along an eigendirection,

then AAVS enlarges the corresponding variance. On the contrary, if

a valley is detected, then AAVS keeps the corresponding variance

unchanged. By this means, the search speed along a slope can be

quickened, and the ﬁne search around a valley can be achieved.

More importantly, proﬁting from its anisotropic scaling, AAVS is

able to make the main search direction of EDA naturally become

consistent with the ﬁtness improvement direction. To ensure con-

vergence, AAVS-EDA also adopts an auxiliary global monitor which

shrinks all the variances if no improvement is achieved in a gener-

ation. Thanks to these ﬁne properties, AAVS-EDA shows desirable

performance on a variety of benchmark functions.

The remainder of this paper is organized as follows.

Section 2 brieﬂy reviews the basic knowledge of GEDA.

Section 3 describes AAVS and the resulting AAVS-EDA in de-

tail. Section 4 presents the experiment settings and analyzes the

experiment results. The conclusions are ﬁnally drawn in Section 5 .

Basic knowledge of GEDA

As a model-based EA, EDA assumes that good solutions approx-

imately obey a certain probability distribution over the solution

space. During the search process, it tries to learn this distribution

and generate new solutions according to the learning results [2–4] .

The general framework of EDA is outlined in Algorithm 1 .

EDA usually starts with an initial population which is ﬁlled with

some randomly generated solutions. After the evaluation, those rel-

atively good solutions are selected generally according to a trunca-

tion selection rule. Then a new probability model is built to pro-

duce solutions for the next generation. EDA executes this iterative

剩余9页未读，继续阅读

评论收藏

内容反馈

weixin_38663452

粉丝: 5
资源: 923

高斯分布估计的各向异性自适应方差缩放

基于图像局部方差分布的自适应反锐化掩模算法

自适应控制-最小方差自校正控制

基于自适应方差估计的双变量收缩去噪 (2010年)

自适应窗长方差估计在多传感器数据融合中的应用* (2008年)

基于方差分量估计的自适应融合滤波在GNSS组合定位中的应用

基于方差分量估计原理的自适应卡尔曼滤波及其应用_胡丛玮.caj

基于自适应噪声方差估计的泊松噪声去噪方法

自适应方差双边滤波

自适应高斯滤波,自适应高斯滤波器对负荷去噪怎么做,matlab

自适应高斯滤波_自适应高斯滤波

复高斯分布的数学基础理论.pdf

C++编程实现高斯分布随机数的产生

白板推导2 数学基础（高斯分布）1

double_Gaussian_model_高斯分布_双高斯_双高斯分布_

随机信号.rar_泊松信号_泊松分布_瑞利分布_高斯分布_高斯随机信号

sigama 自适应高斯滤波器, matlab 程序

uniformgauss.rar_BOX-MULLER_BOX_MULLER_高斯分布 matlab_高斯分布MATLAB_高斯

EM算法在混合高斯分布中的应用MATLAB代码

P37_自适应高斯平滑算法_水声目标识别_

异常检测（高斯分布模型）+测试数据

方差自适应卡尔曼滤波实例.zip

均值为u，方差为g的高斯白噪声

机器学习异常检测（高斯分布）.7z

自适应高斯滤波实现图像去噪.zip

根据均值和方差生成高斯噪声，给图像叠加高斯噪声

高斯分布来建立背景模型

自适应控制--广义最小方差控制

matlab开发-高斯正态分布概率密度函数

最新资源