<div align="center">
<img src="https://github.com/fmfn/BayesianOptimization/blob/master/examples/func.png"><br><br>
</div>
# Bayesian Optimization
![tests](https://github.com/fmfn/BayesianOptimization/actions/workflows/run_tests.yml/badge.svg)
[![Codecov](https://codecov.io/github/fmfn/BayesianOptimization/badge.svg?branch=master&service=github)](https://codecov.io/github/fmfn/BayesianOptimization?branch=master)
[![Pypi](https://img.shields.io/pypi/v/bayesian-optimization.svg)](https://pypi.python.org/pypi/bayesian-optimization)
Pure Python implementation of bayesian global optimization with gaussian
processes.
* PyPI (pip):
```console
$ pip install bayesian-optimization
```
* Conda from conda-forge channel:
```console
$ conda install -c conda-forge bayesian-optimization
```
This is a constrained global optimization package built upon bayesian inference
and gaussian process, that attempts to find the maximum value of an unknown
function in as few iterations as possible. This technique is particularly
suited for optimization of high cost functions, situations where the balance
between exploration and exploitation is important.
## Quick Start
See below for a quick tour over the basics of the Bayesian Optimization package. More detailed information, other advanced features, and tips on usage/implementation can be found in the [examples](https://github.com/fmfn/BayesianOptimization/tree/master/examples) folder. I suggest that you:
- Follow the
[basic tour notebook](https://github.com/fmfn/BayesianOptimization/blob/master/examples/basic-tour.ipynb)
to learn how to use the package's most important features.
- Take a look at the
[advanced tour notebook](https://github.com/fmfn/BayesianOptimization/blob/master/examples/advanced-tour.ipynb)
to learn how to make the package more flexible, how to deal with categorical parameters, how to use observers, and more.
- Check out this
[notebook](https://github.com/fmfn/BayesianOptimization/blob/master/examples/visualization.ipynb)
with a step by step visualization of how this method works.
- To understand how to use bayesian optimization when additional constraints are present, see the
[constrained optimization notebook](https://github.com/fmfn/BayesianOptimization/blob/master/examples/constraints.ipynb).
- Explore this [notebook](https://github.com/fmfn/BayesianOptimization/blob/master/examples/exploitation_vs_exploration.ipynb)
exemplifying the balance between exploration and exploitation and how to
control it.
- Go over this [script](https://github.com/fmfn/BayesianOptimization/blob/master/examples/sklearn_example.py)
for examples of how to tune parameters of Machine Learning models using cross validation and bayesian optimization.
- Explore the [domain reduction notebook](https://github.com/fmfn/BayesianOptimization/blob/master/examples/domain_reduction.ipynb) to learn more about how search can be sped up by dynamically changing parameters' bounds.
- Finally, take a look at this [script](https://github.com/fmfn/BayesianOptimization/blob/master/examples/async_optimization.py)
for ideas on how to implement bayesian optimization in a distributed fashion using this package.
## How does it work?
Bayesian optimization works by constructing a posterior distribution of functions (gaussian process) that best describes the function you want to optimize. As the number of observations grows, the posterior distribution improves, and the algorithm becomes more certain of which regions in parameter space are worth exploring and which are not, as seen in the picture below.
![BayesianOptimization in action](./examples/bo_example.png)
As you iterate over and over, the algorithm balances its needs of exploration and exploitation taking into account what it knows about the target function. At each step a Gaussian Process is fitted to the known samples (points previously explored), and the posterior distribution, combined with a exploration strategy (such as UCB (Upper Confidence Bound), or EI (Expected Improvement)), are used to determine the next point that should be explored (see the gif below).
![BayesianOptimization in action](./examples/bayesian_optimization.gif)
This process is designed to minimize the number of steps required to find a combination of parameters that are close to the optimal combination. To do so, this method uses a proxy optimization problem (finding the maximum of the acquisition function) that, albeit still a hard problem, is cheaper (in the computational sense) and common tools can be employed. Therefore Bayesian Optimization is most adequate for situations where sampling the function to be optimized is a very expensive endeavor. See the references for a proper discussion of this method.
This project is under active development, if you find a bug, or anything that
needs correction, please let me know.
Basic tour of the Bayesian Optimization package
===============================================
## 1. Specifying the function to be optimized
This is a function optimization package, therefore the first and most important ingredient is, of course, the function to be optimized.
**DISCLAIMER:** We know exactly how the output of the function below depends on its parameter. Obviously this is just an example, and you shouldn't expect to know it in a real scenario. However, it should be clear that you don't need to. All you need in order to use this package (and more generally, this technique) is a function `f` that takes a known set of parameters and outputs a real number.
```python
def black_box_function(x, y):
"""Function with unknown internals we wish to maximize.
This is just serving as an example, for all intents and
purposes think of the internals of this function, i.e.: the process
which generates its output values, as unknown.
"""
return -x ** 2 - (y - 1) ** 2 + 1
```
## 2. Getting Started
All we need to get started is to instantiate a `BayesianOptimization` object specifying a function to be optimized `f`, and its parameters with their corresponding bounds, `pbounds`. This is a constrained optimization technique, so you must specify the minimum and maximum values that can be probed for each parameter in order for it to work
```python
from bayes_opt import BayesianOptimization
# Bounded region of parameter space
pbounds = {'x': (2, 4), 'y': (-3, 3)}
optimizer = BayesianOptimization(
f=black_box_function,
pbounds=pbounds,
random_state=1,
)
```
The BayesianOptimization object will work out of the box without much tuning needed. The main method you should be aware of is `maximize`, which does exactly what you think it does.
There are many parameters you can pass to maximize, nonetheless, the most important ones are:
- `n_iter`: How many steps of bayesian optimization you want to perform. The more steps the more likely to find a good maximum you are.
- `init_points`: How many steps of **random** exploration you want to perform. Random exploration can help by diversifying the exploration space.
```python
optimizer.maximize(
init_points=2,
n_iter=3,
)
```
| iter | target | x | y |
-------------------------------------------------
| 1 | -7.135 | 2.834 | 1.322 |
| 2 | -7.78 | 2.0 | -1.186 |
| 3 | -19.0 | 4.0 | 3.0 |
| 4 | -16.3 | 2.378 | -2.413 |
| 5 | -4.441 | 2.105 | -0.005822 |
=================================================
The best combination of parameters and target value found can be accessed via the property `optimizer.max`.
```python
print(optimizer.max)
>>> {'target': -4.441293113411222, 'params': {'y': -0.005822117636089974, 'x': 2.104665051994087}}
```
While the list of all parameters probed and their corresponding target values is available via the property `optimizer.res`.
```python
for i, res in enumerate(optimizer.res):
print("Iteration {}: \n\t{}".for
使用高斯过程进行全局优化的Python实现。_Python_下载.zip
版权申诉
36 浏览量
2023-04-26
10:47:35
上传
评论
收藏 16.96MB ZIP 举报
快撑死的鱼
- 粉丝: 1w+
- 资源: 9152
最新资源
- Javascript重点知识掌握
- 305建筑结构水电欧式6套(14.5x20.2)\施工图\C型施工图二层、三层空调平面图(二)-6.dwg
- 305建筑结构水电欧式6套(14.5x20.2)\施工图\C型施工图地下层、首层空调平面图(一)-03.dwg
- 305建筑结构水电欧式6套(14.5x20.2)\施工图\C型施工图地下层、首层空调平面图(二)-4.dwg
- “Flappy Bird”是一个基于pygame库开发的经典飞行小游戏
- 305建筑结构水电欧式6套(14.5x20.2)\施工图\C型施工图超级多联系统原理图-07.dwg
- 305建筑结构水电欧式6套(14.5x20.2)\施工图\C型施工图\结构楼梯与目录.dwg
- 305建筑结构水电欧式6套(14.5x20.2)\施工图\C型施工图\结构C型结构平面.dwg
- 305建筑结构水电欧式6套(14.5x20.2)\施工图\C型施工图\建筑1-13C型大样031026.dwg
- 305建筑结构水电欧式6套(14.5x20.2)\施工图\C型施工图\建筑1-12门窗详图031026.dwg
资源上传下载、课程学习等过程中有任何疑问或建议,欢迎提出宝贵意见哦~我们会及时处理!
点击此处反馈