<p align="center" width="100%">
<img width="80%" src="docs/images/haven_logo.png">
</p>
## **Create end-to-end ML projects with the following 4 steps**
### 0. Install
```
pip install -r requirements.txt
```
### 1. Create trainval.py with the following code
```python
import tqdm, os
from haven import haven_examples as he
from haven import haven_wizard as hw
# 0. define trainval function
def trainval(exp_dict, savedir, args):
"""
exp_dict: dictionary defining the hyperparameters of the experiment
savedir: the directory where the experiment will be saved
args: arguments passed through the command line
"""
# 1. Create data loader and model
train_loader = he.get_loader(name=exp_dict['dataset'], split='train',
datadir=os.path.dirname(savedir),
exp_dict=exp_dict)
model = he.get_model(name=exp_dict['model'], exp_dict=exp_dict)
# 2. load checkpoint
chk_dict = hw.get_checkpoint(savedir)
# 3. Add main loop
for epoch in range(chk_dict['epoch'], 3):
# 4. train for one epoch
for batch in tqdm.tqdm(train_loader):
train_dict = model.train_on_batch(batch)
# 5. get and save metrics
score_dict = {'epoch':epoch, 'acc': train_dict['train_acc'], 'loss':train_dict['train_loss']}
chk_dict['score_list'] += [score_dict]
hw.save_checkpoint(savedir, score_list=chk_dict['score_list'])
print('Experiment done')
# 6. create main
if __name__ == '__main__':
# 7. define a list of experiments
exp_list = [{'dataset':'mnist', 'model':'linear', 'lr':lr}
for lr in [1e-3, 1e-4]]
# 8. Launch experiments using magic command
hw.run_wizard(func=trainval, exp_list=exp_list)
```
### 2. Run the experiments
```
python trainval.py --reset 1
```
### 3. Visualize
Step 4 creates `trainval_results.ipynb`, open the file on Jupyter to get tables and plots
![](docs/images/table1.png)
You can launch Jupyter with,
```bash
jupyter nbextension enable --py widgetsnbextension --sys-prefix
jupyter notebook
```
### 4. Run the experiments in cluster
Using the `api`,
```
python trainval.py --run_jobs 1 --reset 1
```
Using the `cli`,
```
haven --fname trainval.py --function trainval --dataset mnist --model lenet --lr 0.001,0.1
```
## Projects based on Haven-AI
- COVID19: https://www.elementai.com/news/2020/accelerating-the-creation-of-ai-models-to-combat-covid-19-with-element-ai-orkestrator?fbclid=IwAR33nO8gqq7Bf8F4NyG3WYlH3c7t7QkgPAZEru4UpP2FGzSYfHZ5nnPLKZI
- LCFCN: https://github.com/ElementAI/LCFCN
- Embedding Propagation: https://github.com/ElementAI/embedding-propagation
- Bilevel Augmentation: https://github.com/ElementAI/bilevel_augment
- SSN optimizer: https://github.com/IssamLaradji/ssn
- SLS optimizer: https://github.com/IssamLaradji/sls
- Ada SLS optimizer: https://github.com/IssamLaradji/ada_sls
- SPS optimizer: https://github.com/IssamLaradji/sps
- Fish Dataset: https://github.com/alzayats/DeepFish
- Covid Weak Supervision: https://github.com/IssamLaradji/covid19_weak_supervision
## Motivation
- Haven is a library for building, managing and visualizing large-scale reproducible experiments. It helps developers establish a workflow that allows them to quickly prototype a reliable codebase. It also helps them easily scale that codebase to one that can run, manage, and visualize thousands of experiments seamlessly.
- The library provides a wide range of functionality including best practices for defining experiments, checkpointing, visualizing and debugging experiments, and ensuring reproducible benchmarks.
- This library could strongly boost productivity for building great products, winning machine learning competitions, and getting research papers published.
- The only structure required is that each experiment has the following.
* `<savedir_base>/<exp_id>/exp_dict.json` that defines a single set of hyperparamters as a python `dict`.
* `exp_id` is the hash of that hyperparameter python `dict`.
* `<savedir_base>/<exp_id>/score_list.pkl` that has a list of dicts where each dict contains metrics as keys with scores as their values.
## Expected structure of a Havenized project
```
project/
├── src/
│ ├── __init__.py
│ ├── datasets.py
│ └── models.py
├── scripts/
│ ├── visualize_mnist.py
│ └── train_on_single_image.py
├── exp_configs.py
├── README.md
└── trainval.py
```
## Contributing
We love contributions!
没有合适的资源?快使用搜索试试~ 我知道了~
资源推荐
资源详情
资源评论
收起资源包目录
haven-ai-0.3.0.tar.gz (56个子文件)
haven-ai-0.3.0
PKG-INFO 283B
haven_cli
command_line.py 0B
__init__.py 347B
haven 3KB
setup.cfg 38B
setup.py 881B
haven
haven_wizard.py 7KB
haven_jobs
subprocess_manager.py 0B
__init__.py 17KB
toolkit_manager.py 4KB
slurm_manager.py 0B
borgy_manager.py 3KB
haven_deploy
haven_deploy.py 0B
__init__.py 0B
haven_chk.py 4KB
haven_utils
string_utils.py 521B
pdf_utils.py 822B
__init__.py 23KB
file_utils.py 3KB
image_utils.py 0B
exp_utils.py 13KB
haven_tools
ssh_start.py 4KB
ap_metrics.py 20KB
transformers.py 2KB
proposals.py 0B
__init__.py 25B
samplers.py 0B
ann_utils.py 0B
haven_share
dropbox_share.py 4KB
__init__.py 0B
gdrive_share.py 308B
haven_jupyter
images_tab.py 3KB
utils_widgets.py 0B
plots_tab.py 10KB
__init__.py 10KB
latex_tab.py 4KB
tables_tab.py 8KB
share_tab.py 4KB
__init__.py 0B
haven_img.py 11KB
haven_results
tables_scores.py 8KB
plots_bar.py 2KB
plots_line.py 15KB
pretty_plot.py 14KB
__init__.py 16KB
images_fig.py 4KB
haven_examples
models.py 0B
datasets.py 0B
__init__.py 3KB
README.md 4KB
haven_ai.egg-info
PKG-INFO 283B
requires.txt 234B
not-zip-safe 1B
SOURCES.txt 2KB
top_level.txt 16B
dependency_links.txt 1B
共 56 条
- 1
资源评论
挣扎的蓝藻
- 粉丝: 13w+
- 资源: 15万+
上传资源 快速赚钱
- 我的内容管理 展开
- 我的资源 快来上传第一个资源
- 我的收益 登录查看自己的收益
- 我的积分 登录查看自己的积分
- 我的C币 登录后查看C币余额
- 我的收藏
- 我的下载
- 下载帮助
安全验证
文档复制为VIP权益,开通VIP直接复制
信息提交成功