RLCard是一款卡牌游戏强化学习 (Reinforcement Learning, RL) 的工具包

# RLCard: A Toolkit for Reinforcement Learning in Card Games <img width="500" src="https://dczha.com/files/rlcard/logo.jpg" alt="Logo" /> [](https://github.com/datamllab/rlcard/actions/workflows/python-package.yml) [](https://badge.fury.io/py/rlcard) [](https://coveralls.io/github/datamllab/rlcard?branch=master) [](https://pepy.tech/project/rlcard) [](https://pepy.tech/project/rlcard) [](https://opensource.org/licenses/MIT) [中文文档](README.zh-CN.md) RLCard is a toolkit for Reinforcement Learning (RL) in card games. It supports multiple card environments with easy-to-use interfaces for implementing various reinforcement learning and searching algorithms. The goal of RLCard is to bridge reinforcement learning and imperfect information games. RLCard is developed by [DATA Lab](http://faculty.cs.tamu.edu/xiahu/) at Rice and Texas A&M University, and community contributors. * Official Website: [https://www.rlcard.org](https://www.rlcard.org) * Tutorial in Jupyter Notebook: [https://github.com/datamllab/rlcard-tutorial](https://github.com/datamllab/rlcard-tutorial) * Paper: [https://arxiv.org/abs/1910.04376](https://arxiv.org/abs/1910.04376) * GUI: [RLCard-Showdown](https://github.com/datamllab/rlcard-showdown) * Dou Dizhu Demo: [Demo](https://douzero.org/) * Resources: [Awesome-Game-AI](https://github.com/datamllab/awesome-game-ai) * Related Project: [DouZero Project](https://github.com/kwai/DouZero) * Zhihu: https://zhuanlan.zhihu.com/p/526723604 **Community:** * **Slack**: Discuss in our [#rlcard-project](https://join.slack.com/t/rlcard/shared_invite/zt-rkvktsaq-xkMwz8BfKupCM6zGhO01xg) slack channel. * **QQ Group**: Join our QQ group to discuss. Password: rlcardqqgroup * Group 1: 665647450 * Group 2: 117349516 **News:** * We have updated the tutorials in Jupyter Notebook to help you walk through RLCard! Please check [RLCard Tutorial](https://github.com/datamllab/rlcard-tutorial). * All the algorithms can suppport [PettingZoo](https://github.com/PettingZoo-Team/PettingZoo) now. Please check [here](examples/pettingzoo). Thanks the contribtuion from [Yifei Cheng](https://github.com/ycheng517). * Please follow [DouZero](https://github.com/kwai/DouZero), a strong Dou Dizhu AI and the [ICML 2021 paper](https://arxiv.org/abs/2106.06135). An online demo is available [here](https://douzero.org/). The algorithm is also integrated in RLCard. See [Training DMC on Dou Dizhu](docs/toy-examples.md#training-dmc-on-dou-dizhu). * Our package is used in [PettingZoo](https://github.com/PettingZoo-Team/PettingZoo). Please check it out! * We have released RLCard-Showdown, GUI demo for RLCard. Please check out [here](https://github.com/datamllab/rlcard-showdown)! * Jupyter Notebook tutorial available! We add some examples in R to call Python interfaces of RLCard with reticulate. See [here](docs/toy-examples-r.md) * Thanks for the contribution of [@Clarit7](https://github.com/Clarit7) for supporting different number of players in Blackjack. We call for contributions for gradually making the games more configurable. See [here](CONTRIBUTING.md#making-configurable-environments) for more details. * Thanks for the contribution of [@Clarit7](https://github.com/Clarit7) for the Blackjack and Limit Hold'em human interface. * Now RLCard supports environment local seeding and multiprocessing. Thanks for the testing scripts provided by [@weepingwillowben](https://github.com/weepingwillowben). * Human interface of NoLimit Holdem available. The action space of NoLimit Holdem has been abstracted. Thanks for the contribution of [@AdrianP-](https://github.com/AdrianP-). * New game Gin Rummy and human GUI available. Thanks for the contribution of [@billh0420](https://github.com/billh0420). * PyTorch implementation available. Thanks for the contribution of [@mjudell](https://github.com/mjudell). ## Cite this work If you find this repo useful, you may cite: Zha, Daochen, et al. "RLCard: A Platform for Reinforcement Learning in Card Games." IJCAI. 2020. ```bibtex @inproceedings{zha2020rlcard, title={RLCard: A Platform for Reinforcement Learning in Card Games}, author={Zha, Daochen and Lai, Kwei-Herng and Huang, Songyi and Cao, Yuanpu and Reddy, Keerthana and Vargas, Juan and Nguyen, Alex and Wei, Ruzhe and Guo, Junyu and Hu, Xia}, booktitle={IJCAI}, year={2020} } ``` ## Installation Make sure that you have **Python 3.6+** and **pip** installed. We recommend installing the stable version of `rlcard` with `pip`: ``` pip3 install rlcard ``` The default installation will only include the card environments. To use PyTorch implementation of the training algorithms, run ``` pip3 install rlcard[torch] ``` If you are in China and the above command is too slow, you can use the mirror provided by Tsinghua University: ``` pip3 install rlcard -i https://pypi.tuna.tsinghua.edu.cn/simple ``` Alternatively, you can clone the latest version with (if you are in China and Github is slow, you can use the mirror in [Gitee](https://gitee.com/daochenzha/rlcard)): ``` git clone https://github.com/datamllab/rlcard.git ``` or only clone one branch to make it faster: ``` git clone -b master --single-branch --depth=1 https://github.com/datamllab/rlcard.git ``` Then install with ``` cd rlcard pip3 install -e . pip3 install -e .[torch] ``` We also provide [**conda** installation method](https://anaconda.org/toubun/rlcard): ``` conda install -c toubun rlcard ``` Conda installation only provides the card environments, you need to manually install Pytorch on your demands. ## Examples A **short example** is as below. ```python import rlcard from rlcard.agents import RandomAgent env = rlcard.make('blackjack') env.set_agents([RandomAgent(num_actions=env.num_actions)]) print(env.num_actions) # 2 print(env.num_players) # 1 print(env.state_shape) # [[2]] print(env.action_shape) # [None] trajectories, payoffs = env.run() ``` RLCard can be flexibly connected to various algorithms. See the following examples: * [Playing with random agents](docs/toy-examples.md#playing-with-random-agents) * [Deep-Q learning on Blackjack](docs/toy-examples.md#deep-q-learning-on-blackjack) * [Training CFR (chance sampling) on Leduc Hold'em](docs/toy-examples.md#training-cfr-on-leduc-holdem) * [Having fun with pretrained Leduc model](docs/toy-examples.md#having-fun-with-pretrained-leduc-model) * [Training DMC on Dou Dizhu](docs/toy-examples.md#training-dmc-on-dou-dizhu) * [Evaluating Agents](docs/toy-examples.md#evaluating-agents) * [Training Agents on PettingZoo](examples/pettingzoo) ## Demo Run `examples/human/leduc_holdem_human.py` to play with the pre-trained Leduc Hold'em model. Leduc Hold'em is a simplified version of Texas Hold'em. Rules can be found [here](docs/games.md#leduc-holdem). ``` >> Leduc Hold'em pre-trained model >> Start a new game! >> Agent 1 chooses raise =============== Community Card =============== ┌─────────┐ │░░░░░░░░░│ │░░░░░░░░░│ │░░░░░░░░░│ │░░░░░░░░░│ │░░░░░░░░░│ │░░░░░░░░░│ │░░░░░░░░░│ └─────────┘ =============== Your Hand =============== ┌─────────┐ │J │ │ │ │ │ │ ♥ │ │ │ │ │ │ J│ └─────────┘ =============== Chips =============== Yours: + Agent 1: +++ =========== Actions You Can Choose =========== 0: call, 1: raise,