Honor of Kings Arena: an Environment for
Generalization in Competitive Reinforcement
Learning
Hua Wei
∗†[
, Jingxiao Chen
∗‡[
, Xiyang Ji
∗§
, Hongyang Qin
§
, Minwen Deng
§
, Siqin Li
§
,
Liang Wang
§
, Weinan Zhang
‡
, Yong Yu
‡
, Lin Liu
\
, Lanxiao Huang
\
,
Deheng Ye
§B
, Qiang Fu
§
, Wei Yang
§
§
Tencent AI Lab,
\
Tencent Timi Studio,
†
New Jersey Institute of Technology,
‡
Shanghai Jiao Tong University
hua.wei@njit.edu, timemachine@sjtu.edu.cn, wnzhang@sjtu.edu.cn, yyu@apex.sjtu.edu.cn,
{xiyangji, hongyangqin, danierdeng, gracesqli, enginewang, lincliu, jackiehuang,
dericye, leonfu, willyang}@tencent.com
Abstract
This paper introduces Honor of Kings Arena, a reinforcement learning (RL) envi-
ronment based on Honor of Kings, one of the world’s most popular games at present.
Compared to other environments studied in most previous work, ours presents new
generalization challenges for competitive reinforcement learning. It is a multi-
agent problem with one agent competing against its opponent; and it requires the
generalization ability as it has diverse targets to control and diverse opponents to
compete with. We describe the observation, action, and reward specifications for
the Honor of Kings domain and provide an open-source Python-based interface
for communicating with the game engine. We provide twenty target heroes with
a variety of tasks in Honor of Kings Arena and present initial baseline results for
RL-based methods with feasible computing resources. Finally, we showcase the
generalization challenges imposed by Honor of Kings Arena and possible reme-
dies to the challenges. All of the software, including the environment-class, are
publicly available at:
https://github.com/tencent-ailab/hok_env
. The
documentation is available at: https://aiarena.tencent.com/hok/doc/.
1 Introduction
Games have been used as testbeds to measure AI capabilities in the past few decades, from backgam-
mon [
22
] to chess [
19
] and Atari games [
14
]. In 2016, AlphaGo defeated the world champion through
deep reinforcement learning and Monte Carlo tree search [
19
]. In recent years, reinforcement learning
models have brought huge advancements in robot control [
8
], autonomous driving [
16
], and video
games like StarCraft [23], Dota [1], Minecraft [7] and Honor of Kings [26, 28, 29].
Related to previous AI milestones, the research focus of game AI has shifted from board games
to more complex games, such as imperfect information poker games [
2
] and real-time strategic
games [
28
]. As a sub-genre of real-time strategic games, Multi-player Online Battle Arena (MOBA)
games have attracted much attention recently [
1
,
29
]. The unique playing mechanics of MOBA
involve role/hero play and multi-player. Especially since MOBA games have different roles/heroes
and each role has different actions, a good AI model needs to perform stably well in controlling the
*
Authors contributed equally;
[
work done at Tencent
B
Corresponding author
36th Conference on Neural Information Processing Systems (NeurIPS 2022) Track on Datasets and Benchmarks.
arXiv:2209.08483v3 [cs.LG] 18 Oct 2022
剩余19页未读，继续阅读
资源评论
