HBASE-24620: All new ClusterManager which
submits destructive actions through ZK and it's
agent which picks up those commands and
executes them blindly.
This is the design doc for
HBASE-24620
i.e. a new clustermanager which does not require
passwordless-ssh to execute destructive actions used in Chaos Monkey.
Context:
Current Chaos Monkey has basically one ClusterManager which is default HBaseClusterManager
which uses passwordless-ssh to execute destructive actions, and that sometimes is a blocker if
someone wants to integrate chaos as a tool with their service which uses chaos.
Apart from HBaseClusterManager there are two other ClusterManagers
● RestApiClusterManager : A ClusterManager which controls Cloudera Manager clusters via
RestApis.
● CoprocClusterManager : A recently added Coproc based ClusterManager which only takes
actions against Master and RegionServer processes and those coprocs are itself running
within those processes.
Goal:
Our Goal is to add a ClusterManager which can execute destructive actions without
passwordless-ssh.
Proposed Design:
Higher Level :
On a Higher Level this new ClusterManager will be using an agent-based architecture, where
instead of directly executing commands, it’ll be submitting tasks to the ZooKeeper as a ZNode
and an Agent will be running on every node which will pick up that submitted task and executes
it blindly.
Here we’ll be introducing a new small component chaos agent whose function is simply to
execute commands which have been assigned to them.
