Matrix
======
Welcome to the real world.
This is a test engine designed to validate proper function of real-world
software solutions under a variety of adverse conditions. While this system can
run in a way very similar to bundletester this engine is designed to a
different model. The idea here is to bring up a running deployment, set of a
pattern of application level tests and ensure that the system functions after
operations modelled with Juju are performed. In addition the system supports
large scale failure injection such a removal of units or machines while tests
are executing. It is because of this async nature that the engine is written on
a fresh codebase.
Every effort should be made by the engine to correlate mutations to failure
states and produce helpful logs.
![Interactive Mode](matrix.png)
Extending Matrix from within a Bundle
-------------------------------------
The default suite of tests is designed to provide a general assertion
of functionality for a given bundle based on the common operations and
promises that Juju provides. This ensures that, even without any work
on the part of the charm or bundle author, we can make some assurances
of the quality of the bundle and its charms. However, each bundle will
have its own specific functionality that cannot be tested generically.
Bundles can extend the testing that Matrix does in several ways. The
goal is to allow the bundle author to focus on the things that are
unique to their bundle, while Matrix handles the things that are common
to testing all bundles.
### End-to-end load
The best way for a bundle to verify the functionality unique to that
bundle is to provide an end-to-end load generator that verifies that
the stack as a whole is functioning as expected. This can be done in
two ways:
* `tests/end_to_end.py` This file should contain an `async` function
called `end_to_end` that will be called with two arguments: a
[Juju model instance](https://pypi.python.org/pypi/juju/), and a
standard logger instance where it can emit messages.
* `tests/end_to_end` This file should be executable, and will be
invoked with the name of the model being tested. The output
will be logged, with stderr being logged as errors.
In either case, the load generator will be called after the model has
been deployed and has settled out. It should run indefinitely,
continually generating a reasonable amount of load sufficient to assure
that the system as a whole is functioning properly. If the function
returns or the executable terminates, it will be considered a test
failure. Otherwise, it will be terminated automatically once the
rest of the built-in tests have finished.
### Custom Suite
A bundle can also provide a custom Matrix suite in `tests/matrix.yaml`.
This is a YAML file using the format described below. It can use any
of the built-in Matrix tasks, and it can provide custom tasks as well.
The bundle directory will be included on the Python path, so, for example,
the bundle could provide a `tests/matrix.py` file with custom tasks
and the YAML could refer to them via `tests.matrix.task_name`.
Running Matrix
--------------
Install and run Matrix as a juju plugin by doing the following:
git clone https://github.com/juju-solutions/matrix.git
cd matrix
sudo pip3 install . -f wheelhouse --no-index
juju matrix -p /path/to/bundle
This will run Matrix in interactive mode, with a terminal UI that shows
the progress of each test, their results, the status of the Juju model,
and the Juju debug log. If you prefer to use the non-interactive mode,
invoke Matrix with the `raw` screen option:
juju matrix -p /path/to/bundle -s raw
By default, Matrix runs its built-in suite of tests, along with a `matrix.yaml`
test case if found in the bundle. You can also pass in additional Matrix
tests via the command line:
juju matrix -p /path/to/bundle /path/to/other/test.yaml
See `juju matrix --help` for more information and invocation options.
### Running against bundles from the store
By itself, Matrix can only be run against local copies of bundles. To run
against a bundle in the store, you can use bundletester:
sudo pip2 install bundletester
sudo pip3 install 'git+https://github.com/juju-solutions/matrix.git'
bundletester -t cs:bundle/wiki-simple
In addition to running the bundle and charm tests, bundletester will
run Matrix on the bundle. Note that it will not run it in interactive mode,
so you will only see the end result. The `matrix.log` and `chaos_plan.yaml`
files will be available, however.
### Running with the virtualenv
If you're developing on Matrix, or don't want to install it on your base
system, you can use Tox to run Matrix's unit tests and build a virtualenv
from which you can run Matrix:
git clone https://github.com/juju-solutions/matrix.git
cd matrix/
tox -r
. .tox/py35/bin/activate
juju matrix -p /path/to/bundle
Note that if any of the requirements change, you will need to rebuild the
virtualenv:
deactivate
tox -r
. .tox/py35/bin/activate
### Functional testing
Matrix also includes a full-stack test, which requires you to pass in a
controller name to run:
tox -- -s --controller=lxd
Note that this takes some time, as it runs the default Matrix test suite
against a trivial bundle.
If you want to do more thorough functional testing of the various
quirks and corners of matrix, you can run the functional testing suite
via tox like so:
tox -r -e functional -v
### JaaS Support
You can also run juju-matrix against JaaS controllers. The cloud you
intend to deploy to is ambiguous in JaaS if not specified, however, so
you must specific a cloud with -C/--cloud.
To deploy to aws, for example, you would use the following invocation:
juju matrix -p /path/to/bundle -C aws
High level Design
------------------
Tests are run by an async engine driven by a simple rule engine. The reason to
do things in this way is so we can express the high level test plan in terms of
rules and states (similar to reactive and layer-cake).
tests:
- name: Traffic
description: Traffic in the face of Chaos
rules:
- do:
action: deploy
version: current
- do: test_traffic
until: chaos.complete
after: deploy
- do:
action: matrix.tasks.chaos
while: test_traffic
- do:
action: matrix.tasks.health
periodic: 5
until: chaos.complete
Given this YAML test definition fragment the intention here is as follows.
Define a test relative to a bundle. Deploy that bundle, this will set a state
triggering the next rule and invoking a traffic generating test. The traffic
generating test should be run "until" a state is set (chaos.done) and may be
invoked more than once by the engine. While the engine is running the traffic
suite a state (`test_traffic` based on test name) will be set. This allows
triggering of the "while" rule which launches another task (chaos) on the
current deployment. When that task has done what it deems sufficient it can
exit, which will stop the execution of the traffic test.
Rules are evaluated continuously until the test completes and may run in
parallel. Excessive used of parallelism can make failure analysis more
complicated for the user however.
For a system like this to function we must continuously assert the health of
the running bundle. This means there is a implicit task checking agent/workload
health after every state change in the system. State in this case means states
set by rules and transitions between rules. As Juju grows a real health system
we'd naturally extend to depend on that.
Tasks
-----
The system includes a number of built in tasks that are resolved from any do
clause if no matching file is found in the tests directory. Currently these
tasks are
matrix.tasks.deploy:
version: *current* | prev
ma
没有合适的资源?快使用搜索试试~ 我知道了~
在各种故障条件下自动测试大型软件部署_Python_.zip
共116个文件
py:45个
whl:37个
yaml:9个
1.该资源内容由用户上传,如若侵权请联系客服进行举报
2.虚拟产品一经售出概不退款(资源遇到问题,请及时私信上传者)
2.虚拟产品一经售出概不退款(资源遇到问题,请及时私信上传者)
版权申诉
0 下载量 26 浏览量
2023-04-13
23:40:22
上传
评论
收藏 5.03MB ZIP 举报
温馨提示
在各种故障条件下自动测试大型软件部署_Python_.zip
资源推荐
资源详情
资源评论
收起资源包目录
在各种故障条件下自动测试大型软件部署_Python_.zip (116个子文件)
bad_bundle_file 0B
copyright 680B
.coveragerc 32B
.gitignore 115B
health 145B
MANIFEST.in 61B
tox.ini 894B
test_2.matrix 894B
test_broken_connection.matrix 485B
chaos.matrix 405B
crashdump_test.matrix 283B
test_deploy.matrix 272B
test_uncaught_exception.matrix 259B
test_ha_gating.matrix 239B
test_non_gating.matrix 237B
test_gating.matrix 197B
README.md 10KB
matrix.png 286KB
rules.py 20KB
view.py 18KB
model.py 11KB
main.py 9KB
utils.py 8KB
bus.py 6KB
main.py 6KB
actions.py 4KB
test_main.py 4KB
plan.py 3KB
selectors.py 3KB
deploy.py 3KB
test_e2e.py 2KB
health.py 2KB
end_to_end.py 2KB
config.py 2KB
test_rules.py 2KB
run_action.py 2KB
chaos.py 2KB
gating_test.py 2KB
harness.py 1KB
spell_test.py 1KB
test_utils.py 1KB
setup.py 1001B
test_view.py 953B
test_config.py 913B
test_selector.py 909B
test_deploy.py 823B
env_test.py 761B
fail.py 660B
connection_test.py 604B
health2.py 581B
test_actions.py 494B
test_full_stack.py 479B
basic_test.py 444B
traffic.py 310B
__init__.py 253B
reset.py 178B
conftest.py 172B
tags.py 137B
test_plan.py 111B
__init__.py 32B
__init__.py 0B
__init__.py 0B
__init__.py 0B
changelog.rst 1KB
somefile 0B
test_chaos 143B
test_prog 883B
requirements.txt 147B
test-requirements.txt 44B
Pygments-2.1.3-py2.py3-none-any.whl 737KB
ipython-5.1.0-py3-none-any.whl 730KB
requests-2.12.1-py2.py3-none-any.whl 561KB
requests-2.11.1-py2.py3-none-any.whl 503KB
setuptools-28.6.0-py2.py3-none-any.whl 461KB
prompt_toolkit-1.0.8-py3-none-any.whl 235KB
urwid-1.3.1-cp35-cp35m-linux_x86_64.whl 224KB
python_dateutil-2.6.0-py2.py3-none-any.whl 190KB
coverage-4.2-cp35-cp35m-linux_x86_64.whl 179KB
pytest-3.0.5-py2.py3-none-any.whl 167KB
juju-0.4.1-py3-none-any.whl 163KB
pbr-1.10.0-py2.py3-none-any.whl 95KB
py-1.4.31-py2.py3-none-any.whl 80KB
traitlets-4.3.1-py2.py3-none-any.whl 73KB
wrapt-1.10.8-cp35-cp35m-linux_x86_64.whl 59KB
mock-2.0.0-py2.py3-none-any.whl 55KB
pexpect-4.2.1-py2.py3-none-any.whl 54KB
PyYAML-3.11-cp35-cp35m-linux_x86_64.whl 42KB
websockets-3.2-py33.py34.py35-none-any.whl 38KB
jujubundlelib-0.5.2-py2.py3-none-any.whl 31KB
ipython_genutils-0.1.0-py2.py3-none-any.whl 26KB
blessings-1.6-cp35-none-any.whl 26KB
blessings-1.6-py3-none-any.whl 26KB
wcwidth-0.1.7-py2.py3-none-any.whl 21KB
pytest_cov-2.4.0-py2.py3-none-any.whl 20KB
enforce-0.3.1-py3-none-any.whl 19KB
ubuntui-0.1.1-py3-none-any.whl 18KB
theblues-0.3.8-py3-none-any.whl 17KB
attrs-16.0.0-py2.py3-none-any.whl 16KB
ptyprocess-0.5.1-py2.py3-none-any.whl 13KB
ipdb-0.10.1-cp35-none-any.whl 11KB
共 116 条
- 1
- 2
资源评论
快撑死的鱼
- 粉丝: 1w+
- 资源: 9154
上传资源 快速赚钱
- 我的内容管理 展开
- 我的资源 快来上传第一个资源
- 我的收益 登录查看自己的收益
- 我的积分 登录查看自己的积分
- 我的C币 登录后查看C币余额
- 我的收藏
- 我的下载
- 下载帮助
安全验证
文档复制为VIP权益,开通VIP直接复制
信息提交成功