# RCSB DB
## A collection of Python Database Utility Classes
[![Build Status](https://dev.azure.com/rcsb/RCSB%20PDB%20Python%20Projects/_apis/build/status/rcsb.py-rcsb_db?branchName=master)](https://dev.azure.com/rcsb/RCSB%20PDB%20Python%20Projects/_build/latest?definitionId=12&branchName=master)
## Introduction
This module contains a collection of utility classes for processing and loading PDB repository and
derived data content using relational and document database servers. One target data store for
these tools is a document database used to exchange content within the RCSB PDB data pipeline.
### Installation
Download the library source software from the project repository:
```bash
git clone --recurse-submodules https://github.com/rcsb/py-rcsb_db.git
```
Optionally, run test suite (Python versions 2.7, 3.6, and 3.7) using
[setuptools](https://setuptools.readthedocs.io/en/latest/) or
[tox](http://tox.readthedocs.io/en/latest/example/platform.html):
```bash
python setup.py test
or simply run
tox
```
Installation is via the program [pip](https://pypi.python.org/pypi/pip). To run tests
from the source tree, the package must be installed in editable mode (i.e. -e):
```bash
pip install -e .
```
#### Installing in Ubuntu Linux (tested in 18.04)
You will need a few packages, before `pip install .` can work:
```bash
sudo apt install default-libmysqlclient-dev flex bison
```
### Installing on macOS
To use and develop this package on macOS requires a number of packages that are not
distributed as part of the base macOS operating system.
The following steps provide one approach to creating the development environment for this
package. First, install the Apple [XCode](https://developer.apple.com/xcode/) package and associate command-line tools.
This will provide essential compilers and supporting tools. The [HomeBrew](https://brew.sh/) package
manager provides further access to a variety of common open source services and tools.
Follow the instructions provided by at the [HomeBrew](https://brew.sh/) site to
install this system. Once HomeBrew is installed, you can further install the
[MariaDB](https://mariadb.com/kb/en/library/installing-mariadb-on-macos-using-homebrew/) and
[MongoDB](https://docs.mongodb.com/manual/tutorial/install-mongodb-on-os-x/) packages which
are required to support the ExDB tools. HomeBrew also provides a variety of options for
managing a [Python virtual environments](https://gist.github.com/Geoyi/f55ed54d24cc9ff1c14bd95fac21c042).
### Command Line Interfaces
A convenience CLI `schema_update_cli` is provided for generating operational schema from
PDBx/mmCIF dictionary metadata. Schema are encoded for the ExDB API (rcsb), and
for the document schema encoded in JSON and BSON formats. The latter schema can be used to
validate the loadable document objects produced for the collections served by MongoDB.
```bash
=> schema_update_cli --help
usage: schema_update_cli [-h] [--update_chem_comp_ref]
[--update_chem_comp_core_ref]
[--update_bird_chem_comp_ref]
[--update_bird_chem_comp_core_ref]
[--update_bird_ref] [--update_bird_family_ref]
[--update_pdbx] [--update_pdbx_core]
[--update_repository_holdings]
[--update_entity_sequence_clusters]
[--update_data_exchange] [--update_ihm_dev]
[--update_drugbank_core] [--update_config_all]
[--update_config_deployed] [--update_config_test]
[--config_path CONFIG_PATH]
[--config_name CONFIG_NAME]
[--cache_path SCHEMA_CACHE_PATH]
[--schema_types SCHEMA_TYPES]
[--schema_levels SCHEMA_LEVELS] [--debug] [--mock]
optional arguments:
-h, --help show this help message and exit
--update_chem_comp_ref
Update schema for Chemical Component reference
definitions
--update_chem_comp_core_ref
Update core schema for Chemical Component reference
definitions
--update_bird_chem_comp_ref
Update schema for Bird Chemical Component reference
definitions
--update_bird_chem_comp_core_ref
Update core schema for Bird Chemical Component
reference definitions
--update_bird_ref Update schema for Bird reference definitions
--update_bird_family_ref
Update schema for Bird Family reference definitions
--update_pdbx Update schema for PDBx entry data
--update_pdbx_core Update schema for PDBx core entry/entity data
--update_repository_holdings
Update schema for repository holdings
--update_entity_sequence_clusters
Update schema for entity sequence clusters
--update_data_exchange
Update schema for data exchange status
--update_ihm_dev Update schema for I/HM dev entry data
--update_drugbank_core
Update DrugBank schema
--update_config_all Update using configuration settings (e.g.
DATABASE_NAMES_ALL)
--update_config_deployed
Update using configuration settings (e.g.
DATABASE_NAMES_DEPLOYED)
--update_config_test Update using configuration settings (e.g.
DATABASE_NAMES_TEST)
--config_path CONFIG_PATH
Path to configuration options file
--config_name CONFIG_NAME
Configuration section name
--cache_path CACHE_PATH
Schema cache directory path
--schema_types SCHEMA_TYPES
Schema encoding (rcsb|json|bson) (comma separated)
--schema_levels SCHEMA_LEVELS
Schema validation level (full|min) (comma separated)
--debug Turn on verbose logging
--mock Use MOCK repository configuration for dependencies and
testing
________________________________________________________________________________
```
For example, the following command will generate the JSON and BSON schema for the collections in the
pdbx_core schema.
```bash
schema_update_cli --mock --schema_types json,bson \
--schema_level full \
--update_pdbx_core \
--cache_path . \
--config_path ./rcsb/db/config/exdb-config-example.yml \
--config_name site_info_configuration
```
A convenience CLI `exdb_repo_load_cli` is provided to support loading PDB repositories
containing entry and chemical reference data content types in the form of document collections
compatible with MongoDB.
```bash
exdb_repo_load_cli --help
usage: exdb_repo_load_cli [-h] [--full] [--replace] [--load_chem_comp_ref]
[--load_chem_comp_core_ref]
[--load_bird_chem_comp_ref]
[--load_bird_chem_comp_core_ref] [--load_bird_ref]
[--load_bird_family_ref] [--load_entry_data]
[--load_pdbx_core] [--load_pdbx_core_merge]
[--load_pdbx_core_entry] [--load_pdbx_core_entity]
[--load_pdbx_core_entity_monomer]
[--load_pdbx_core_assembly] [--load_ihm_dev]
[--config_path CONFIG_PATH]
[--config_name CONFIG_NAME] [--db_type DB_TYPE]
[--document_style DOCUMENT_STYLE]
[--read_back_check] [--schema_level SCHEMA_LEVEL]
[--load_file_list_path LOAD_FILE_LIST_PATH]
没有合适的资源?快使用搜索试试~ 我知道了~
资源推荐
资源详情
资源评论
收起资源包目录
rcsb.db-1.683.tar.gz (77个子文件)
rcsb.db-1.683
MANIFEST.in 100B
PKG-INFO 27KB
rcsb
db
define
ContentDefinition.py 32KB
DataTypeApplicationInfo.py 15KB
SchemaDefAccess.py 34KB
DataTypeInstanceInfo.py 4KB
SchemaDefBuild.py 57KB
DataTypeApiProvider.py 4KB
__init__.py 0B
processors
ClusterDataPrep.py 7KB
DataTransformFactory.py 19KB
SchemaDefDataPrep.py 39KB
DataExchangeStatus.py 5KB
RepoHoldingsDataPrep.py 31KB
__init__.py 0B
SchemaDefReShape.py 26KB
wf
__init__.py 0B
RepoLoadWorkflow.py 9KB
mysql
Connection.py 3KB
MyDbAdapter.py 18KB
ConnectionBase.py 4KB
MyDbUtil.py 12KB
MysqlSchemaImporter.py 5KB
__init__.py 0B
SchemaDefLoader.py 19KB
mongo
DocumentLoader.py 15KB
MongoDbUtil.py 23KB
Connection.py 6KB
ConnectionBase.py 5KB
PdbxLoader.py 62KB
__init__.py 0B
__init__.py 65B
crate
Connection.py 5KB
CrateDbLoader.py 8KB
__init__.py 0B
CrateDbUtil.py 10KB
sql
QueryDirectives.py 23KB
__init__.py 0B
SqlGen.py 40KB
cockroach
CockroachDbUtil.py 9KB
Connection.py 5KB
__init__.py 0B
CockroachDbLoader.py 10KB
utils
CaseNormalizedDict.py 3KB
TimeUtil.py 2KB
SchemaProvider.py 20KB
__init__.py 0B
ProvenanceProvider.py 4KB
PdbxSchemaMapReader.py 12KB
TextUtil.py 1KB
makePathList.py 2KB
unescape.py 1KB
helpers
DocumentDefinitionHelper.py 33KB
r.py 2KB
__init__.py 0B
ContentDefinitionHelper.py 17KB
cli
RepoScanExec.py 9KB
SequenceClustersEtlWorker.py 8KB
RepoLoadExec.py 20KB
SchemaUpdateExec.py 11KB
ETLExec.py 9KB
__init__.py 155B
RepoHoldingsEtlWorker.py 6KB
__init__.py 65B
LICENSE 552B
rcsb.db.egg-info
PKG-INFO 27KB
requires.txt 428B
not-zip-safe 1B
SOURCES.txt 2KB
entry_points.txt 210B
top_level.txt 5B
dependency_links.txt 1B
HISTORY.txt 26KB
setup.cfg 108B
requirements.txt 459B
setup.py 3KB
README.md 26KB
共 77 条
- 1
资源评论
挣扎的蓝藻
- 粉丝: 13w+
- 资源: 15万+
上传资源 快速赚钱
- 我的内容管理 展开
- 我的资源 快来上传第一个资源
- 我的收益 登录查看自己的收益
- 我的积分 登录查看自己的积分
- 我的C币 登录后查看C币余额
- 我的收藏
- 我的下载
- 下载帮助
安全验证
文档复制为VIP权益,开通VIP直接复制
信息提交成功