# RCSB DB
## A collection of Python Database Utility Classes
[![Build Status](https://dev.azure.com/rcsb/RCSB%20PDB%20Python%20Projects/_apis/build/status/rcsb.py-rcsb_db?branchName=master)](https://dev.azure.com/rcsb/RCSB%20PDB%20Python%20Projects/_build/latest?definitionId=12&branchName=master)
## Introduction
This module contains a collection of utility classes for processing and loading PDB repository and
derived data content using relational and document database servers. One target data store for
these tools is a document database used to exchange content within the RCSB PDB data pipeline.
### Installation
Download the library source software from the project repository:
```bash
git clone --recurse-submodules https://github.com/rcsb/py-rcsb_db.git
```
Optionally, run test suite (Python versions 2.7, 3.6, and 3.7) using
[setuptools](https://setuptools.readthedocs.io/en/latest/) or
[tox](http://tox.readthedocs.io/en/latest/example/platform.html):
```bash
python setup.py test
or simply run
tox
```
Installation is via the program [pip](https://pypi.python.org/pypi/pip). To run tests
from the source tree, the package must be installed in editable mode (i.e. -e):
```bash
pip install -e .
```
#### Installing in Ubuntu Linux (tested in 18.04)
You will need a few packages, before `pip install .` can work:
```bash
sudo apt install default-libmysqlclient-dev flex bison
```
### Installing on macOS
To use and develop this package on macOS requires a number of packages that are not
distributed as part of the base macOS operating system.
The following steps provide one approach to creating the development environment for this
package. First, install the Apple [XCode](https://developer.apple.com/xcode/) package and associate command-line tools.
This will provide essential compilers and supporting tools. The [HomeBrew](https://brew.sh/) package
manager provides further access to a variety of common open source services and tools.
Follow the instructions provided by at the [HomeBrew](https://brew.sh/) site to
install this system. Once HomeBrew is installed, you can further install the
[MariaDB](https://mariadb.com/kb/en/library/installing-mariadb-on-macos-using-homebrew/) and
[MongoDB](https://docs.mongodb.com/manual/tutorial/install-mongodb-on-os-x/) packages which
are required to support the ExDB tools. HomeBrew also provides a variety of options for
managing a [Python virtual environments](https://gist.github.com/Geoyi/f55ed54d24cc9ff1c14bd95fac21c042).
### Command Line Interfaces
A convenience CLI `schema_update_cli` is provided for generating operational schema from
PDBx/mmCIF dictionary metadata. Schema are encoded for the ExDB API (rcsb), and
for the document schema encoded in JSON and BSON formats. The latter schema can be used to
validate the loadable document objects produced for the collections served by MongoDB.
```bash
=> schema_update_cli --help
usage: schema_update_cli [-h] [--update_chem_comp_ref]
[--update_chem_comp_core_ref]
[--update_bird_chem_comp_ref]
[--update_bird_chem_comp_core_ref]
[--update_bird_ref] [--update_bird_family_ref]
[--update_pdbx] [--update_pdbx_core]
[--update_repository_holdings]
[--update_entity_sequence_clusters]
[--update_data_exchange] [--update_ihm_dev]
[--update_drugbank_core] [--update_config_all]
[--update_config_deployed] [--update_config_test]
[--config_path CONFIG_PATH]
[--config_name CONFIG_NAME]
[--cache_path SCHEMA_CACHE_PATH]
[--schema_types SCHEMA_TYPES]
[--schema_levels SCHEMA_LEVELS] [--debug] [--mock]
optional arguments:
-h, --help show this help message and exit
--update_chem_comp_ref
Update schema for Chemical Component reference
definitions
--update_chem_comp_core_ref
Update core schema for Chemical Component reference
definitions
--update_bird_chem_comp_ref
Update schema for Bird Chemical Component reference
definitions
--update_bird_chem_comp_core_ref
Update core schema for Bird Chemical Component
reference definitions
--update_bird_ref Update schema for Bird reference definitions
--update_bird_family_ref
Update schema for Bird Family reference definitions
--update_pdbx Update schema for PDBx entry data
--update_pdbx_core Update schema for PDBx core entry/entity data
--update_repository_holdings
Update schema for repository holdings
--update_entity_sequence_clusters
Update schema for entity sequence clusters
--update_data_exchange
Update schema for data exchange status
--update_ihm_dev Update schema for I/HM dev entry data
--update_drugbank_core
Update DrugBank schema
--update_config_all Update using configuration settings (e.g.
DATABASE_NAMES_ALL)
--update_config_deployed
Update using configuration settings (e.g.
DATABASE_NAMES_DEPLOYED)
--update_config_test Update using configuration settings (e.g.
DATABASE_NAMES_TEST)
--config_path CONFIG_PATH
Path to configuration options file
--config_name CONFIG_NAME
Configuration section name
--cache_path CACHE_PATH
Schema cache directory path
--schema_types SCHEMA_TYPES
Schema encoding (rcsb|json|bson) (comma separated)
--schema_levels SCHEMA_LEVELS
Schema validation level (full|min) (comma separated)
--debug Turn on verbose logging
--mock Use MOCK repository configuration for dependencies and
testing
________________________________________________________________________________
```
For example, the following command will generate the JSON and BSON schema for the collections in the
pdbx_core schema.
```bash
schema_update_cli --mock --schema_types json,bson \
--schema_level full \
--update_pdbx_core \
--cache_path . \
--config_path ./rcsb/db/config/exdb-config-example.yml \
--config_name site_info_configuration
```
A convenience CLI `exdb_repo_load_cli` is provided to support loading PDB repositories
containing entry and chemical reference data content types in the form of document collections
compatible with MongoDB.
```bash
exdb_repo_load_cli --help
usage: exdb_repo_load_cli [-h] [--full] [--replace] [--load_chem_comp_ref]
[--load_chem_comp_core_ref]
[--load_bird_chem_comp_ref]
[--load_bird_chem_comp_core_ref] [--load_bird_ref]
[--load_bird_family_ref] [--load_entry_data]
[--load_pdbx_core] [--load_pdbx_core_merge]
[--load_pdbx_core_entry] [--load_pdbx_core_entity]
[--load_pdbx_core_entity_monomer]
[--load_pdbx_core_assembly] [--load_ihm_dev]
[--config_path CONFIG_PATH]
[--config_name CONFIG_NAME] [--db_type DB_TYPE]
[--document_style DOCUMENT_STYLE]
[--read_back_check] [--schema_level SCHEMA_LEVEL]
[--load_file_list_path LOAD_FILE_LIST_PATH]
没有合适的资源?快使用搜索试试~ 我知道了~
资源详情
资源评论
资源推荐
收起资源包目录
rcsb.db-1.666.tar.gz (75个子文件)
rcsb.db-1.666
MANIFEST.in 57B
PKG-INFO 713B
rcsb
db
define
ContentDefinition.py 31KB
DataTypeApplicationInfo.py 15KB
SchemaDefAccess.py 32KB
DataTypeInstanceInfo.py 4KB
SchemaDefBuild.py 57KB
DataTypeApiProvider.py 4KB
__init__.py 0B
processors
ClusterDataPrep.py 7KB
DataTransformFactory.py 19KB
SchemaDefDataPrep.py 37KB
DataExchangeStatus.py 5KB
RepoHoldingsDataPrep.py 30KB
__init__.py 0B
SchemaDefReShape.py 26KB
wf
__init__.py 0B
RepoLoadWorkflow.py 9KB
mysql
Connection.py 3KB
MyDbAdapter.py 18KB
ConnectionBase.py 4KB
MyDbUtil.py 12KB
MysqlSchemaImporter.py 5KB
__init__.py 0B
SchemaDefLoader.py 18KB
mongo
DocumentLoader.py 15KB
MongoDbUtil.py 23KB
Connection.py 6KB
ConnectionBase.py 5KB
PdbxLoader.py 61KB
__init__.py 0B
__init__.py 65B
crate
Connection.py 5KB
CrateDbLoader.py 8KB
__init__.py 0B
CrateDbUtil.py 10KB
sql
QueryDirectives.py 23KB
__init__.py 0B
SqlGen.py 40KB
cockroach
CockroachDbUtil.py 9KB
Connection.py 5KB
__init__.py 0B
CockroachDbLoader.py 10KB
utils
CaseNormalizedDict.py 3KB
TimeUtil.py 2KB
SchemaProvider.py 20KB
__init__.py 0B
ProvenanceProvider.py 4KB
PdbxSchemaMapReader.py 12KB
TextUtil.py 1KB
makePathList.py 2KB
unescape.py 1KB
helpers
DocumentDefinitionHelper.py 33KB
r.py 2KB
__init__.py 0B
ContentDefinitionHelper.py 17KB
cli
RepoScanExec.py 9KB
SequenceClustersEtlWorker.py 8KB
RepoLoadExec.py 20KB
SchemaUpdateExec.py 11KB
ETLExec.py 9KB
__init__.py 155B
RepoHoldingsEtlWorker.py 6KB
__init__.py 65B
rcsb.db.egg-info
PKG-INFO 713B
requires.txt 415B
not-zip-safe 1B
SOURCES.txt 2KB
entry_points.txt 210B
top_level.txt 5B
dependency_links.txt 1B
HISTORY.txt 25KB
setup.cfg 108B
setup.py 3KB
README.md 26KB
共 75 条
- 1
挣扎的蓝藻
- 粉丝: 13w+
- 资源: 15万+
上传资源 快速赚钱
- 我的内容管理 展开
- 我的资源 快来上传第一个资源
- 我的收益 登录查看自己的收益
- 我的积分 登录查看自己的积分
- 我的C币 登录后查看C币余额
- 我的收藏
- 我的下载
- 下载帮助
最新资源
- 目标检测-零售食品LOGO检测数据集-5000张图-+对应VOC-COCO-YOLO三种格式标签+数据集划分脚本
- 目标检测-零售食品LOGO检测数据集-1000张图-+对应VOC-COCO-YOLO三种格式标签+数据集划分脚本
- Delphi 12 控件之LMD.VCL.Full.Version.zip
- 计算机科学选修课:人工智能导论 第二节 PPT
- 计算机科学选修课:人工智能导论 第一章 PPT
- 忘记navicat密码时,此工具可以帮您查看密码
- 电子商务-电子商务平台-水果销售-在线购物-果库小程序.zip
- 基于React的后台开发框架(javascript)
- 用户界面设计-滑动选项卡-交互组件-用户体验-滑动选项卡小程序.zip
- 371系列智能型电动执行机构说明书
资源上传下载、课程学习等过程中有任何疑问或建议,欢迎提出宝贵意见哦~我们会及时处理!
点击此处反馈
安全验证
文档复制为VIP权益,开通VIP直接复制
信息提交成功
评论0