Yahoo! Cloud Serving Benchmark (YCSB) 的 Go 版本.zip

# go-ycsb go-ycsb is a Go port of [YCSB](https://github.com/brianfrankcooper/YCSB). It fully supports all YCSB generators and the Core workload so we can do the basic CRUD benchmarks with Go. ## Why another Go YCSB? + We want to build a standard benchmark tool in Go. + We are not familiar with Java. ## Getting Started ### Download https://github.com/pingcap/go-ycsb/releases/latest **Linux** ``` wget -c https://github.com/pingcap/go-ycsb/releases/latest/download/go-ycsb-linux-amd64.tar.gz -O - | tar -xz # give it a try ./go-ycsb --help ``` **OSX** ``` wget -c https://github.com/pingcap/go-ycsb/releases/latest/download/go-ycsb-darwin-amd64.tar.gz -O - | tar -xz # give it a try ./go-ycsb --help ``` ### Building from source ```bash git clone https://github.com/pingcap/go-ycsb.git cd go-ycsb make # give it a try ./bin/go-ycsb --help ``` Notice: + Minimum supported go version is 1.16. + To use FoundationDB, you must install [client](https://www.foundationdb.org/download/) library at first, now the supported version is 6.2.11. + To use RocksDB, you must follow [INSTALL](https://github.com/facebook/rocksdb/blob/master/INSTALL.md) to install RocksDB at first. ## Usage Mostly, we can start from the official document [Running-a-Workload](https://github.com/brianfrankcooper/YCSB/wiki/Running-a-Workload). ### Shell ```basic ./bin/go-ycsb shell basic » help YCSB shell command Usage: shell [command] Available Commands: delete Delete a record help Help about any command insert Insert a record read Read a record scan Scan starting at key table Get or [set] the name of the table update Update a record ``` ### Load ```bash ./bin/go-ycsb load basic -P workloads/workloada ``` ### Run ```bash ./bin/go-ycsb run basic -P workloads/workloada ``` ## Supported Database - MySQL / TiDB - TiKV - FoundationDB - Aerospike - Badger - Cassandra / ScyllaDB - Pegasus - PostgreSQL / CockroachDB / AlloyDB / Yugabyte - RocksDB - Spanner - Sqlite - MongoDB - Redis and Redis Cluster - BoltDB - etcd - DynamoDB ## Output configuration |field|default value|description| |-|-|-| |measurementtype|"histogram"|The mechanism for recording measurements, one of `histogram`, `raw` or `csv`| |measurement.output_file|""|File to write output to, default writes to stdout| ## Database Configuration You can pass the database configurations through `-p field=value` in the command line directly. Common configurations: |field|default value|description| |-|-|-| |dropdata|false|Whether to remove all data before test| |verbose|false|Output the execution query| |debug.pprof|":6060"|Go debug profile address| ### MySQL & TiDB |field|default value|description| |-|-|-| |mysql.host|"127.0.0.1"|MySQL Host| |mysql.port|3306|MySQL Port| |mysql.user|"root"|MySQL User| |mysql.password||MySQL Password| |mysql.db|"test"|MySQL Database| |tidb.cluster_index|true|Whether to use cluster index, for TiDB only| |tidb.instances|""|Comma-seperated address list of tidb instances (eg: `tidb-0:4000,tidb-1:4000`)| ### TiKV |field|default value|description| |-|-|-| |tikv.pd|"127.0.0.1:2379"|PD endpoints, seperated by comma| |tikv.type|"raw"|TiKV mode, "raw", "txn", or "coprocessor"| |tikv.conncount|128|gRPC connection count| |tikv.batchsize|128|Request batch size| |tikv.async_commit|true|Enalbe async commit or not| |tikv.one_pc|true|Enable one phase or not| |tikv.apiversion|"V1"|[api-version](https://docs.pingcap.com/tidb/stable/tikv-configuration-file#api-version-new-in-v610) of tikv server, "V1" or "V2"| ### FoundationDB |field|default value|description| |-|-|-| |fdb.cluster|""|The cluster file used for FoundationDB, if not set, will use the [default](https://apple.github.io/foundationdb/administration.html#default-cluster-file)| |fdb.dbname|"DB"|The cluster database name| |fdb.apiversion|510|API version, now only 5.1 is supported| ### PostgreSQL & CockroachDB & AlloyDB & Yugabyte |field|default value|description| |-|-|-| |pg.host|"127.0.0.1"|PostgreSQL Host| |pg.port|5432|PostgreSQL Port| |pg.user|"root"|PostgreSQL User| |pg.password||PostgreSQL Password| |pg.db|"test"|PostgreSQL Database| |pg.sslmode|"disable|PostgreSQL ssl mode| ### Aerospike |field|default value|description| |-|-|-| |aerospike.host|"localhost"|The port of the Aerospike service| |aerospike.port|3000|The port of the Aerospike service| |aerospike.ns|"test"|The namespace to use| ### Badger |field|default value|description| |-|-|-| |badger.dir|"/tmp/badger"|The directory to save data| |badger.valuedir|"/tmp/badger"|The directory to save value, if not set, use badger.dir| |badger.sync_writes|false|Sync all writes to disk| |badger.num_versions_to_keep|1|How many versions to keep per key| |badger.max_table_size|64MB|Each table (or file) is at most this size| |badger.level_size_multiplier|10|Equals SizeOf(Li+1)/SizeOf(Li)| |badger.max_levels|7|Maximum number of levels of compaction| |badger.value_threshold|32|If value size >= this threshold, only store value offsets in tree| |badger.num_memtables|5|Maximum number of tables to keep in memory, before stalling| |badger.num_level0_tables|5|Maximum number of Level 0 tables before we start compacting| |badger.num_level0_tables_stall|10|If we hit this number of Level 0 tables, we will stall until L0 is compacted away| |badger.level_one_size|256MB|Maximum total size for L1| |badger.value_log_file_size|1GB|Size of single value log file| |badger.value_log_max_entries|1000000|Max number of entries a value log file can hold (approximately). A value log file would be determined by the smaller of its file size and max entries| |badger.num_compactors|3|Number of compaction workers to run concurrently| |badger.do_not_compact|false|Stops LSM tree from compactions| |badger.table_loading_mode|options.LoadToRAM|How should LSM tree be accessed| |badger.value_log_loading_mode|options.MemoryMap|How should value log be accessed| ### RocksDB |field|default value|description| |-|-|-| |rocksdb.dir|"/tmp/rocksdb"|The directory to save data| |rocksdb.allow_concurrent_memtable_writes|true|Sets whether to allow concurrent memtable writes| |rocksdb.allow_mmap_reads|false|Enable/Disable mmap reads for reading sst tables| |rocksdb.allow_mmap_writes|false|Enable/Disable mmap writes for writing sst tables| |rocksdb.arena_block_size|0(write_buffer_size / 8)|Sets the size of one block in arena memory allocation| |rocksdb.db_write_buffer_size|0(disable)|Sets the amount of data to build up in memtables across all column families before writing to disk| |rocksdb.hard_pending_compaction_bytes_limit|256GB|Sets the bytes threshold at which all writes are stopped if estimated bytes needed to be compaction exceed this threshold| |rocksdb.level0_file_num_compaction_trigger|4|Sets the number of files to trigger level-0 compaction| |rocksdb.level0_slowdown_writes_trigger|20|Sets the soft limit on number of level-0 files| |rocksdb.level0_stop_writes_trigger|36|Sets the maximum number of level-0 files. We stop writes at this point| |rocksdb.max_bytes_for_level_base|256MB|Sets the maximum total data size for base level| |rocksdb.max_bytes_for_level_multiplier|10|Sets the max Bytes for level multiplier| |rocksdb.max_total_wal_size|0(\[sum of all write_buffer_size * max_write_buffer_number\] * 4)|Sets the maximum total wal size in bytes. Once write-ahead logs exceed this size, we will start forcing the flush of column families whose memtables are backed by the oldest live WAL file (i.e. the ones that are causing all the space amplification)| |rocksdb.memtable_huge_page_size|0|Sets the page size for huge page for arena used by the memtable| |rocksdb.num_levels|7|Sets the number of levels for this database| |rocksdb.use_direct_reads|false|Enable/Disable direct I/O mode (O_DIRECT) for reads| |rocksdb.use_fsync|false|Enable/Disable fsync| |rocksdb.write_buffer_size|64MB|Sets the amount of data to build up in memory (backed by an unsorted log on disk) before converting to a sor