PolarDB-X工具，如数据迁移、CSV文件导入、基准测试等

[](https://github.com/ApsaraDB/galaxysql/blob/main/LICENSE) [](https://www.java.com/) ## Batch-Tool 介绍 Batch Tool工具是专为 PolarDB-X数据库提供数据导入导出服务的工具。 其结合分布式数据库特点实现一站式且高效地从文件导入、导出到文件以及跨库的离线数据迁移(MySQL / PolarDB-X 1.0 / PolarDB-X 2.0)等功能， 在此基础上，还支持基于文本文件批量更新、删除等功能 (实验特性)。 ## 快速上手 常见场景与问题排查可参考文档 [usage-details](docs/usage-details.md)。 ### 源码打包 `mvn clean package -DskipTests` ### 参数介绍 命令行用法：`java -jar batch-tool.jar --help` ``` usage: BatchTool [-batchsize <size>] [-col <col1;col2;col3>] [-comp <NONE | GZIP>] [-con <consumer count>] [-config <filepath>] [-cs <charset>] [-D <database>] [-DDL <NONE | ONLY | WITH>] [-dir <directory path>] [-encrypt <NONE | AES | SM4>] [-error <max error count>] [-f <filepath1;filepath2>] [-F <file count>] [-fcon <parallelism>] [-format <NONE | TXT | CSV | XLS | XLSX>] [-func <true | false>] [-h <host>] [-H <filepath>] [-header <true | false>] [-help] [-i <true | false>] [-in <true | false>] [-initSqls <sqls>] [-key <string-type key>] [-L <line count>] [-lastSep <true | false>] [-lb <true | false>] [-local <true | false>] [-mask <Json format config>] [-maxConn <max connection>] [-maxWait <wait time(ms)>] [-minConn <min connection>] [-noEsc <true | false>] [-o <operation>] [-O <asc | desc>] [-OC <col1;col2;col3>] [-p <password>] [-P <port>] [-para <true | false>] [-param <key1=val1&key2=val2>] [-perf <true | false>] [-pre <prefix>] [-pro <producer count>] [-quote <AUTO | FORCE | NONE>] [-readsize <size(MB)>] [-rfonly <true | false>] [-ringsize <size (power of 2)>] [-s <separator char or string>] [-sharding <true | false>] [-t <tableName>] [-tps <tps limit>] [-u <username>] [-v] [-w <where condition>] -batchsize,--batchSize <size> Batch size of insert. -col,--columns <col1;col2;col3> Target columns for export. -comp,--compress <NONE | GZIP> Export or import compressed file (default NONE). -con,--consumer <consumer count> Configure number of consumer threads. -config,--configFile <filepath> Use yaml config file. -cs,--charset <charset> The charset of files. -D,--database <database> Database name. -DDL,--DDL <NONE | ONLY | WITH> Export or import with DDL sql mode (default NONE). -dir,--directory <directory path> Directory path including files to import. -encrypt,--encrypt <NONE | AES | SM4> Export or import with encrypted file (default NONE). -error,--maxError <max error count> Max error count threshold, program exits when the limit is exceeded. -f,--file <filepath1;filepath2> Source file(s). -F,--filenum <file count> Fixed number of exported files. -fcon,--forceConsumer <parallelism> Configure if allow force consumer parallelism. -format,--fileFormat <NONE | TXT | CSV | XLS | XLSX> File format (default NONE). -func,--sqlFunc <true | false> Use sql function to update (default false). -h,--host <host> Host of database. -H,--historyFile <filepath> History file name. -header,--header <true | false> Whether the header line is column names (default false). -help,--help Help message. -i,--ignore <true | false> Flag of insert ignore and resume breakpoint (default false). -in,--whereIn <true | false> Using where cols in (values). -initSqls,--initSqls <sqls> Connection init sqls (druid). -key,--secretKey <string-type key> Secret key used during encryption. -L,--line <line count> Max line limit of one single export file. -lastSep,--withLastSep <true | false> Whether line ends with separator (default false). -lb,--loadbalance <true | false> Use jdbc load balance, filling the arg in $host like 'host1:port1,host2:port2' (default false). -local,--localMerge <true | false> Use local merge sort (default false). -mask,--mask <Json format config> Masking sensitive columns while exporting data. -maxConn,--maxConnection <max connection> Max connection count (druid). -maxWait,--connMaxWait <wait time(ms)> Max wait time when getting a connection. -minConn,--minConnection <min connection> Min connection count (druid). -noEsc,--noEscape <true | false> Do not escape value for sql (default false). -o,--operation <operation> Batch operation type: export / import / delete / update. -O,--orderby <asc | desc> Order by type: asc / desc. -OC,--orderCol <col1;col2;col3> Ordered column names. -p,--password <password> Password of user. -P,--port <port> Port number of database. -para,--paraMerge <true | false> Use parallel merge when doing order by export (default false). -param,--connParam <key1=val1&key2=val2> Jdbc connection params. -perf,--perfMode <true | false> Use performance mode at the sacrifice of compatibility (default false). -pre,--prefix <prefix> Export file name prefix. -pro,--producer <producer count> Configure number of producer threads (export / import). -quote,--quoteMode <AUTO | FORCE | NONE> The mode of how field values are enclosed by double-quotes when exporting table (default AUTO). -readsize,--readSize <size(MB)> Read block size. -rfonly,--readFileOnly <true | false> Only read and process file, no sql execution (default false). -ringsize,--ringSize <size (power of 2)> Ring buffer size. -s,--sep <separator char or string> Separator between fields (delimiter). -sharding,--sharding <true | false> Whether enable sharding mode (default value depends on operation). -t,--table <tableName> Target table. -tps,--tpsLimit <tps limit> Configure of tps limit (default -1: no limit). -u,--user <username> User for login. -v,--version