md-tokenizer:流式降价标记器

md-tokenizer 流式降价标记器。 例子 README.md： # Hello World Some text var fs = require ( "fs" ) var mdtok = require ( "md-tokenizer" ) var t = mdtok ( function ( src , token ) { JSON . stringify ( [ token . type , src ] ) } ) fs . createReadStream ( "README.md" ) . pipe ( t ) 输出： [ "heading 1" , "# " ] [ "text" , "Hello" ] [ "whitespace" , " " ] [ "text" , "World" ] [ "new line" , "\n" ] [ "text" ,