skynet_vs2017.zip

Introduction ====== Sproto is an efficient serialization library for C, and focuses on lua binding. It's like Google protocol buffers, but much faster. The design is simple. It only supports a few types that lua supports. It can be easily bound to other dynamic languages, or be used directly in C. In my i5-2500 @3.3GHz CPU, the benchmark is below: The schema in sproto: ``` .Person { name 0 : string id 1 : integer email 2 : string .PhoneNumber { number 0 : string type 1 : integer } phone 3 : *PhoneNumber } .AddressBook { person 0 : *Person } ``` It's equal to: ``` message Person { required string name = 1; required int32 id = 2; optional string email = 3; message PhoneNumber { required string number = 1; optional int32 type = 2 ; } repeated PhoneNumber phone = 4; } message AddressBook { repeated Person person = 1; } ``` Use the data: ```lua local ab = { person = { { name = "Alice", id = 10000, phone = { { number = "123456789" , type = 1 }, { number = "87654321" , type = 2 }, } }, { name = "Bob", id = 20000, phone = { { number = "01234567890" , type = 3 }, } } } } ``` library| encode 1M times | decode 1M times | size -------| --------------- | --------------- | ---- sproto | 2.15s | 7.84s | 83 bytes sproto (nopack) |1.58s | 6.93s | 130 bytes pbc-lua | 6.94s | 16.9s | 69 bytes lua-cjson | 4.92s | 8.30s | 183 bytes * pbc-lua is a google protocol buffers library https://github.com/cloudwu/pbc * lua-cjson is a json library https://github.com/efelix/lua-cjson Parser ======= ```lua local parser = require "sprotoparser" ``` * `parser.parse` parses a sproto schema to a binary string. The parser is needed for parsing the sproto schema. You can use it to generate binary string offline. The schema text and the parser is not needed when your program is running. Lua API ======= ```lua local sproto = require "sproto" local sprotocore = require "sproto.core" -- optional ``` * `sproto.new(spbin)` creates a sproto object by a schema binary string (generates by parser). * `sprotocore.newproto(spbin)` creates a sproto c object by a schema binary string (generates by parser). * `sproto.sharenew(spbin)` share a sproto object from a sproto c object (generates by sprotocore.newproto). * `sproto.parse(schema)` creares a sproto object by a schema text string (by calling parser.parse) * `sproto:exist_type(typename)` detect whether a type exist in sproto object. * `sproto:encode(typename, luatable)` encodes a lua table with typename into a binary string. * `sproto:decode(typename, blob [,sz])` decodes a binary string generated by sproto.encode with typename. If blob is a lightuserdata (C ptr), sz (integer) is needed. * `sproto:pencode(typename, luatable)` The same with sproto:encode, but pack (compress) the results. * `sproto:pdecode(typename, blob [,sz])` The same with sproto.decode, but unpack the blob (generated by sproto:pencode) first. * `sproto:default(typename, type)` Create a table with default values of typename. Type can be nil , "REQUEST", or "RESPONSE". RPC API ======= There is a lua wrapper for the core API for RPC . `sproto:host([packagename])` creates a host object to deliver the rpc message. `host:dispatch(blob [,sz])` unpack and decode (sproto:pdecode) the binary string with type the host created (packagename). If .type is exist, it's a REQUEST message with .type , returns "REQUEST", protoname, message, responser, .ud. The responser is a function for encode the response message. The responser will be nil when .session is not exist. If .type is not exist, it's a RESPONSE message for .session . Returns "RESPONSE", .session, message, .ud . `host:attach(sprotoobj)` creates a function(protoname, message, session, ud) to pack and encode request message with sprotoobj. If you don't want to use host object, you can also use these following apis to encode and decode the rpc message: `sproto:request_encode(protoname, tbl)` encode a request message with protoname. `sproto:response_encode(protoname, tbl)` encode a response message with protoname. `sproto:request_decode(protoname, blob [,sz])` decode a request message with protoname. `sproto:response_decode(protoname, blob [,sz]` decode a response message with protoname. Read testrpc.lua for detail. Schema Language ========== Like Protocol Buffers (but unlike json), sproto messages are strongly-typed and are not self-describing. You must define your message structure in a special language. You can use sprotoparser library to parse the schema text to a binary string, so that the sproto library can use it. You can parse them offline and save the string, or you can parse them during your program running. The schema text is like this: ``` # This is a comment. .Person { # . means a user defined type name 0 : string # string is a build-in type. id 1 : integer email 2 : string .PhoneNumber { # user defined type can be nest. number 0 : string type 1 : integer } phone 3 : *PhoneNumber # *PhoneNumber means an array of PhoneNumber. height 4 : integer(2) # (2) means a 1/100 fixed-point number. data 5 : binary # Some binary data } .AddressBook { person 0 : *Person(id) # (id) is optional, means Person.id is main index. } foobar 1 { # define a new protocol (for RPC used) with tag 1 request Person # Associate the type Person with foobar.request response { # define the foobar.response type ok 0 : boolean } } ``` A schema text can be self-described by the sproto schema language. ``` .type { .field { name 0 : string buildin 1 : integer type 2 : integer # type is fixed-point number precision when buildin is SPROTO_TINTEGER; When buildin is SPROTO_TSTRING, it means binary string when type is 1. tag 3 : integer array 4 : boolean key 5 : integer # If key exists, array must be true, and it's a map. } name 0 : string fields 1 : *field } .protocol { name 0 : string tag 1 : integer request 2 : integer # index response 3 : integer # index confirm 4 : boolean # response nil where confirm == true } .group { type 0 : *type protocol 1 : *protocol } ``` Types ======= * **string** : string * **binary** : binary string (it's a sub type of string) * **integer** : integer, the max length of an integer is signed 64bit. It can be a fixed-point number with specified precision. * **boolean** : true or false You can add * before the typename to declare an array. You can also specify a main index, the array whould be encode as an unordered map. User defined type can be any name in alphanumeric characters except the build-in typenames, and nested types are supported. * Where are double or real types? I have been using Google protocol buffers for many years in many projects, and I found the real types were seldom used. If you really need it, you can use string to serialize the double numbers. When you need decimal, you can specify the fixed-point presision. * Where is enum? In lua, enum types are not very useful. You can use integer to define an enum table in lua. Wire protocol ======== Each integer number must be serialized in little-endian format. The sproto message must be a user defined type struct, and a struct is encoded in three parts. The header, the field part, and the data part. The tag and small integer or boolean will be encoded in field part, and others are in data part. All the fields must be encoded in ascending order (by tag, base 0). The tags of fields can be discontinuous, if a field is nil. (default value in lua), don't encode it in message. The header is a 16bit integer. It is the number of fields. Each f