Introduction
======
Sproto is an efficient serialization library for C, and focuses on lua binding. It's like Google protocol buffers, but much faster.
The design is simple. It only supports a few types that lua supports. It can be easily bound to other dynamic languages, or be used directly in C.
In my i5-2500 @3.3GHz CPU, the benchmark is below:
The schema in sproto:
```
.Person {
name 0 : string
id 1 : integer
email 2 : string
.PhoneNumber {
number 0 : string
type 1 : integer
}
phone 3 : *PhoneNumber
}
.AddressBook {
person 0 : *Person
}
```
It's equal to:
```
message Person {
required string name = 1;
required int32 id = 2;
optional string email = 3;
message PhoneNumber {
required string number = 1;
optional int32 type = 2 ;
}
repeated PhoneNumber phone = 4;
}
message AddressBook {
repeated Person person = 1;
}
```
Use the data:
```lua
local ab = {
person = {
{
name = "Alice",
id = 10000,
phone = {
{ number = "123456789" , type = 1 },
{ number = "87654321" , type = 2 },
}
},
{
name = "Bob",
id = 20000,
phone = {
{ number = "01234567890" , type = 3 },
}
}
}
}
```
library| encode 1M times | decode 1M times | size
-------| --------------- | --------------- | ----
sproto | 2.15s | 7.84s | 83 bytes
sproto (nopack) |1.58s | 6.93s | 130 bytes
pbc-lua | 6.94s | 16.9s | 69 bytes
lua-cjson | 4.92s | 8.30s | 183 bytes
* pbc-lua is a google protocol buffers library https://github.com/cloudwu/pbc
* lua-cjson is a json library https://github.com/efelix/lua-cjson
Lua API
=======
```lua
local parser = require "sprotoparser"
```
* `parser.parse` parses a sproto schema to a binary string.
The parser is needed for parsing the sproto schema. You can use it to generate binary string offline. The schema text and the parser is not needed when your program is running.
```lua
local sproto = require "sproto.core"
```
* `sproto.newproto(sp)` creates a sproto object by a schema string (generates by parser).
* `sproto.querytype(sp, typename)` queries a type object from a sproto object by typename.
* `sproto.encode(st, luatable)` encodes a lua table by a type object, and generates a string message.
* `sproto.decode(st, message)` decodes a message string generated by sproto.encode with type.
* `sproto.pack(sprotomessage)` packs a string encoded by sproto.encode to reduce the size.
* `sproto.unpack(packedmessage)` unpacks the string packed by sproto.pack.
The sproto supports protocol tag for RPC. Use `sproto.protocol(tagorname)` to convert the protocol name to the tag id, or convert back from tag id to the name, and returns the request/response message type objects of this protocol.
RPC API
=======
There is a lua wrapper for the core API for RPC .
Read testrpc.lua for detail.
Schema Language
==========
Like Protocol Buffers (but unlike json), sproto messages are strongly-typed and are not self-describing. You must define your message structure in a special language.
You can use sprotoparser library to parse the schema text to a binary string, so that the sproto library can use it.
You can parse them offline and save the string, or you can parse them during your program running.
The schema text is like this:
```
# This is a comment.
.Person { # . means a user defined type
name 0 : string # string is a build-in type.
id 1 : integer
email 2 : string
.PhoneNumber { # user defined type can be nest.
number 0 : string
type 1 : integer
}
phone 3 : *PhoneNumber # *PhoneNumber means an array of PhoneNumber.
}
.AddressBook {
person 0 : *Person(id) # (id) is optional, means Person.id is main index.
}
foobar 1 { # define a new protocol (for RPC used) with tag 1
request Person # Associate the type Person with foobar.request
response { # define the foobar.response type
ok 0 : boolean
}
}
```
A schema text can be self-described by the sproto schema language.
```
.type {
.field {
name 0 : string
type 1 : string
id 2 : integer
array 3 : boolean
key 4 : integer # optional tag for map
}
name 0 : string
fields 1 : *field
}
.protocol {
name 0 : string
id 1 : integer
request 2 : string
response 3 : string
}
.group {
type 0 : *type
protocol 1 : *protocol
}
```
Types
=======
* **string** : binary string
* **integer** : integer, the max length of an integer is signed 64bit.
* **boolean** : true or false
You can add * before the typename to declare an array.
You can also specify a main index, the array whould be encode as an unordered map.
User defined type can be any name in alphanumeric characters except the build-in typenames, and nested types are supported.
* Where are double or real types?
I have been using Google protocol buffers for many years in many projects, and I found the real types were seldom used. If you really need it, you can use string to serialize the double numbers.
* Where is enum?
In lua, enum types are not very useful. You can use integer to define an enum table in lua.
Wire protocol
========
Each integer number must be serialized in little-endian format.
The sproto message must be a user defined type struct, and a struct is encoded in three parts. The header, the field part, and the data part.
The tag and small integer or boolean will be encoded in field part, and others are in data part.
All the fields must be encoded in ascending order (by tag). The tags of fields can be discontinuous, if a field is nil. (default value in lua), don't encode it in message.
The header is a 16bit integer. It is the number of fields.
Each field in field part is a 16bit integer (n). If n is zero, that means the field data is encoded in data part ;
If n is even (and not zero), the value of this field is n/2-1 ;
If n is odd, that means the tags is not continuous, and we should add current tag by (n+1)/2 .
Read the examples below to see more details.
Notice: If the tag is not declared in schema, the decoder will simply ignore the field for protocol version compatibility.
Example 1:
```
person { name = "Alice" , age = 13, marital = false }
03 00 (fn = 3)
00 00 (id = 0, value in data part)
1C 00 (id = 1, value = 13)
02 00 (id = 2, value = false)
05 00 00 00 (sizeof "Alice")
41 6C 69 63 65 ("Alice")
```
Example 2:
```
person {
name = "Bob",
age = 40,
children = {
{ name = "Alice" , age = 13, marital = false },
}
}
04 00 (fn = 4)
00 00 (id = 0, value in data part)
52 00 (id = 1, value = 40)
01 00 (skip id = 2)
00 00 (id = 3, value in data part)
03 00 00 00 (sizeof "Bob")
42 6F 62 ("Bob")
11 00 00 00 (sizeof struct)
03 00 (fn = 3)
00 00 (id = 0, value in data part)
1C 00 (id = 1, value = 13)
02 00 (id = 2, value = false)
05 00 00 00 (sizeof "Alice")
41 6C 69 63 65 ("Alice")
```
0 Packing
=======
The algorithm is very similar to [Cap'n proto](http://kentonv.github.io/capnproto/), but 0x00 is not treated specially.
In packed format, the message if padding to 8. Each 8 byte is reduced to a tag byte followed by zero to eight content bytes.
The bits of the tag byte correspond to the bytes of the unpacked word, with the least-significant bit corresponding to the first byte.
Each zero bit indicates that the corresponding byte is zero. The non-zero bytes are packed following the tag.
For example:
```
unpacked (hex): 08 00 00 00 03 00 02 00 19 00 00 00 aa 01 00 00
packed (hex): 51 08 03 02 31 19 aa 01
```
Tag 0xff is treated specially. A number N is following the 0xff tag. N means (N+1)*8 bytes should be copied directly.
The bytes may or may not contain zeros. Because of this rule, the worst-case space overhead of packing is 2 bytes per 2 KiB of inpu
没有合适的资源?快使用搜索试试~ 我知道了~
skynet vs2013编译版本
共974个文件
tlog:195个
lua:115个
c:92个
需积分: 21 26 下载量 23 浏览量
2018-07-05
20:55:34
上传
评论
收藏 13.48MB RAR 举报
温馨提示
windows环境下编译的 skynet 源码,使用vs2013编译器,方便调试学习
资源详情
资源评论
资源推荐
收起资源包目录
skynet vs2013编译版本 (974个子文件)
start_skynet.bat 26B
lparser.c 45KB
lstrlib.c 43KB
lvm.c 42KB
socket_server.c 39KB
lgc.c 36KB
lptree.c 35KB
lapi.c 31KB
lauxlib.c 31KB
lpcode.c 30KB
sproto.c 26KB
lua-bson.c 25KB
lua-crypt.c 25KB
lcode.c 24KB
loadlib.c 23KB
ldo.c 22KB
liolib.c 19KB
ltable.c 19KB
ldebug.c 19KB
skynet_server.c 18KB
service_harbor.c 18KB
lsproto.c 18KB
lua.c 18KB
llex.c 17KB
lua-sharedata.c 16KB
lpcap.c 16KB
lua-socket.c 15KB
lbaselib.c 14KB
lobject.c 14KB
serialize.c 13KB
ldblib.c 12KB
lua-seri.c 12KB
lua-mongo.c 12KB
lua-cluster.c 10KB
lpvm.c 10KB
lua-netpack.c 10KB
luac.c 10KB
lstring.c 10KB
ltablib.c 10KB
skiplist.c 10KB
lmathlib.c 10KB
service_gate.c 10KB
loslib.c 9KB
lsha1.c 9KB
lua-skynet.c 8KB
lstate.c 8KB
lutf8lib.c 7KB
md5.c 6KB
lua-debugchannel.c 6KB
skynet_timer.c 6KB
lundump.c 6KB
ltask.c 6KB
lpprint.c 5KB
atomic_lock.c 5KB
skynet_start.c 5KB
lua-stm.c 5KB
skynet_handle.c 5KB
md5lib.c 5KB
skynet_socket.c 5KB
malloc_hook.c 5KB
lbitlib.c 5KB
schedule.c 5KB
skynet_mq.c 5KB
lua-skiplist.c 5KB
ldump.c 4KB
handlemap.c 4KB
unistd.c 4KB
lua-clientsocket.c 4KB
lfunc.c 4KB
ltm.c 4KB
lua-profile.c 4KB
service_snlua.c 4KB
lua-mysqlaux.c 4KB
lcorolib.c 4KB
skynet_module.c 4KB
lua-multicast.c 4KB
lopcodes.c 3KB
skynet_main.c 3KB
lmem.c 3KB
lctype.c 2KB
skynet_log.c 2KB
linit.c 2KB
skynet_daemon.c 2KB
lzio.c 2KB
queue.c 2KB
service_logger.c 1KB
skynet_error.c 1KB
skynet_harbor.c 1KB
skynet_monitor.c 1KB
lua-memory.c 894B
skynet_env.c 782B
compat-5.2.c 706B
dlfcn.c 486B
config.c1 293B
config.c2 293B
config 569B
config_log 245B
cpoll.cpp 8KB
skynet.def 2KB
posix.def 442B
共 974 条
- 1
- 2
- 3
- 4
- 5
- 6
- 10
GJQI12
- 粉丝: 73
- 资源: 132
上传资源 快速赚钱
- 我的内容管理 展开
- 我的资源 快来上传第一个资源
- 我的收益 登录查看自己的收益
- 我的积分 登录查看自己的积分
- 我的C币 登录后查看C币余额
- 我的收藏
- 我的下载
- 下载帮助
安全验证
文档复制为VIP权益,开通VIP直接复制
信息提交成功
评论0