File format
===========
<beginning_of_file>
[data block 1]
[data block 2]
...
[data block N]
[meta block 1]
...
[meta block K]
[metaindex block]
[index block]
[Footer] (fixed size; starts at file_size - sizeof(Footer))
<end_of_file>
The file contains internal pointers. Each such pointer is called
a BlockHandle and contains the following information:
offset: varint64
size: varint64
(1) The sequence of key/value pairs in the file are stored in sorted
order and partitioned into a sequence of data blocks. These blocks
come one after another at the beginning of the file. Each data block
is formatted according to the code in block_builder.cc, and then
optionally compressed.
(2) After the data blocks we store a bunch of meta blocks. The
supported meta block types are described below. More meta block types
may be added in the future. Each meta block is again formatted using
block_builder.cc and then optionally compressed.
(3) A "metaindex" block. It contains one entry for every other meta
block where the key is the name of the meta block and the value is a
BlockHandle pointing to that meta block.
(4) An "index" block. This block contains one entry per data block,
where the key is a string >= last key in that data block and before
the first key in the successive data block. The value is the
BlockHandle for the data block.
(6) At the very end of the file is a fixed length footer that contains
the BlockHandle of the metaindex and index blocks as well as a magic number.
metaindex_handle: char[p]; // Block handle for metaindex
index_handle: char[q]; // Block handle for index
padding: char[40-p-q]; // 0 bytes to make fixed length
// (40==2*BlockHandle::kMaxEncodedLength)
magic: fixed64; // == 0xdb4775248b80fb57
"filter" Meta Block
-------------------
If a "FilterPolicy" was specified when the database was opened, a
filter block is stored in each table. The "metaindex" block contains
an entry that maps from "filter.<N>" to the BlockHandle for the filter
block where "<N>" is the string returned by the filter policy's
"Name()" method.
The filter block stores a sequence of filters, where filter i contains
the output of FilterPolicy::CreateFilter() on all keys that are stored
in a block whose file offset falls within the range
[ i*base ... (i+1)*base-1 ]
Currently, "base" is 2KB. So for example, if blocks X and Y start in
the range [ 0KB .. 2KB-1 ], all of the keys in X and Y will be
converted to a filter by calling FilterPolicy::CreateFilter(), and the
resulting filter will be stored as the first filter in the filter
block.
The filter block is formatted as follows:
[filter 0]
[filter 1]
[filter 2]
...
[filter N-1]
[offset of filter 0] : 4 bytes
[offset of filter 1] : 4 bytes
[offset of filter 2] : 4 bytes
...
[offset of filter N-1] : 4 bytes
[offset of beginning of offset array] : 4 bytes
lg(base) : 1 byte
The offset array at the end of the filter block allows efficient
mapping from a data block offset to the corresponding filter.
"stats" Meta Block
------------------
This meta block contains a bunch of stats. The key is the name
of the statistic. The value contains the statistic.
TODO(postrelease): record following stats.
data size
index size
key size (uncompressed)
value size (uncompressed)
number of entries
number of data blocks
没有合适的资源?快使用搜索试试~ 我知道了~
温馨提示
共130个文件
cc:62个
h:52个
html:3个
Leveldb是一个google实现的非常高效的kv数据库,版本1.2已能支持billion级别的数据量了。 在这个数量级别下还有着非常高的性能,主要归功于它的良好的设计。特别是LSM算法。 那么数据库最怕的的随机IO他是如何解决的呢? 先说随机写,它的写都是先记录到日志文件去的,在日志文件满之前只是简单的更新memtable,那么就把随机写转化成了顺序写。在日志满了后,把日志里面的数据排序写成sst表同时和之前的sst进行合并,这个动作也是顺序读和写。大家都知道传统磁盘raid的顺序读写吞吐量是很大的,100M左右是没有问题。在写日志文件的时候,用到是buffer IO,也就是说如果操作系统有足够的内存,这个读写全部由操作系统缓冲,效果非常好。即使是sync写模式,也是以数据累计到4K为一个单位写的,所以效率高。 那么随机读呢?这个它解决不了。但是ssd盘最擅长随机读了。这个硬件很自然的解决了这个问题。 所以leveldb的绝配是ssd盘的raid.
资源推荐
资源详情
资源评论
收起资源包目录
leveldb-1.5.0.tar.gz (130个子文件)
AUTHORS 193B
build_detect_platform 6KB
c_test.c 11KB
db_test.cc 55KB
version_set.cc 44KB
db_impl.cc 43KB
db_bench.cc 28KB
table_test.cc 23KB
db_bench_sqlite3.cc 22KB
db_bench_tree_db.cc 16KB
env_posix.cc 15KB
c.cc 15KB
crc32c.cc 15KB
log_test.cc 13KB
repair.cc 12KB
skiplist_test.cc 10KB
corruption_test.cc 10KB
memenv.cc 9KB
cache.cc 9KB
table.cc 8KB
table_builder.cc 8KB
db_iter.cc 8KB
block.cc 8KB
log_reader.cc 8KB
version_edit.cc 7KB
memenv_test.cc 7KB
coding_test.cc 5KB
version_set_test.cc 5KB
merger.cc 5KB
coding.cc 5KB
memtable.cc 5KB
histogram.cc 5KB
two_level_iterator.cc 5KB
cache_test.cc 5KB
format.cc 4KB
dbformat.cc 4KB
write_batch.cc 4KB
filename.cc 4KB
dbformat_test.cc 4KB
bloom_test.cc 4KB
filter_block_test.cc 4KB
block_builder.cc 4KB
table_cache.cc 3KB
write_batch_test.cc 3KB
filter_block.cc 3KB
filename_test.cc 3KB
bloom.cc 3KB
log_writer.cc 3KB
env_test.cc 2KB
comparator.cc 2KB
builder.cc 2KB
env.cc 2KB
arena.cc 2KB
status.cc 2KB
logging.cc 2KB
crc32c_test.cc 2KB
iterator.cc 2KB
arena_test.cc 2KB
testharness.cc 2KB
testutil.cc 1KB
port_posix.cc 1KB
version_edit_test.cc 1KB
hash.cc 980B
options.cc 764B
filter_policy.cc 333B
doc.css 1KB
.gitignore 61B
version_set.h 13KB
env.h 11KB
skiplist.h 11KB
c.h 9KB
dbformat.h 8KB
options.h 7KB
db_impl.h 6KB
db.h 6KB
atomic_pointer.h 5KB
testharness.h 5KB
port_example.h 5KB
coding.h 4KB
port_posix.h 4KB
log_reader.h 3KB
cache.h 3KB
iterator.h 3KB
table_builder.h 3KB
status.h 3KB
format.h 3KB
slice.h 3KB
version_edit.h 3KB
table.h 3KB
filter_policy.h 3KB
filename.h 3KB
posix_logger.h 3KB
memtable.h 3KB
comparator.h 2KB
filter_block.h 2KB
random.h 2KB
iterator_wrapper.h 2KB
table_cache.h 2KB
write_batch.h 2KB
arena.h 2KB
共 130 条
- 1
- 2
资源评论
- shine12702016-11-01版本较老,有点贵
- 覃强2013-03-16哈哈,相当不错。对于无法用git的人是个好消息
- cx0012015-11-11一般般 配置caffe的附加
mipmap
- 粉丝: 1
- 资源: 19
上传资源 快速赚钱
- 我的内容管理 展开
- 我的资源 快来上传第一个资源
- 我的收益 登录查看自己的收益
- 我的积分 登录查看自己的积分
- 我的C币 登录后查看C币余额
- 我的收藏
- 我的下载
- 下载帮助
最新资源
- C语言基础-C语言编程基础之Leetcode编程题解之第39题组合总和.zip
- C语言基础-C语言编程基础之Leetcode编程题解之第38题外观数列.zip
- C语言基础-C语言编程基础之Leetcode编程题解之第37题解数独.zip
- C语言基础-C语言编程基础之Leetcode编程题解之第36题有效的数独.zip
- C语言基础-C语言编程基础之Leetcode编程题解之第35题搜索插入位置.zip
- index.wxml
- C语言基础-C语言编程基础之Leetcode编程题解之第33题搜索旋转排序数组.zip
- 基于Python实现的手写数字识别系统源码.zip
- 从网页提取禁止转载的文字
- C语言基础-C语言编程基础之Leetcode编程题解之第32题最长有效括号.zip
资源上传下载、课程学习等过程中有任何疑问或建议,欢迎提出宝贵意见哦~我们会及时处理!
点击此处反馈
安全验证
文档复制为VIP权益,开通VIP直接复制
信息提交成功