# S2 Compression
S2 is an extension of [Snappy](https://github.com/google/snappy).
S2 is aimed for high throughput, which is why it features concurrent compression for bigger payloads.
Decoding is compatible with Snappy compressed content, but content compressed with S2 cannot be decompressed by Snappy.
This means that S2 can seamlessly replace Snappy without converting compressed content.
S2 can produce Snappy compatible output, faster and better than Snappy.
If you want full benefit of the changes you should use s2 without Snappy compatibility.
S2 is designed to have high throughput on content that cannot be compressed.
This is important, so you don't have to worry about spending CPU cycles on already compressed data.
## Benefits over Snappy
* Better compression
* Adjustable compression (3 levels)
* Concurrent stream compression
* Faster decompression, even for Snappy compatible content
* Concurrent Snappy/S2 stream decompression
* Skip forward in compressed stream
* Random seeking with indexes
* Compatible with reading Snappy compressed content
* Smaller block size overhead on incompressible blocks
* Block concatenation
* Block Dictionary support
* Uncompressed stream mode
* Automatic stream size padding
* Snappy compatible block compression
## Drawbacks over Snappy
* Not optimized for 32 bit systems
* Streams use slightly more memory due to larger blocks and concurrency (configurable)
# Usage
Installation: `go get -u github.com/klauspost/compress/s2`
Full package documentation:
[![godoc][1]][2]
[1]: https://godoc.org/github.com/klauspost/compress?status.svg
[2]: https://godoc.org/github.com/klauspost/compress/s2
## Compression
```Go
func EncodeStream(src io.Reader, dst io.Writer) error {
enc := s2.NewWriter(dst)
_, err := io.Copy(enc, src)
if err != nil {
enc.Close()
return err
}
// Blocks until compression is done.
return enc.Close()
}
```
You should always call `enc.Close()`, otherwise you will leak resources and your encode will be incomplete.
For the best throughput, you should attempt to reuse the `Writer` using the `Reset()` method.
The Writer in S2 is always buffered, therefore `NewBufferedWriter` in Snappy can be replaced with `NewWriter` in S2.
It is possible to flush any buffered data using the `Flush()` method.
This will block until all data sent to the encoder has been written to the output.
S2 also supports the `io.ReaderFrom` interface, which will consume all input from a reader.
As a final method to compress data, if you have a single block of data you would like to have encoded as a stream,
a slightly more efficient method is to use the `EncodeBuffer` method.
This will take ownership of the buffer until the stream is closed.
```Go
func EncodeStream(src []byte, dst io.Writer) error {
enc := s2.NewWriter(dst)
// The encoder owns the buffer until Flush or Close is called.
err := enc.EncodeBuffer(buf)
if err != nil {
enc.Close()
return err
}
// Blocks until compression is done.
return enc.Close()
}
```
Each call to `EncodeBuffer` will result in discrete blocks being created without buffering,
so it should only be used a single time per stream.
If you need to write several blocks, you should use the regular io.Writer interface.
## Decompression
```Go
func DecodeStream(src io.Reader, dst io.Writer) error {
dec := s2.NewReader(src)
_, err := io.Copy(dst, dec)
return err
}
```
Similar to the Writer, a Reader can be reused using the `Reset` method.
For the best possible throughput, there is a `EncodeBuffer(buf []byte)` function available.
However, it requires that the provided buffer isn't used after it is handed over to S2 and until the stream is flushed or closed.
For smaller data blocks, there is also a non-streaming interface: `Encode()`, `EncodeBetter()` and `Decode()`.
Do however note that these functions (similar to Snappy) does not provide validation of data,
so data corruption may be undetected. Stream encoding provides CRC checks of data.
It is possible to efficiently skip forward in a compressed stream using the `Skip()` method.
For big skips the decompressor is able to skip blocks without decompressing them.
## Single Blocks
Similar to Snappy S2 offers single block compression.
Blocks do not offer the same flexibility and safety as streams,
but may be preferable for very small payloads, less than 100K.
Using a simple `dst := s2.Encode(nil, src)` will compress `src` and return the compressed result.
It is possible to provide a destination buffer.
If the buffer has a capacity of `s2.MaxEncodedLen(len(src))` it will be used.
If not a new will be allocated.
Alternatively `EncodeBetter`/`EncodeBest` can also be used for better, but slightly slower compression.
Similarly to decompress a block you can use `dst, err := s2.Decode(nil, src)`.
Again an optional destination buffer can be supplied.
The `s2.DecodedLen(src)` can be used to get the minimum capacity needed.
If that is not satisfied a new buffer will be allocated.
Block function always operate on a single goroutine since it should only be used for small payloads.
# Commandline tools
Some very simply commandline tools are provided; `s2c` for compression and `s2d` for decompression.
Binaries can be downloaded on the [Releases Page](https://github.com/klauspost/compress/releases).
Installing then requires Go to be installed. To install them, use:
`go install github.com/klauspost/compress/s2/cmd/s2c@latest && go install github.com/klauspost/compress/s2/cmd/s2d@latest`
To build binaries to the current folder use:
`go build github.com/klauspost/compress/s2/cmd/s2c && go build github.com/klauspost/compress/s2/cmd/s2d`
## s2c
```
Usage: s2c [options] file1 file2
Compresses all files supplied as input separately.
Output files are written as 'filename.ext.s2' or 'filename.ext.snappy'.
By default output files will be overwritten.
Use - as the only file name to read from stdin and write to stdout.
Wildcards are accepted: testdir/*.txt will compress all files in testdir ending with .txt
Directories can be wildcards as well. testdir/*/*.txt will match testdir/subdir/b.txt
File names beginning with 'http://' and 'https://' will be downloaded and compressed.
Only http response code 200 is accepted.
Options:
-bench int
Run benchmark n times. No output will be written
-blocksize string
Max block size. Examples: 64K, 256K, 1M, 4M. Must be power of two and <= 4MB (default "4M")
-c Write all output to stdout. Multiple input files will be concatenated
-cpu int
Compress using this amount of threads (default 32)
-faster
Compress faster, but with a minor compression loss
-help
Display help
-index
Add seek index (default true)
-o string
Write output to another file. Single input file only
-pad string
Pad size to a multiple of this value, Examples: 500, 64K, 256K, 1M, 4M, etc (default "1")
-q Don't write any output to terminal, except errors
-rm
Delete source file(s) after successful compression
-safe
Do not overwrite output files
-slower
Compress more, but a lot slower
-snappy
Generate Snappy compatible output stream
-verify
Verify written files
```
## s2d
```
Usage: s2d [options] file1 file2
Decompresses all files supplied as input. Input files must end with '.s2' or '.snappy'.
Output file names have the extension removed. By default output files will be overwritten.
Use - as the only file name to read from stdin and write to stdout.
Wildcards are accepted: testdir/*.txt will compress all files in testdir ending with .txt
Directories can be wildcards as well. testdir/*/*.txt will match testdir/subdir/b.txt
File names beginning with 'http://' and 'https://' will be downloaded and decompressed.
Extensions on downloaded files are ignored. Only http response code 200 is accepted.
Options:
-bench int
Run benchmark n times
没有合适的资源?快使用搜索试试~ 我知道了~
Nging是一个网站服务程序,可以代替Nginx或Apache来搭建Web开发测试环境,并附带了实用的周边工具
共2000个文件
go:6727个
png:951个
js:883个
1.该资源内容由用户上传,如若侵权请联系客服进行举报
2.虚拟产品一经售出概不退款(资源遇到问题,请及时私信上传者)
2.虚拟产品一经售出概不退款(资源遇到问题,请及时私信上传者)
版权申诉
0 下载量 114 浏览量
2023-06-07
09:01:38
上传
评论
收藏 72.43MB ZIP 举报
温馨提示
Nging是一个网站服务程序,可以代替Nginx或Apache来搭建Web开发测试环境,并附带了实用的周边工具。例如:计划任务、MySQL管理、Redis管理、FTP管理、SSH管理、服务器管理等。本软件项目不仅仅实现了一些网站服务工具,本身还是一个具有很好扩展性的通用网站后台管理系统,通过本项目,您可以很轻松的构建一个全新的网站项目,省去从头构建项目的麻烦,减少重复性劳动。漂亮的Go语言通用后台管理系统,包含:计划任务 / MySQL管理 / Redis管理 / FTP管理 / SSH管理 / 服务器管理 / Caddy配置 / DDNS / FRP可视化配置 / 云存储管理 等功能。可运行于 Windows / Linux / MacOS 等主流系统平台和 树莓派 / 路由器 等ARM设备
资源推荐
资源详情
资源评论
收起资源包目录
Nging是一个网站服务程序,可以代替Nginx或Apache来搭建Web开发测试环境,并附带了实用的周边工具 (2000个子文件)
sqlite3-binding.c 8.35MB
c_helpers.c 4KB
iostat_darwin.c 4KB
smc_darwin.c 4KB
sqlite3_opt_unlock_notify.c 2KB
gccgo_c.c 2KB
cpu_gccgo_x86.c 1KB
style.min.css 223KB
flat-ui.css 150KB
bootstrap.css 142KB
style.css 128KB
flat-ui.min.css 128KB
bootstrap.min.css 117KB
bootstrap.min.css 107KB
fuelux.css 99KB
iconfont.css 92KB
fuelux.min.css 89KB
editormd.css 75KB
skin.min.css 60KB
skin.min.css 60KB
editormd.min.css 60KB
editormd.preview.css 58KB
editormd.preview.min.css 47KB
video-js.css 44KB
selectpage.css 41KB
video-js.min.css 39KB
font-awesome.css 37KB
font-awesome.min.css 30KB
ambiance.css 26KB
bootstrap-theme.css 25KB
katex.css 25KB
bootstrap-theme.min.css 23KB
katex.min.css 22KB
content.min.css 21KB
content.inline.min.css 21KB
content.inline.min.css 21KB
content.min.css 21KB
bootstrap-editable.css 21KB
skin.mobile.min.css 21KB
skin.mobile.min.css 21KB
bootstrap-table.css 19KB
select2.css 19KB
checkbox.min.css 18KB
component.css 18KB
bootstrap-table.min.css 17KB
swiper.css 15KB
bootstrap-editable.min.css 14KB
bootstrap-switch.css 14KB
swiper.min.css 13KB
colorpicker.css 13KB
font_OpenSans400_300_600_400italic_700_800.css 13KB
webx.css 12KB
dropzone.css 12KB
bootstrap-table-bulma.css 11KB
dropzone.css 11KB
bootstrap-table-semantic.css 11KB
bootstrap-table-foundation.css 11KB
bootstrap-table-materialize.css 11KB
webx.min.css 10KB
vertical-menu.css 10KB
bootstrap-table-bulma.min.css 10KB
bootstrap-table-semantic.min.css 10KB
bootstrap-table-foundation.min.css 10KB
bootstrap-table.css 10KB
bootstrap-table-materialize.min.css 10KB
dropzone.min.css 10KB
dropzone.min.css 9KB
daterangepicker.css 9KB
checkbox.css 9KB
bootstrap-table.min.css 9KB
bootstrap-switch.min.css 9KB
demo.css 8KB
vertical-menu.min.css 8KB
codemirror.css 8KB
jquery.accordion-menu.css 8KB
daterangepicker.min.css 7KB
magnific-popup.css 7KB
introjs.css 6KB
bootstrap-gtreetable.css 6KB
bootstrap-gtreetable.min.css 6KB
introjs.min.css 6KB
codemirror.min.css 5KB
magnific-popup.min.css 5KB
mdn-like.css 5KB
custom.css 5KB
solarized.css 5KB
slider.css 4KB
xterm.css 4KB
jquery-ui.custom.css 4KB
default.css 4KB
custom.min.css 4KB
select2-bootstrap.css 3KB
reset.css 3KB
jquery.Jcrop.css 3KB
merge.css 3KB
jquery-ui.custom.min.css 3KB
lint.css 3KB
xq-dark.css 3KB
bootstrap-dialog.css 2KB
jquery.uploadPreviewer.css 2KB
共 2000 条
- 1
- 2
- 3
- 4
- 5
- 6
- 20
资源评论
Java程序员-张凯
- 粉丝: 1w+
- 资源: 6651
上传资源 快速赚钱
- 我的内容管理 展开
- 我的资源 快来上传第一个资源
- 我的收益 登录查看自己的收益
- 我的积分 登录查看自己的积分
- 我的C币 登录后查看C币余额
- 我的收藏
- 我的下载
- 下载帮助
最新资源
- tensorflow-gpu-2.7.4-cp37-cp37m-manylinux2010-x86-64.whl
- 多段线、 圆、弧转多段线(仅我可见)
- tensorflow-2.7.2-cp38-cp38-manylinux2010-x86-64.whl
- yeyue-p8Yi4-ve4a83792.apk
- tensorflow-gpu-2.7.3-cp38-cp38-manylinux2010-x86-64.whl
- 五相感应电机矢量控制模型MATLAB
- RGLED (1) (1).circ
- IMG_20240427_215747.jpg
- python下前端WEB学习笔记
- 田间种植行排号自动生成工具
资源上传下载、课程学习等过程中有任何疑问或建议,欢迎提出宝贵意见哦~我们会及时处理!
点击此处反馈
安全验证
文档复制为VIP权益,开通VIP直接复制
信息提交成功