# unique-stream
node.js through stream that emits a unique stream of objects based on criteria
[![Build Status](https://travis-ci.org/eugeneware/unique-stream.svg?branch=master)](https://travis-ci.org/eugeneware/unique-stream)
[![Coverage Status](https://coveralls.io/repos/eugeneware/unique-stream/badge.svg?branch=master&service=github)](https://coveralls.io/github/eugeneware/unique-stream?branch=master)
## Installation
Install via [npm](https://www.npmjs.com/):
```
$ npm install unique-stream
```
## Examples
### Dedupe a ReadStream based on JSON.stringify:
``` js
var unique = require('unique-stream')
, Stream = require('stream');
// return a stream of 3 identical objects
function makeStreamOfObjects() {
var s = new Stream;
s.readable = true;
var count = 3;
for (var i = 0; i < 3; i++) {
setImmediate(function () {
s.emit('data', { name: 'Bob', number: 123 });
--count || end();
});
}
function end() {
s.emit('end');
}
return s;
}
// Will only print out one object as the rest are dupes. (Uses JSON.stringify)
makeStreamOfObjects()
.pipe(unique())
.on('data', console.log);
```
### Dedupe a ReadStream based on an object property:
``` js
// Use name as the key field to dedupe on. Will only print one object
makeStreamOfObjects()
.pipe(unique('name'))
.on('data', console.log);
```
### Dedupe a ReadStream based on a custom function:
``` js
// Use a custom function to dedupe on. Use the 'number' field. Will only print one object.
makeStreamOfObjects()
.pipe(function (data) {
return data.number;
})
.on('data', console.log);
```
## Dedupe multiple streams
The reason I wrote this was to dedupe multiple object streams:
``` js
var aggregator = unique();
// Stream 1
makeStreamOfObjects()
.pipe(aggregator);
// Stream 2
makeStreamOfObjects()
.pipe(aggregator);
// Stream 3
makeStreamOfObjects()
.pipe(aggregator);
aggregator.on('data', console.log);
```
## Use a custom store to record keys that have been encountered
By default a set is used to store keys encountered so far, in order to check new ones for
uniqueness. You can supply your own store instead, providing it supports the add(key) and
has(key) methods. This could allow you to use a persistent store so that already encountered
objects are not re-streamed when node is reloaded.
``` js
var keyStore = {
store: {},
add: function(key) {
this.store[key] = true;
},
has: function(key) {
return this.store[key] !== undefined;
}
};
makeStreamOfObjects()
.pipe(unique('name', keyStore))
.on('data', console.log);
```
## Contributing
unique-stream is an **OPEN Open Source Project**. This means that:
> Individuals making significant and valuable contributions are given commit-access to the project to contribute as they see fit. This project is more like an open wiki than a standard guarded open source project.
See the [CONTRIBUTING.md](https://github.com/eugeneware/unique-stream/blob/master/CONTRIBUTING.md) file for more details.
### Contributors
unique-stream is only possible due to the excellent work of the following contributors:
<table><tbody>
<tr><th align="left">Eugene Ware</th><td><a href="https://github.com/eugeneware">GitHub/eugeneware</a></td></tr>
<tr><th align="left">Craig Ambrose</th><td><a href="https://github.com/craigambrose">GitHub/craigambrose</a></td></tr>
<tr><th align="left">Shinnosuke Watanabe</th><td><a href="https://github.com/shinnn">GitHub/shinnn</a></td></tr>
<tr><th align="left">Rouven Weßling</th><td><a href="https://github.com/realityking">GitHub/realityking</a></td></tr>
</tbody></table>
node.js通过基于条件发出唯一对象流的流.zip
需积分: 5 193 浏览量
2023-03-16
23:31:55
上传
评论
收藏 12KB ZIP 举报
快撑死的鱼
- 粉丝: 1w+
- 资源: 9153
最新资源
- 雷克沙u盘加密软件windows客户端
- JavaSwing订单管理系统.zip
- 数据库管理工具:dbeaver-ce-23.2.4-macos-aarch64.dmg
- 一个能够使用的幻兽帕鲁修改器
- java期末大作业课程设计学生成绩管理系统源码+数据库(高分项目)
- 数据库管理工具:dbeaver-ce-23.2.2-stable.x86-64.rpm
- 数据库管理工具:dbeaver-ce-23.2.2-macos-x86-64.dmg
- 数据库管理工具:dbeaver-ce-23.2.2-macos-aarch64.dmg
- 数据库管理工具:dbeaver-ce-23.1.5-stable.x86-64.rpm
- 以下是一些适用于英语六级作文的万能句型模板,涵盖了引言、正文和结论部分的各类表达方式.docx
资源上传下载、课程学习等过程中有任何疑问或建议,欢迎提出宝贵意见哦~我们会及时处理!
点击此处反馈