# Logstash
Logstash is part of the [Elastic Stack](https://www.elastic.co/products) along with Beats, Elasticsearch and Kibana. Logstash is a server-side data processing pipeline that ingests data from a multitude of sources simultaneously, transforms it, and then sends it to your favorite "stash." (Ours is Elasticsearch, naturally.). Logstash has over 200 plugins, and you can write your own very easily as well.
For more info, see <https://www.elastic.co/products/logstash>
## Documentation and Getting Started
You can find the documentation and getting started guides for Logstash
on the [elastic.co site](https://www.elastic.co/guide/en/logstash/current/getting-started-with-logstash.html)
For information about building the documentation, see the README in https://github.com/elastic/docs
## Downloads
You can download officially released Logstash binaries, as well as debian/rpm packages for the
supported platforms, from [downloads page](https://www.elastic.co/downloads/logstash).
## Need Help?
- [Logstash Forum](https://discuss.elastic.co/c/logstash)
- [Logstash Documentation](https://www.elastic.co/guide/en/logstash/current/index.html)
- [#logstash on freenode IRC](https://webchat.freenode.net/?channels=logstash)
- [Logstash Product Information](https://www.elastic.co/products/logstash)
- [Elastic Support](https://www.elastic.co/subscriptions)
## Logstash Plugins
Logstash plugins are hosted in separate repositories under the [logstash-plugins](https://github.com/logstash-plugins) github organization. Each plugin is a self-contained Ruby gem which gets published to RubyGems.org.
### Writing your own Plugin
Logstash is known for its extensibility. There are hundreds of plugins for Logstash and you can write your own very easily! For more info on developing and testing these plugins, please see the [working with plugins section](https://www.elastic.co/guide/en/logstash/current/contributing-to-logstash.html)
### Plugin Issues and Pull Requests
**Please open new issues and pull requests for plugins under its own repository**
For example, if you have to report an issue/enhancement for the Elasticsearch output, please do so [here](https://github.com/logstash-plugins/logstash-output-elasticsearch/issues).
Logstash core will continue to exist under this repository and all related issues and pull requests can be submitted here.
## Developing Logstash Core
### Prerequisites
* Install JDK version 8 or 11. Make sure to set the `JAVA_HOME` environment variable to the path to your JDK installation directory. For example `set JAVA_HOME=<JDK_PATH>`
* Install JRuby 9.2.x It is recommended to use a Ruby version manager such as [RVM](https://rvm.io/) or [rbenv](https://github.com/sstephenson/rbenv).
* Install `rake` and `bundler` tool using `gem install rake` and `gem install bundler` respectively.
### RVM install (optional)
If you prefer to use rvm (ruby version manager) to manage Ruby versions on your machine, follow these directions. In the Logstash folder:
```sh
gpg --keyserver hkp://keys.gnupg.net --recv-keys 409B6B1796C275462A1703113804BB82D39DC0E3
\curl -sSL https://get.rvm.io | bash -s stable --ruby=$(cat .ruby-version)
```
### Check Ruby version
Before you proceed, please check your ruby version by:
```sh
$ ruby -v
```
The printed version should be the same as in the `.ruby-version` file.
### Building Logstash
The Logstash project includes the source code for all of Logstash, including the Elastic-Licensed X-Pack features and functions; to run Logstash from source using only the OSS-licensed code, export the `OSS` environment variable with a value of `true`:
``` sh
export OSS=true
```
* To run Logstash from the repo you must first bootstrap the environment:
```sh
rake bootstrap
```
* You can then use `bin/logstash` to start Logstash, but there are no plugins installed. To install default plugins, you can run:
```sh
rake plugin:install-default
```
This will install the 80+ default plugins which makes Logstash ready to connect to multiple data sources, perform transformations and send the results to Elasticsearch and other destinations.
To verify your environment, run the following to send your first event:
```sh
bin/logstash -e 'input { stdin { } } output { stdout {} }'
```
This should start Logstash with stdin input waiting for you to enter an event
```sh
hello world
2016-11-11T01:22:14.405+0000 0.0.0.0 hello world
```
**Advanced: Drip Launcher**
[Drip](https://github.com/ninjudd/drip) is a tool that solves the slow JVM startup problem while developing Logstash. The drip script is intended to be a drop-in replacement for the java command. We recommend using drip during development, in particular for running tests. Using drip, the first invocation of a command will not be faster but the subsequent commands will be swift.
To tell logstash to use drip, set the environment variable `` JAVACMD=`which drip` ``.
Example (but see the *Testing* section below before running rspec for the first time):
JAVACMD=`which drip` bin/rspec
**Caveats**
Drip does not work with STDIN. You cannot use drip for running configs which use the stdin plugin.
## Building Logstash Documentation
To build the Logstash Reference (open source content only) on your local
machine, clone the following repos:
[logstash](https://github.com/elastic/logstash) - contains main docs about core features
[logstash-docs](https://github.com/elastic/logstash-docs) - contains generated plugin docs
[docs](https://github.com/elastic/docs) - contains doc build files
Make sure you have the same branch checked out in `logstash` and `logstash-docs`.
Check out `master` in the `docs` repo.
Run the doc build script from within the `docs` repo. For example:
```
./build_docs.pl --doc ../logstash/docs/index.asciidoc --chunk=1 -open
```
## Testing
Most of the unit tests in Logstash are written using [rspec](http://rspec.info/) for the Ruby parts. For the Java parts, we use junit. For testing you can use the *test* `rake` tasks and the `bin/rspec` command, see instructions below:
### Core tests
1- To run the core tests you can use the Gradle task:
./gradlew test
or use the `rspec` tool to run all tests or run a specific test:
bin/rspec
bin/rspec spec/foo/bar_spec.rb
Note that before running the `rspec` command for the first time you need to set up the RSpec test dependencies by running:
./gradlew bootstrap
2- To run the subset of tests covering the Java codebase only run:
./gradlew javaTests
3- To execute the complete test-suite including the integration tests run:
./gradlew check
4- To execute a single Ruby test run:
SPEC_OPTS="-fd -P logstash-core/spec/logstash/api/commands/default_metadata_spec.rb" ./gradlew :logstash-core:rubyTests --tests org.logstash.RSpecTests
5- To execute single spec for integration test, run:
./gradlew integrationTests -PrubyIntegrationSpecs=specs/slowlog_spec.rb
Sometimes you might find a change to a piece of Logstash code causes a test to hang. These can be hard to debug.
If you set `LS_JAVA_OPTS="-agentlib:jdwp=transport=dt_socket,server=y,suspend=n,address=5005"` you can connect to a running Logstash with your IDEs debugger which can be a great way of finding the issue.
### Plugins tests
To run the tests of all currently installed plugins:
rake test:plugin
You can install the default set of plugins included in the logstash package:
rake test:install-default
---
Note that if a plugin is installed using the plugin manager `bin/logstash-plugin install ...` do not forget to also install the plugins development dependencies using the following command after the plugin installation:
bin/logstash-plugin install --development
## Building Artifacts
Built artifacts will be placed in the `LS_HOME/build` directory, and will create the directory if it is not already present.
You can build a Logstash snapshot package as tarball or zip file
```sh
./gradlew assembleTarDi
没有合适的资源?快使用搜索试试~ 我知道了~
Logstash日志管理系统.rar
共2964个文件
json:1123个
rb:555个
java:482个
需积分: 5 0 下载量 22 浏览量
2023-07-21
14:47:39
上传
评论
收藏 20.85MB RAR 举报
温馨提示
由于传统的单体应用时代,大多数应用采用的是通过登录SSH客户端登服务器查看,使用less或者tail等指令。 使用ELK(Elasticsearch、Logstash、Kibana)和Kafka实现微服务统一日志管理,可以通过以下几个步骤来完成: 部署Kafka集群:首先需要部署一个Kafka集群,它将作为中间件来缓冲数据,确保高效的日志传输和处理。可以选择在Kubernetes中部署这些组件以实现容器化管理[2]。 部署ELK集群:在系统中部署Elasticsearch、Logstash和Kibana三个组件,搭建ELK日志系统。其中,Elasticsearch负责日志数据的存储与检索,Logstash负责从Kafka消费数据并将数据传输到Elasticsearch,Kibana则负责从Elasticsearch中读取日志数据并进行可视化展示[6]。 日志采集:可以使用Filebeat或者其他日志采集器收集微服务的日志数据,然后将数据发送到Kafka中[1]。 配置Logstash:配置Logstash以从Kafka消费数据,并将消费后的数据写入到Elasticsearch
资源推荐
资源详情
资源评论
收起资源包目录
Logstash日志管理系统.rar (2964个子文件)
releasenotes.asciidoc 217KB
pluginbody.asciidoc 46KB
configuration.asciidoc 37KB
advanced-pipeline.asciidoc 35KB
monitoring-apis.asciidoc 24KB
azure-module.asciidoc 24KB
transforming-data.asciidoc 20KB
persistent-queues.asciidoc 18KB
contributing-patch.asciidoc 16KB
arcsight-module.asciidoc 15KB
java-codec.asciidoc 13KB
settings-file.asciidoc 13KB
dead-letter-queues.asciidoc 13KB
getting-started-with-logstash.asciidoc 12KB
running-logstash-command-line.asciidoc 12KB
deploying.asciidoc 11KB
pipeline-pipeline-config.asciidoc 11KB
index.asciidoc 10KB
java-input.asciidoc 10KB
maintainer-guide.asciidoc 10KB
java-filter.asciidoc 10KB
performance-checklist.asciidoc 9KB
index.asciidoc 9KB
breaking-changes.asciidoc 9KB
java-output.asciidoc 9KB
logstash.asciidoc 9KB
running-logstash-windows.asciidoc 8KB
filter.asciidoc 8KB
api-keys.asciidoc 8KB
upgrading.asciidoc 8KB
logstash-glossary.asciidoc 8KB
docker.asciidoc 8KB
setting-up-logstash.asciidoc 7KB
ts-kafka.asciidoc 7KB
logging.asciidoc 7KB
filebeat-modules.asciidoc 7KB
modules.asciidoc 7KB
keystore.asciidoc 7KB
doc-for-plugin.asciidoc 7KB
input.asciidoc 7KB
netflow-module.asciidoc 7KB
monitoring-mb.asciidoc 6KB
java_line.asciidoc 6KB
glossary.asciidoc 6KB
java_plain.asciidoc 6KB
introduction.asciidoc 6KB
ts-logstash.asciidoc 6KB
plugin-manager.asciidoc 5KB
fb-ls-kafka-example.asciidoc 5KB
monitoring-internal-legacy.asciidoc 5KB
field-reference.asciidoc 5KB
monitoring-settings-legacy.asciidoc 5KB
centralized-pipelines.asciidoc 5KB
configuration-management-settings.asciidoc 5KB
event-api.asciidoc 4KB
shared-module-options.asciidoc 4KB
life-of-an-event.asciidoc 4KB
managing-multiline-events.asciidoc 4KB
best-practice.asciidoc 4KB
jvm.asciidoc 4KB
offline-plugins.asciidoc 4KB
output.asciidoc 4KB
shutdown.asciidoc 4KB
javapluginpkg.asciidoc 4KB
ecs-compatibility.asciidoc 4KB
java_generator.asciidoc 3KB
ls-ls-config.asciidoc 3KB
index.asciidoc 3KB
config-details.asciidoc 3KB
index.asciidoc 3KB
pipeline-viewer.asciidoc 3KB
ingest-convert.asciidoc 3KB
index.asciidoc 3KB
reloading-config.asciidoc 3KB
java_uuid.asciidoc 3KB
plugin-tracing.asciidoc 3KB
mem-queue.asciidoc 3KB
multiple-pipelines.asciidoc 2KB
collectors-legacy.asciidoc 2KB
index.asciidoc 2KB
submitting-a-plugin.asciidoc 2KB
ls-to-cloud.asciidoc 2KB
running-logstash.asciidoc 2KB
monitoring-output-legacy.asciidoc 2KB
javapluginsetup.asciidoc 2KB
private-gem-repo.asciidoc 2KB
processing-info.asciidoc 2KB
configuring-centralized-pipelines.asciidoc 2KB
gs-index.asciidoc 2KB
contributing-java-plugin.asciidoc 2KB
glob-support.asciidoc 2KB
contributing-to-logstash.asciidoc 2KB
reserved-fields.asciidoc 1KB
java_stdout.asciidoc 1KB
plugin_header.asciidoc 1KB
contrib-acceptance.asciidoc 1KB
redirects.asciidoc 1KB
plugin_header-integration.asciidoc 1KB
monitoring-overview.asciidoc 1KB
resiliency.asciidoc 1KB
共 2964 条
- 1
- 2
- 3
- 4
- 5
- 6
- 30
资源评论
野生的大熊
- 粉丝: 230
- 资源: 247
上传资源 快速赚钱
- 我的内容管理 展开
- 我的资源 快来上传第一个资源
- 我的收益 登录查看自己的收益
- 我的积分 登录查看自己的积分
- 我的C币 登录后查看C币余额
- 我的收藏
- 我的下载
- 下载帮助
最新资源
- 最全空间计量实证方法(空间杜宾模型和检验以及结果解释文档).txt
- 5uonly.apk
- 蓝桥杯Python组的历年真题
- 2023-04-06-项目笔记 - 第一百十九阶段 - 4.4.2.117全局变量的作用域-117 -2024.04.30
- 2023-04-06-项目笔记 - 第一百十九阶段 - 4.4.2.117全局变量的作用域-117 -2024.04.30
- 前端开发技术实验报告:内含4四实验&实验报告
- Highlight Plus v20.0.1
- 林周瑜-论文.docx
- 基于MIC+NE555光敏电阻的声光控电路Multisim仿真原理图
- 基于JSP毕业设计-基于WEB操作系统课程教学网站的设计与实现(源代码+论文).zip
资源上传下载、课程学习等过程中有任何疑问或建议,欢迎提出宝贵意见哦~我们会及时处理!
点击此处反馈
安全验证
文档复制为VIP权益,开通VIP直接复制
信息提交成功