Logstash 的搭建和安装

发表于 分类于 ， ， ， Valine：

Logstash 介绍

Logstash 是免费且开放的服务器端数据处理管道，能够从多个来源采集数据，转换数据，然后将数据发送到您最喜欢的 “存储库” 中。

  • 官网：https://www.elastic.co/cn/products/logstash
  • 下载：https://www.elastic.co/cn/downloads/logstash

Logstash 安装

$ wget https://artifacts.elastic.co/downloads/logstash/logstash-7.5.1.tar.gz

$ tar -xvf logstash-7.5.1.tar.gz

Logstash 测试

通过最基本的 Logstash 管道来测试一下刚才安装的 Logstash：

$ ./logstash -e 'input { stdin {} } output { stdout {} }'
hello world
Thread.exclusive is deprecated, use Thread::Mutex
Sending Logstash logs to /home/ant/logstash-7.5.1/logs which is now configured via log4j2.properties
[2021-12-20T17:03:31,600][WARN ][logstash.config.source.multilocal] Ignoring the 'pipelines.yml' file because modules or command line options are specified
[2021-12-20T17:03:31,837][INFO ][logstash.runner          ] Starting Logstash {"logstash.version"=>"7.5.1"}
[2021-12-20T17:03:35,048][INFO ][org.reflections.Reflections] Reflections took 60 ms to scan 1 urls, producing 20 keys and 40 values
[2021-12-20T17:03:38,149][WARN ][org.logstash.instrument.metrics.gauge.LazyDelegatingGauge][main] A gauge metric of an unknown type (org.jruby.RubyArray) has been create for key: cluster_uuids. This may result in invalid serialization.  It is recommended to log an issue to the responsible developer/development team.
[2021-12-20T17:03:38,178][INFO ][logstash.javapipeline    ][main] Starting pipeline {:pipeline_id=>"main", "pipeline.workers"=>40, "pipeline.batch.size"=>125, "pipeline.batch.delay"=>50, "pipeline.max_inflight"=>5000, "pipeline.sources"=>["config string"], :thread=>"#<Thread:0x3e417478 run>"}
[2021-12-20T17:03:38,373][INFO ][logstash.javapipeline    ][main] Pipeline started {"pipeline.id"=>"main"}
The stdin plugin is now waiting for input:
[2021-12-20T17:03:38,503][INFO ][logstash.agent           ] Pipelines running {:count=>1, :running_pipelines=>[:main], :non_running_pipelines=>[]}
[2021-12-20T17:03:38,976][INFO ][logstash.agent           ] Successfully started Logstash API endpoint {:port=>9600}
/home/ant/logstash-7.5.1/vendor/bundle/jruby/2.5.0/gems/awesome_print-1.7.0/lib/awesome_print/formatters/base_formatter.rb:31: warning: constant ::Fixnum is deprecated
{
       "message" => "hello world",
      "@version" => "1",
          "host" => "dm31.jd.163.org",
    "@timestamp" => 2021-12-20T09:03:38.481Z
}

启动 Logstash

在 config 文件夹下新增配置 first-pipeline.conf：

# Sample Logstash configuration for creating a simple
# Beats -> Logstash -> Elasticsearch pipeline.

input {
  beats {
    port => 5044
  }
}

output {
  elasticsearch {
    hosts => ["localhost:9200"]
    index => "logstash-%{[@metadata][beat]}-%{+YYYY.MM}"
    #user => "elastic"
    #password => "changeme"
  }
}

启动 Logstash：

$ ./bin/logstash -f config/first-pipeline.conf

--config.reload.automatic 选项的意思是启用自动配置加载，以至于每次你修改完配置文件以后无需停止然后重启 Logstash

$ ./bin/logstash -f config/first-pipeline.conf --config.reload.automatic &

--config.test_and_exit 选项的意思是解析配置文件并报告任何错误

$ ./bin/logstash -f config/first-pipeline.conf --config.test_and_exit &
相关文章