Logstash Elasticsearch输入-仅运行一次? [英] Logstash Elasticsearch Input - runs once only?

问题描述

我正在尝试将一小部分数据从索引A复制到索引B.

I'm trying to copy a small subset of data from index A into index B.

我有一个在EC2实例上运行的Logstash配置，其中包含Elasticsearch输入，一些过滤器和Elasticsearch输出.

I have a Logstash config running on an EC2 instance with the Elasticsearch input, some filters and the Elasticsearch output.

这是我第一次运行时的正常工作，但是如果将任何新数据添加到索引A(满足输入的查询条件)，则Logstash配置不会检测到它并将其输出到索引B.

This works fine the first time I run it, but if any new data is added to Index A (that meets the input's query criteria) the Logstash config doesn't detect it and output it to Index B.

这让我想，Elasticsearch输入是否仅运行一次?与文件或S3输入之类的输入不同，它们会获取添加的新文件.

This has me thinking, does the Elasticsearch input only run once? Unlike inputs like the file or S3 input that will pick up new files added.

推荐答案

Logstash elasticsearch输入插件使用可选查询进行一次搜索(通过扫描/滚动)，并在完成后退出.

The Logstash elasticsearch input plugin does a single search (via scan/scroll) with an optional query and when done it quits.

您是对的，它的行为与file输入插件的行为不同，后者只会拖尾一个或多个文件并动态处理新行.

You're right in that it doesn't behave the same way as the for file input plugin, which will simply tail one or more files and handle the new lines on the fly.

如果您想使用 Watcher 插件，可以进行调查当新文档与查询匹配时，需要一种通知方式.观察者可以很好地检测到与查询匹配的新文档的存在，然后调用一个Webhook，可以通过

You might want to investigate the Watcher plugin if you need a way to be notified when a new document matches a query. Watcher could well detect the presence of a new document matching a query and then call a webhook, which could be materialized by the http input plugin in your Logstash.

这篇关于Logstash Elasticsearch输入-仅运行一次?的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！