Logstash动态分配模板 [英] Logstash Dynamically assign template

问题描述

我已阅读到可以为这样的索引分配动态名称:

I have read that it is possible to assign dynamic names to the indexes like this:

elasticsearch {
            cluster => "logstash"
            index => "logstash-%{clientid}-%{+YYYY.MM.dd}"
    }

我想知道的是是否还可以动态分配模板:

What I am wondering is if it is possible to assign the template dynamically as well:

elasticsearch {
            cluster => "logstash"
            template => "/etc/logstash/conf.d/%{clientid}-template.json"
    }

变量％{clientid}也是哪里来的?

Also where does the variable %{clientid} come from?

谢谢！

推荐答案

完整披露:我是Elastic的Logstash开发人员

Full disclosure: I am a Logstash developer at Elastic

您无法动态分配模板，因为在Logstash初始化时模板仅上传一次.没有流量，就不会发生确定性的变量完成.由于初始化期间没有流量，因此没有任何东西可以为％{clientid} 填空.

You cannot dynamically assign a template because templates are uploaded only once, at Logstash initialization. Without the flow of traffic, deterministic variable completion does not happen. Since there is no traffic flow during initialization, there is nothing there which can "fill in the blank" for %{clientid}.

还要记住，仅当创建新索引时才使用Elasticsearch索引模板，因此每次文档到达Logstash中的Elasticsearch输出块时都不会上传模板，这很重要-您能想象多少?如果Logstash必须这样做，速度会变慢吗?如果您打算有多个模板，则需要先将它们上传到Elasticsearch，然后再将任何数据发送到那里.您可以使用自己编写的使用curl和Elasticsearch API调用的脚本来执行此操作.这也使您无需重新启动Logstash即可更新模板.您可以在索引过渡之前的任何时间运行脚本，并且在创建新索引时，它们将具有新的模板设置.

It is also important to remember that Elasticsearch index templates are only used when a new index is created, and so it is that templates are not uploaded every time a document reached the Elasticsearch output block in Logstash--can you imagine how much slower it would be if Logstash had to do that? If you intend to have multiple templates, they need to be uploaded to Elasticsearch before any data gets sent there. You can do this with a script of your own making using curl and Elasticsearch API calls. This also permits you to update templates without having to restart Logstash. You could run the script any time before index rollover, and when the new indices get created, they'll have the new template settings.

Logstash可以将数据发送到动态配置的索引名，就像上面一样.如果不存在模板，Elasticsearch将创建最佳猜测映射，而不是您想要的映射.模板可以而且应该完全独立于Logstash.添加此功能是为了为新用户提供更好的即用型体验.默认模板对于高级用例而言并不理想，如果您有多个索引模板，则Logstash并不是模板管理的好工具.

Logstash can send data to a dynamically configured index name, just as you have above. If there is no template present, Elasticsearch will create a best-guess mapping, rather than what you wanted. Templates can and ought to be completely independent of Logstash. This functionality was added for an improved out-of-the-box experience for brand new users. The default template is less than ideal for advanced use cases, and Logstash is not a good tool for template management if you have more than one index template.

这篇关于Logstash动态分配模板的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！