为cloudwatchLogs中的每个日志文件创建一个logStream [英] Create a logStream for each log file in cloudwatchLogs
问题描述
我使用 AWS CloudWatch 日志代理将应用程序日志推送到AWS Cloudwatch。
I use AWS CloudWatch log agent to push my application log to AWS Cloudwatch.
在<$我的 EC2实例中的c $ c> cloudwatchLogs 配置文件，我有以下条目：
In the cloudwatchLogs config file inside my EC2 instance, I have this entry:
[/scripts/application]
datetime_format = %Y-%m-%d %H:%M:%S
file = /workingdir/customer/logfiles/*.log
buffer_duration = 5000
log_stream_name = {instance_id}
initial_position = start_of_file
log_group_name = /scripts/application
根据此配置，workingdir目录中的所有日志文件都被发送到cloudwatchLogs，同一流中的名称是实例ID。
According to this configuration, all log files in workingdir directory are being sent to cloudwatchLogs in the same stream were the name is the instance Id.
我的问题是，我要为每个日志文件创建一个单独的 logStream ，以便日志读取可以更快，更容易解析。换句话说，每次我有一个新的日志文件时，都会自动创建一个新的日志流。
My question is, I want for each log file, create a separate logStream, so that the logs reading can be more fast and parseable. In other words, every time I have a new log file, a new logstream is created automatically.
我想到了通过shell进行操作脚本中执行cron作业，但是随后我必须更改架构中的许多其他配置，因此我正在寻找一种在config文件中执行此操作的方法。在文档中，他们说：
I thought of doing that by a shell script in a cron job but then I'll have to change many other configurations in the architecture, so I'm looking for a way to do it in the config file. In the documentation, they say that :
log_stream_name
log_stream_name
指定目标日志流。您可以使用文字字符串或
预定义变量（{instance_id}，{hostname}，{ip_address}）或
两者的组合来定义日志流名称。如果日志流尚不存在，则会自动创建
。
Specifies the destination log stream. You can use a literal string or predefined variables ({instance_id}, {hostname}, {ip_address}), or combination of both to define a log stream name. A log stream is created automatically if it doesn't already exist.
日志文件的名称不能是100％可预测的，但是它们始终具有以下结构：
The names of the log files can't be 100% predictible, but they always have this structure though:
CustomerName-YYYY-mm-dd.log
另外，另一个问题是：
必须停止并重新启动正在运行的代理，配置
的更改才能生效。
A running agent must be stopped and restarted for configuration changes to take effect.
如何设置
任何想法，建议或解决方法都非常感谢。
Any ideas or suggestions or workarounds are very appreciated.
推荐答案
我知道现在已经快两年了，但是我想做同样的事情，并且找不到使它工作的方法。
我求助于AWS支持，然后确认无法完成。就像您发布的一样，我们仅限于文档中提供的选项。但是，您可以让日志组包含直到第一个点的日志文件路径：
I know it's been almost two years now, but I wanted to do the exact same thing and couldn't find a way to get it to work. I resorted to the AWS Support, which then confirmed this cannot be done. We're limited to the options offered in the documentation, just like you posted. You can, however, have log groups contain the log file path up to the first dot:
log_group_name –可选的。指定在CloudWatch Logs中用作日志组名称
的内容。允许的字符包括az，AZ，0-9，'_'
（下划线），'-'（连字符），'/'（正斜杠）和'。'（句点）。
log_group_name – Optional. Specifies what to use as the log group name in CloudWatch Logs. Allowed characters include a-z, A-Z, 0-9, '_' (underscore), '-' (hyphen), '/' (forward slash), and '.' (period).
我们建议您指定此字段以防止混淆。 如果您
忽略此字段，则将最后一个点之前的文件路径用作日志
组名。例如，如果文件路径为
/tmp/TestLogFile.log.2017-07-11-14，则日志组名称为
/tmp/TestLogFile.log。
We recommend that you specify this field to prevent confusion. If you omit this field, the file path up to the final dot is used as the log group name. For example, if the file path is /tmp/TestLogFile.log.2017-07-11-14, the log group name is /tmp/TestLogFile.log.
这篇关于为cloudwatchLogs中的每个日志文件创建一个logStream的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！
