无法过滤syslog-ng中格式错误的消息 [英] unable to filter badly-formatted messages in syslog-ng

问题描述

我正在就syslog-NG的问题与您联系. 我们的某些设备(主要是HP交换机和SAN)发送的系统日志消息似乎不符合系统日志RFC 5424.

I am contacting you regarding an issue I have with syslog-NG. some of our devices (mainly HP switches and SANs) are sending syslog messages that do not respect the syslog RFC 5424 it seems.

给你一些例子:

如果我嗅探服务器的网络接口，我们可以看到类似这样的错误"消息(如您​​所见，在PRI之后，我们可以看到PROGRAM，然后是时区和以逗号分隔的字段，简而言之，混合字段，缺少字段等.不是标准的syslog消息):

if I sniff the network interface of the server, we can see these "wrong" messages like that (as you can see, after the PRI, we can see the PROGRAM, then timezone and fields separated with commas. in short, mixed fields, missing fields etc. not a standard syslog message) :

<190>raslogd: 2017/03/08-16:03:20, [SEC-1203], 53642, WWN 10:00:50:eb:1a:6c:21:38 | FID 128, INFO, cswc-mo8x-SAN01, Login information: Login successful via TELNET/SSH/RSH. IP Addr: 1.2.3.4

因此，我目前无法过滤这些消息. 我试图定义一个常规过滤器，并尝试打印MSGHDR，MSG，PRI等字段.但是什么也看不到.

therefore, I am currently unable to filter these messages. I tried to define a regular filter and try to print fields such as MSGHDR, MSG, PRI etc.. but couldn't see anything.

仅通过定义外部接口和文件作为输出而没有任何模板来过滤此消息的唯一方法.

the ONLY WAY of having this message filtered is by just defining the external interface, and a file as output, without any template.

例如，如果我做这样的过滤器:

e.g if I do a filter like this :

destination d_INCOMING_ALL   { file("/app/syslog-ng/logs/incoming_all.log"); };

log {
  source(s_EXTERNAL);
  destination(d_INCOMING_ALL);
};

我可以在日志文件中看到消息，但是消息是以某种方式格式化的(我想用syslog-NG重新格式化它们):

I can see messages in the log file, but formatted, somehow (I suppose syslog-NG reformats them) :

[root@xxxxxxxxxxxx logs]# grep -i cswc incoming_all.log
Mar  9 09:44:20 cswc-mo8x-hpsan01 raslogd: 2017/03/09-08:34:50, [SEC-1203], 53647, WWN 10:00:50:eb:1a:6c:21:38 | FID 128, INFO, cswc-mo8x-SAN01, Login information: Login successful via TELNET/SSH/RSH. IP Addr: 1.2.3.4
[root@xm1p1034vmo logs]#

问题是我无法像这样过滤这些消息(我们从1000多个设备中接收日志) 在那里，我需要过滤来自这些设备的消息！ 我唯一的方法是在主机名(cswc- )或程序名(raslogd )

the problem is that I cannot filter these messages like that (we receive logs from more than 1000 devices) there, i need to filter messages coming from these devices ! and the only way I can do it is on the hostname (cswc-) or program name (raslogd)

所以我尝试通过向该文件添加模板来显示字段，例如:

so I tried to display the fields by adding a template to that file, example :

destination d_test { 
    file ("/app/syslog-ng/logs/test_olivier.log" 
    template("pri=${priority} date=${ISODATE} host=${HOST} program=${PROGRAM} message=${MSG}\n") 
    ); 
};

但是没有任何效果，什么也没有显示. 我尝试了所有字段，MSG，MESSAGE，MSGHDR等.无法显示任何内容. 唯一的工作是没有过滤器或模板的解析.

but nothing works, nothing is displayed. I tried all fields, MSG, MESSAGE, MSGHDR etc.. can't manage do display ANYTHING. the only working thing is the parsing without filters or templates.

自然地，如果我尝试了各种过滤器(如下所示)，则该过滤器将不起作用(由于字段混合):

naturally, if I tred all kind of filters, like these below, it does not work (as fields are mixed) :

filter f_is_SAN     {
    host("cswc.*" flags(ignore-case));
};

相同于:

filter f_is_SAN     {
    match(".*cswc.*" flags(ignore-case));
};

关于如何为来自这些设备(主机名或程序名)的这些消息创建过滤器的任何提示?

any hints on how I create filters for these messages coming from these devices (on hostname or programname) ?

谢谢 问候，

推荐答案

mmh，谢谢罗伯特，我尝试过像这样的单个路口，而且行得通！ (即使没有路口也能正常工作) 不幸的是，我的其他过滤器不再工作了(我在这里只显示1个，但是我有20个以上，原始文件更大)

mmh thanks Robert, i tried with a single junction like this, and it works ! (it works even without the junction) unfortunately, my other filters are not working anymore (I'm just showing 1 here, but i have more than 20, original file is much bigger)

我们不能使用相同的端口但使用不同的标志侦听两个源吗? :(

can't we listen on two sources using the same port, but with differents flags ? :(

source s_EXTERNAL {
    udp();
};

source s_EXTERNAL_NOPARSE {
    udp(flags(no-parse) persist-name('noparse'));
};

destination d_INCOMING_ALL { file("/app/syslog-ng/logs/incoming_all.log"); };
destination d_OUTGOING_ISERIES   { file("/app/syslog-ng/logs/outgoing_iseries.log"); };
destination d_olivier2 { file ("/app/syslog-ng/logs/test_olivier2.log" template("host=${HOST} message=${MESSAGE}\n") ); };

filter f_is_iSeries {
  match ("PowerTech.*Interact" value("MESSAGE") flags(ignore-case));
};

filter f_is_Network     {
  host("cswc.*" flags(ignore-case))
  or program("raslogd" flags(ignore-case));
};

# -------- not working anymore
log {
 source(s_EXTERNAL);
 filter(f_is_iSeries);
 destination(d_OUTGOING_ISERIES);
 flags(final);
};

# ------- working now
log {
 source(s_EXTERNAL_NOPARSE);
 junction {
    channel {
            filter(f_is_Network);
    };
 };
 destination(d_olivier2);
};

我现在可以在测试d_olivier2 dest中看到此日志:

i can now see this log in the test d_olivier2 dest :

[root@xm3p1034vmo etc]# cat ../logs/test_olivier2.log
host=cswc-mo8x-hpsan01 message=<190>raslogd: 2017/03/23-16:29:44, [SEC-1203], 53656, WWN 10:00:50:eb:1a:6c:21:38 | FID 128, INFO, cswc-mo8x-SAN01, Login information: Login successful via TELNET/SSH/RSH. IP Addr: 1.2.3.4

，但/app/syslog-ng/logs/outgoing_iseries.log 中没有任何内容:(

but nothing in /app/syslog-ng/logs/outgoing_iseries.log :(

这篇关于无法过滤syslog-ng中格式错误的消息的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！