为什么输输事件有声呢? [英] Why is inotify losing events?

问题描述

我需要使用Perl和 Linux:处理大量(约100秒钟)系统日志消息:Inotify2 .

I need to process large (~100s) of syslog messages using Perl and Linux::Inotify2.

我编写了一个测试脚本，该脚本连续生成日志消息.为了处理事件，我的Perl脚本看起来像这样-

I wrote a test script which generates log messages continuously. To process events, my Perl script looks like this-

#!/usr/bin/perl
use Linux::Inotify2 ;
use Time::HiRes qw(usleep nanosleep);
# create a new object
 my $inotify = new Linux::Inotify2
    or die "Unable to create new inotify object: $!" ;

 # create watch
 $inotify->watch ("/var/log/messages",  IN_ACCESS|IN_OPEN|IN_CLOSE|IN_MODIFY|IN_Q_OVERFLOW)
    or die "watch creation failed" ;
my $count=0;
 while () {
   my @events = $inotify->read;
   unless (@events > 0) {
     print "read error: $!";
     last ;
   }
   #printf "mask\t%d\n", $_->mask foreach @events ; 
   $count++;
   print $count."\n";
   usleep(100000);
 }

如果我取消注释usleep函数以模拟处理，我会注意到当我停止日志生成器脚本时，inotify脚本无法跟上它的步伐.换句话说，inotify Perl脚本丢失了事件.

If I un-comment usleep function to simulate processing, I notice that when I stop the log generator script, the inotify script doesn't catch up with it. In other words, the inotify Perl script is losing events.

我也看不到任何溢出"消息.

Neither do I see any Overflow message.

如何确保即使处理缓慢也不会丢失消息.换句话说，我该如何定义一个缓冲区"来临时存储消息?

How do I make sure that even if my processing is slow, I don't lose messages. In other words, how do I define a "buffer" where messages can be stored temporarily?

推荐答案

即使它不能完全回答您的问题，我也会尝试解决您的根本问题.

I'm going to try to address your root problem, even though it doesn't exactly answer your question.

我认为使用inotify太底层了，无法解决这个问题.如果您是我，那么我正在寻找一种更高级别的解决方案，该解决方案将使您从查找新事件的细节中解放出来，并使您专注于处理日志事件.当已经有了很多好玩的东西时，无需重新发明轮子.

I think using inotify is too low-level a way of resolving this. If I were you, I'd be looking at a higher-level solution that frees you from the details of finding new events, and lets you concentrate on processing the log events. No need to reinvent the wheel when there's a bunch of good ones already available.

我的首选是使用现代的syslog守护程序(个人而言，我更喜欢Syslog-NG，但rsyslog也可以很好地工作)，并将其直接绑定到脚本中.您的脚本只处理 stdin ，而不是脚本来跟踪新事件何时进入的所有工作，并且syslog守护程序会在收到新日志时自动将新日志发送到您的脚本.这种方法在很多场合都成功了.

My first choice would be to use a modern syslog daemon (personally I prefer Syslog-NG, but rsyslog would work well enough, too), and tie that directly to your script. Instead of your script going through all the work of tracking when new events come in, your script just processes stdin, and the syslog daemon automatically sends new logs to your script as they come in. I've used this method successfully on numerous occasions.

我的第二个选择是让更高级别的Perl模块承担尽可能多的繁重任务.我要寻找的第一个地方是 File :: Tail .像以前一样，这使您摆脱了查看文件本身的麻烦，而使您可以专注于处理过程.

My second choice would be to let a higher-level Perl module do as much of the heavy lifting as possible. The first place I'd look for that would be File::Tail. Like before, this frees you from the problems of watching the file itself, and lets you concentrate on the processing.

这篇关于为什么输输事件有声呢?的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！