跟踪日志文件并将结果写入新文件 [英] Tailing Log File and Write results to new file
问题描述
我不确定该如何措辞，所以我将其键入，然后编辑并回答出现的任何问题.
I'm not sure how to word this so I'll type it out and then edit and answer any questions that come up..
当前在我的本地网络设备(基于PHP4)上，使用它跟踪实时系统日志文件:
Currently on my local network device (PHP4 based) I'm using this to tail a live system log file: http://commavee.com/2007/04/13/ajax-logfile-tailer-viewer/
这很好用，并且每1秒钟加载一个执行tail -n 100 logfile.log的外部页面(logfile.php).该脚本不进行任何缓冲，因此它在屏幕上显示的结果是日志文件的最后100行.
This works well and every 1 second it loads an external page (logfile.php) that does a tail -n 100 logfile.log The script doesn't do any buffering so the results it displayes onscreen are the last 100 lines from the log file.
logfile.php包含:
The logfile.php contains :
<? // logtail.php $cmd = "tail -10 /path/to/your/logs/some.log"; exec("$cmd 2>&1", $output);
foreach($output as $outputline) {
echo ("$outputline\n");
}
?>
这部分工作正常.
我已经修改了logfile.php页面，只需使用fwrite($fp,$outputline."\n");
I have adapted the logfile.php page to write the $outputline to a new text file, simply using fwrite($fp,$outputline."\n");
虽然这行得通，但是在创建的新文件中出现重复问题.
Whilst this works I am having issues with duplication in the new file that is created.
很明显，每次运行tail -n 100都会产生结果，下次运行它可能会产生一些相同的行，因为重复一遍，我最终会在新文本文件中出现多行重复.
Obviously each time tail -n 100 is run produces results, the next time it runs it could produce some of the same lines, as this repeats I can end up with multiple lines of duplication in the new text file.
由于可能存在相同的匹配项，因此我无法直接将要写的行与以前的行进行比较.
I can't directly compare the line I'm about to write to previous lines as there could be identical matches.
有什么办法可以将当前的100行代码块与前一个代码块进行比较，然后只写不匹配的行. B将包含所需的相同行...
Is there any way I can compare this current block of 100 lines with the previous block and then only write the lines that are not matching.. Again possible issue that block A & B will contain identical lines that are needed...
是否可以更新logfile.php以记下它在我的日志文件中的最后位置，然后仅从那里读取接下来的100行并将其写入新文件?
Is it possible to update logfile.php to note the position it last tooked at in my logfile and then only read the next 100 lines from there and write those to the new file ?
该日志文件最大为500MB，所以我不想每次都读取它..
The log file could be upto 500MB so I don't want to read it all in each time..
欢迎任何建议.
谢谢
更新@ 16:30
我已经使用:
$file = "/logs/syst.log";
$handle = fopen($file, "r");
if(isset($_SESSION['ftell'])) {
clearstatcache();
fseek($handle, $_SESSION['ftell']);
while ($buffer = fgets($handle)) {
echo $buffer."<br/>";
@ob_flush(); @flush();
}
fclose($handle);
@$_SESSION['ftell'] = ftell($handle);
} else {
fseek($handle, -1024, SEEK_END);
fclose($handle);
@$_SESSION['ftell'] = ftell($handle);
}
这似乎可行，但它会先加载整个文件，然后再加载更新.
This seems to work, but it loads the entire file first and then just the updates.
我如何从最后50行开始，然后只是更新?
How would I get it start with the last 50 lines and then just the updates ?
谢谢:)
更新2013年6月6日 尽管这行得通，但是处理大文件的速度非常慢.
UPDATE 04/06/2013 Whilst this works it's very slow with large files.
我尝试了这段代码，它看起来更快，但它不仅是从中断处读取的.
I've tried this code and it seems faster, but it doesn't just read from where it left off.
function last_lines($path, $line_count, $block_size = 512){
$lines = array();
// we will always have a fragment of a non-complete line
// keep this in here till we have our next entire line.
$leftover = "";
$fh = fopen($path, 'r');
// go to the end of the file
fseek($fh, 0, SEEK_END);
do{
// need to know whether we can actually go back
// $block_size bytes
$can_read = $block_size;
if(ftell($fh) < $block_size){
$can_read = ftell($fh);
}
// go back as many bytes as we can
// read them to $data and then move the file pointer
// back to where we were.
fseek($fh, -$can_read, SEEK_CUR);
$data = fread($fh, $can_read);
$data .= $leftover;
fseek($fh, -$can_read, SEEK_CUR);
// split lines by \n. Then reverse them,
// now the last line is most likely not a complete
// line which is why we do not directly add it, but
// append it to the data read the next time.
$split_data = array_reverse(explode("\n", $data));
$new_lines = array_slice($split_data, 0, -1);
$lines = array_merge($lines, $new_lines);
$leftover = $split_data[count($split_data) - 1];
}
while(count($lines) < $line_count && ftell($fh) != 0);
if(ftell($fh) == 0){
$lines[] = $leftover;
}
fclose($fh);
// Usually, we will read too many lines, correct that here.
return array_slice($lines, 0, $line_count);
}
可以通过任何方式对此进行修改，以便从最近的已知位置读取..
Any way this can be amend so it will read from the last known position.. ?
谢谢
推荐答案
简介
您可以通过跟踪最后一个位置来尾部文件；
You can tail a file by tracking the last position;
示例
$file = __DIR__ . "/a.log";
$tail = new TailLog($file);
$data = $tail->tail(100) ;
// Save $data to new file
TailLog是我为此任务编写的一个简单类，这里是一个简单的示例，用于显示其实际尾随文件
TailLog is a simple class i wrote for this task here is a simple example to show its actually tailing the file
简单测试
$file = __DIR__ . "/a.log";
$tail = new TailLog($file);
// Some Random Data
$data = array_chunk(range("a", "z"), 3);
// Write Log
file_put_contents($file, implode("\n", array_shift($data)));
// First Tail (2) Run
print_r($tail->tail(2));
// Run Tail (2) Again
print_r($tail->tail(2));
// Write Another data to Log
file_put_contents($file, "\n" . implode("\n", array_shift($data)), FILE_APPEND);
// Call Tail Again after writing Data
print_r($tail->tail(2));
// See the full content
print_r(file_get_contents($file));
输出
// First Tail (2) Run
Array
(
[0] => c
[1] => b
)
// Run Tail (2) Again
Array
(
)
// Call Tail Again after writing Data
Array
(
[0] => f
[1] => e
)
// See the full content
a
b
c
d
e
f
实时拖尾
while(true) {
$data = $tail->tail(100);
// write data to another file
sleep(5);
}
注意:拖尾100行并不意味着它将总是返回100行.它会返回新行，而添加的100就是要返回的最大行数.如果您有
Note: Tailing 100 lines does not mean it would always return 100 lines. It would return new lines added 100 is just the maximum number of lines to return. This might not be efficient where you have heavy logging of more than 100 line per sec is there is any
尾巴类
class TailLog {
private $file;
private $data;
private $timeout = 5;
private $lock;
function __construct($file) {
$this->file = $file;
$this->lock = new TailLock($file);
}
public function tail($lines) {
$pos = - 2;
$t = $lines;
$fp = fopen($this->file, "r");
$break = false;
$line = "";
$text = array();
while($t > 0) {
$c = "";
// Seach for End of line
while($c != "\n" && $c != PHP_EOL) {
if (fseek($fp, $pos, SEEK_END) == - 1) {
$break = true;
break;
}
if (ftell($fp) < $this->lock->getPosition()) {
break;
}
$c = fgetc($fp);
$pos --;
}
if (ftell($fp) < $this->lock->getPosition()) {
break;
}
$t --;
$break && rewind($fp);
$text[$lines - $t - 1] = fgets($fp);
if ($break) {
break;
}
}
// Move to end
fseek($fp, 0, SEEK_END);
// Save Position
$this->lock->save(ftell($fp));
// Close File
fclose($fp);
return array_map("trim", $text);
}
}
尾锁
class TailLock {
private $file;
private $lock;
private $data;
function __construct($file) {
$this->file = $file;
$this->lock = $file . ".tail";
touch($this->lock);
if (! is_file($this->lock))
throw new Exception("can't Create Lock File");
$this->data = json_decode(file_get_contents($this->lock));
// Check if file is valida json
// Check if Data in the original files as not be delete
// You expect data to increate not decrease
if (! $this->data || $this->data->size > filesize($this->file)) {
$this->reset($file);
}
}
function getPosition() {
return $this->data->position;
}
function reset() {
$this->data = new stdClass();
$this->data->size = filesize($this->file);
$this->data->modification = filemtime($this->file);
$this->data->position = 0;
$this->update();
}
function save($pos) {
$this->data = new stdClass();
$this->data->size = filesize($this->file);
$this->data->modification = filemtime($this->file);
$this->data->position = $pos;
$this->update();
}
function update() {
return file_put_contents($this->lock, json_encode($this->data, 128));
}
}
这篇关于跟踪日志文件并将结果写入新文件的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！
