在Linux二进制grep的？ [英] Binary grep on Linux?

问题描述

说我有产生以下二进制文件：

Say I have generated the following binary file:

# generate file:
python -c 'import sys;[sys.stdout.write(chr(i)) for i in (0,0,0,0,2,4,6,8,0,1,3,0,5,20)]' > mydata.bin

# get file size in bytes
stat -c '%s' mydata.bin

# 14

和说，我想找到全零的位置（ 0×00 ），采用了类似grep的语法。

And say, I want to find the locations of all zeroes (0x00), using a grep-like syntax.

＆NBSP;

我目前可以做的最好的是：

The best I can do so far is:

$ hexdump -v -e "1/1 \" %02x\n\"" mydata.bin | grep -n '00'

1: 00
2: 00
3: 00
4: 00
9: 00
12: 00

然而，这隐含在原二进制文件中的每个字节到多字节ASCII重新presentation，在其上的grep 操作转换;不完全优化的最好的例子：）

However, this implicitly converts each byte in the original binary file into a multi-byte ASCII representation, on which grep operates; not exactly the prime example of optimization :)

有什么样的二进制的grep 为Linux？可能的话，还东西会支持常规的前pression的语法，也为字节人物 - 也就是说，我可以写的东西像 A（\\ X00 *）b '搭配'零个或多个事件字节之间的字节0的'A'（97）和b（98）？

Is there something like a binary grep for Linux? Possibly, also, something that would support a regular expression-like syntax, but also for byte "characters" - that is, I could write something like 'a(\x00*)b' and match 'zero or more' occurrences of byte 0 between bytes 'a' (97) and 'b' (98)?

编辑：上下文是我工作的一个司机，我在那里拍摄的8位数据;不顺心的事中的数据，可以是千字节到兆字节，我想检查特定的签名和它们发生在哪里。 （到目前为止，我和千字节的片段工作，所以优化不是那么重要 - 但如果我开始越来越兆字节长捕捉一些错误，我需要分析这些，我的猜测是，我想的东西更优化:)。特别是，我喜欢的东西，我可以的grep为一个字节作为字符 - hexdump都迫使我寻找每个字节串的）

The context is that I'm working on a driver, where I capture 8-bit data; something goes wrong in the data, which can be kilobytes up to megabytes, and I'd like to check for particular signatures and where they occur. (so far, I'm working with kilobyte snippets, so optimization is not that important - but if I start getting some errors in megabyte long captures, and I need to analyze those, my guess is I would like something more optimized :) . And especially, I'd like something where I can "grep" for a byte as a character - hexdump forces me to search strings per byte)

EDIT2：同一个问题，不同的论坛:) <一个href=\"http://www.linuxquestions.org/questions/linux-software-2/grepping-through-a-binary-file-for-a-sequence-of-bytes-717503/\">grepping通过对字节序列

same question, different forum :) grepping through a binary file for a sequence of bytes

EDIT3：多亏了@tchrist答案，在这里也有'grepping'和匹配，并显示结果的例子（虽然不大相同的问题作为OP 的）：

Thanks to the answer by @tchrist, here is also an example with 'grepping' and matching, and displaying results (although not quite the same question as OP):

$ perl -ln0777e 'print unpack("H*",$1), "\n", pos() while /(.....\0\0\0\xCC\0\0\0.....)/g' /path/to/myfile.bin

ca000000cb000000cc000000cd000000ce     # Matched data (hex)
66357                                  # Offset (dec)

有匹配的数据组合在一起，形成一个字节（两个十六进制字符）各话H2 H2 H2 ......需要进行匹配的字符串为尽可能多的字节为单位指定的有;因为我的比赛 ..... \\ 0 \\ 0 \\ 0 \\ XCC \\ 0 \\ 0 \\ 0 ..... 涵盖了17个字节，我可以写'在Perl H2X17 。每个这些H2的将返回一个单独的变量（如在一个列表），所以加入也需要用于在它们之间添加位 - 最终

To have the matched data be grouped as one byte (two hex characters) each, then "H2 H2 H2 ..." needs to be specified for as many bytes are there in the matched string; as my match '.....\0\0\0\xCC\0\0\0.....' covers 17 bytes, I can write '"H2"x17' in Perl. Each of these "H2" will return a separate variable (as in a list), so join also needs to be used to add spaces between them - eventually:

$ perl -ln0777e 'print join(" ", unpack("H2 "x17,$1)), "\n", pos() while /(.....\0\0\0\xCC\0\0\0.....)/g' /path/to/myfile.bin

ca 00 00 00 cb 00 00 00 cc 00 00 00 cd 00 00 00 ce
66357

嗯..的确Perl是很不错的二元grepping设施，我必须一个人只要正确地学习语法承认:)：）

Well.. indeed Perl is very nice 'binary grepping' facility, I must admit :) As long as one learns the syntax properly :)

推荐答案

下面是较短的单行版本：

One-Liner Input

Here’s the shorter one-liner version:

% perl -ln0e 'print tell' < inputfile

而这里的一个稍微长一行代码：

And here's a slightly longer one-liner:

% perl -e '($/,$\) = ("\0","\n"); print tell while <STDIN>' < inputfile

连接这两个单行的方法是uncompiling第一位的程序：

The way to connect those two one-liners is by uncompiling the first one’s program:

% perl -MO=Deparse,-p -ln0e 'print tell'
BEGIN { $/ = "\000"; $\ = "\n"; }
LINE: while (defined(($_ = <ARGV>))) {
    chomp($_);
    print(tell);
}

编程输入

如果你想要把在一个文件，而不是通过命令行调用它，这里有一个较为明确的版本：

Programmed Input

If you want to put that in a file instead of a calling it from the command line, here’s a somewhat more explicit version:

#!/usr/bin/env perl

use English qw[ -no_match_vars ];

$RS  = "\0";    # input  separator for readline, chomp
$ORS = "\n";    # output separator for print

while (<STDIN>) {
    print tell();
}

和这里的很长的版本：

#!/usr/bin/env perl

use strict;
use autodie;  # for perl5.10 or better
use warnings qw[ FATAL all  ];

use IO::Handle;

IO::Handle->input_record_separator("\0");
IO::Handle->output_record_separator("\n");

binmode(STDIN);   # just in case

while (my $null_terminated = readline(STDIN)) {
    # this just *past* the null we just read:
    my $seek_offset = tell(STDIN);
    print STDOUT $seek_offset;  

}

close(STDIN);
close(STDOUT);

单行输出

BTW，创建测试输入文件，我没有用你的大，长Python脚本;我只是用这个简单的Perl一行代码：

One-Liner Output

BTW, to create the test input file, I didn’t use your big, long Python script; I just used this simple Perl one-liner:

% perl -e 'print 0.0.0.0.2.4.6.8.0.1.3.0.5.20' > inputfile

您会发现往往Perl的卷起是2-3倍短于Python中做同样的工作。你不必对清晰度妥协;有什么比这更简单，上面的单行？

You’ll find that Perl often winds up being 2-3 times shorter than Python to do the same job. And you don’t have to compromise on clarity; what could be simpler that the one-liner above?

我知道，我知道。如果你还不知道的语言，这可能是更清晰的：

I know, I know. If you don’t already know the language, this might be clearer:

#!/usr/bin/env perl
@values = (
    0,  0,  0,  0,  2,
    4,  6,  8,  0,  1,
    3,  0,  5, 20,
);
print pack("C*", @values);

虽然这个工程，也：

although this works, too:

print chr for @values;

一样

print map { chr } @values;

虽然对于那些谁喜欢一切都严谨细致和所有，这可能是更多，你会看到什么：

Although for those who like everything all rigorous and careful and all, this might be more what you would see:

#!/usr/bin/env perl

use strict;
use warnings qw[ FATAL all ];
use autodie;

binmode(STDOUT);

my @octet_list = (
    0,  0,  0,  0,  2,
    4,  6,  8,  0,  1,
    3,  0,  5, 20,
);

my $binary = pack("C*", @octet_list);
print STDOUT $binary;

close(STDOUT); 

TMTOWTDI

Perl的支持做的事情，这样你可以挑选你最舒服的一个方法不止一种。如果这是我的东西计划检查作为学校或工作项目，我肯定会选择更长，更仔细的版本 - 或者至少把评论的shell脚本，如果我用了单行

TMTOWTDI

Perl supports more than one way to do things so that you can pick the one that you’re most comfortable with. If this were something I planned to check in as school or work project, I would certainly select the longer, more careful versions — or at least put a comment in the shell script if I were using the one-liners.

您可以找到你自己的系统上Perl文档。只需键入

You can find documentation for Perl on your own system. Just type

% man perl
% man perlrun
% man perlvar
% man perlfunc

等在shell提示符下。如果你想在网络上pretty十岁上下的版本代替，得到 perl的，的perlrun ， perlvar 和<一个HREF =htt​​p://perldoc.perl.org/index-functions.html> perlfunc 从 HTTP：//perldoc.perl。组织。

这篇关于在Linux二进制grep的？的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！