如何使用bash工具搜索非ASCII字符? [英] How to search for non-ASCII characters with bash tools?

问题描述

我有一个很大的文本文件，其中包含一些会使LaTeX崩溃的unicode字符.如何在带有sed的文件中以及Linux bash中的类似文件中找到非ASCII字符?

I have a large text file that contains a few unicode characters that make LaTeX crash. How can I find non-ASCII characters in a file with sed, and the like in a Linux bash?

推荐答案

尝试:

nonascii() { LANG=C grep --color=always '[^ -~]\+'; }

可以这样使用:

printf 'ŨTF8\n' | nonascii

在[]之内^表示不是".因此[^ -~]表示字符不在空格和〜之间.因此，除了控制字符外，它与非ASCII字符匹配，并且在以下[^\x00-\x7f]的准确度稍低的情况下，其移植性更高. \+表示1 or more，并且将使再见字符在整个字符周围显示颜色，而不是散布在每个字节中，从而破坏了多字节序列

Within [] ^ means "not". So [^ -~] means characters not between space and ~. So excluding control chars, this matches non ASCII characters, and is a more portable though slightly less accurate version of [^\x00-\x7f] below. The \+ means 1 or more and will get multibye characters to have a color shown around the complete character(s), rather than interspersed in each byte, thus corrupting the multibyte sequence

这篇关于如何使用bash工具搜索非ASCII字符?的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！