管道文本到外部程序附加一个尾随换行符 [英] Piping Text To An External Program Appends A Trailing Newline

问题描述

我一直在比较多个系统之间的哈希值，并惊讶地发现 PowerShell 的哈希值与其他终端的不同.

I have been comparing hash values between multiple systems and was surprised to find that PowerShells hash values are different than that of other terminals.

Linux 终端(CygWin、Bash for Windows 等)和 Windows 命令提示符都显示相同的哈希值，而 PowerShell 显示不同的哈希值.

Linux terminals (CygWin, Bash for Windows, etc.) and Windows Command Prompt are all showing the same hash where as PowerShell is showing a different hash value.

这是使用 SHA256 测试的，但在使用其他算法(如 md5)时发现了相同的问题.

This was tested using SHA256 but found the same issue when using other algorithms like md5.

编码更新:

尝试更改 PShell 编码，但对返回的哈希值没有任何影响.

Tried changing the PShell encoding but it did not have any effect on the returned hash values.

[Console]::OutputEncoding.BodyName 
iso-8859-1
[Console]::OutputEncoding = [Text.UTF8Encoding]::UTF8
utf-8

GitHub PowerShell 问题

https://github.com/PowerShell/PowerShell/issues/5974

推荐答案

tl;dr:

当 PowerShell 将字符串通过管道传输到外部程序时:

When PowerShell pipes a string to an external program:

• 它使用存储在 $OutputEncoding 偏好变量
• 它总是附加一个尾随(适合平台的)换行符.

• It encodes it using the character encoding stored in the $OutputEncoding preference variable
• It invariably appends a trailing (platform-appropriate) newline.

因此，关键是避免PowerShell的管道而支持本机shell的管道，从而防止隐式添加尾随换行符:

• 如果您在类似 Unix 的平台上运行命令(使用 PowerShell Core):

sh -c "printf %s 'string' | openssl dgst -sha256 -hmac authcode"

^{printf %s 是 echo -n 的便携替代方案.如果字符串包含 ' 个字符，则将它们加倍或使用 `"...`" 引用代替.}

^{printf %s is the portable alternative to echo -n. If the string contains ' chars., double them or use `"...`" quoting instead.}

• 如果您需要在 Windows 上通过 cmd.exe 执行此操作，事情会变得更加棘手，因为 cmd.exe 不直接支持没有尾随换行符的回显:

• In case you need to do this on Windows via cmd.exe, things get even trickier, because cmd.exe doesn't directly support echoing without a trailing newline:

cmd /c "<NUL set /p =`"string`"| openssl dgst -sha256 -hmac authcode"

^{请注意，必须在之前没有空格 | 才能使其工作.有关此解决方案的说明和局限性，请参阅此答案.}

^{Note that there must be no space before | for this to work. For an explanation and the limitations of this solution, see this answer.}

仅当字符串包含非 ASCII 字符并且您在 Windows PowerShell 中运行时才会出现编码问题；在这种情况下，首先将 $OutputEncoding 设置为目标实用程序期望的编码，通常是 UTF-8:$OutputEncoding = [Text.Utf8Encoding]::new()

Encoding issues would only arise if the string contained non-ASCII characters and you're running in Windows PowerShell; in that event, first set $OutputEncoding to the encoding that the target utility expects, typically UTF-8: $OutputEncoding = [Text.Utf8Encoding]::new()

• PowerShell，从 Windows PowerShell v5.1/PowerShell (Core) v7.2 开始，总是在您发送一个尾随换行符没有字符串的字符串通过管道连接到外部实用程序，这就是您观察到的差异的原因(尾随换行符仅在 Unix 平台上是 LF，而 CRLF 序列在 Windows 上).

• PowerShell, as of Windows PowerShell v5.1 / PowerShell (Core) v7.2, invariably appends a trailing newline when you send a string without one via the pipeline to an external utility, which is the reason for the difference you're observing (that trailing newline will be a LF only on Unix platforms, and a CRLF sequence on Windows).

• 您可以在 GitHub 问题 #5974 中跟踪解决此问题的努力，由 OP 打开.

• You can keep track of efforts to address this problem in GitHub issue #5974, opened by the OP.

此外，在将数据传送到外部程序时，PowerShell的管道总是基于文本；内部基于 UTF-16LE 的 PowerShell (.NET) 字符串根据存储在自动 $OutputEncoding 变量中的编码进行转码，该变量在 Windows 中默认为仅 ASCII 编码PowerShell，以及 PowerShell Core 中的 UTF-8 编码(在 Windows 和类 Unix 平台上).

Additionally, PowerShell's pipeline is invariably text-based when it comes to piping data to external programs; the internally UTF-16LE-based PowerShell (.NET) strings are transcoded based on the encoding stored in the automatic $OutputEncoding variable, which defaults to ASCII-only encoding in Windows PowerShell, and to UTF-8 encoding in PowerShell Core (both on Windows and on Unix-like platforms).

• 在 PowerShell Core 中，正在讨论一项更改 用于在外部程序之间传输原始字节流.

• In PowerShell Core, a change is being discussed for piping raw byte streams between external programs.

echo -n 在 PowerShell 中不会生成没有尾随换行符的字符串的事实因此附带您的问题;为了完整起见，这里有一个解释:

The fact that echo -n in PowerShell does not produce a string without a trailing newline is therefore incidental to your problem; for the sake of completeness, here's an explanation:

• echo 是 PowerShell 的 Write-Output cmdlet，它 - 在管道到 外部 程序的上下文中 - 将 text 写入next管道段中程序的标准输入(类似于Bash/cmd.exe的echo).
• -n 被解释为 Write-Output 的 -NoEnumerate 开关的(明确的)缩写.
• -NoEnumerate 只适用于写入多个对象的情况，所以这里没有效果.
• 因此，简而言之:在 PowerShell 中，echo -n "string" 与 Write-Output -NoEnumerate "string" 相同，这 - 因为只有输出单个字符串 - 与 Write-Output string" 相同，而后者又与仅使用 string" 相同，依赖于PowerShell 的隐式输出行为.
• Write-Output 没有选项来抑制尾随的换行符，即使有，使用管道通过管道连接到 外部 程序会将其重新添加进去.

• echo is an alias for PowerShell's Write-Output cmdlet, which - in the context of piping to external programs - writes text to the standard input of the program in the next pipeline segment (similar to Bash / cmd.exe's echo).
• -n is interpreted as an (unambiguous) abbreviation for Write-Output's -NoEnumerate switch.
• -NoEnumerate only applies when writing multiple objects, so it has no effect here.
• Therefore, in short: in PowerShell, echo -n "string" is the same as Write-Output -NoEnumerate "string", which - because only a single string is output - is the same as Write-Output "string", which, in turn, is the same as just using "string", relying on PowerShell's implicit output behavior.
• Write-Output has no option to suppress a trailing newline, and even if it did, using a pipeline to pipe to an external program would add it back in.

这篇关于管道文本到外部程序附加一个尾随换行符的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！