在舞者中启用utf8，但不启用脚本 [英] utf8 on in dancer but not script
问题描述
这是对我先前的有关显示unicode字符串差异的问题的跟踪调查。事实证明，这些字符串似乎是相同的，但是其中之一中的UTF8标志已打开。
SV = PVMG （0x4cca750）at 0x4b3fc90
REFCNT = 1
标志=（PADMY，POK，pPOK，UTF8）
IV = 0
NV = 0
PV = 0x1eda410 flurbe \0 [UTF8 flurbe]
CUR = 6
LEN = 16
vs
SV = PV（0xf28090）at 0xf4b6a0
REFCNT = 1
标志=（ PADMY，POK，pPOK）
PV = 0xf37b90 flurbe \0
CUR = 6
LEN = 16
当我加密字符串时，这似乎在所产生的sha512哈希值之间有所不同。据我所知，Dancer是导致第一个结果具有utf8的原因，我的其他脚本只是一个命令行，而没有使用舞者，我该如何强制其以相同的方式运行？
（这不是评论，而是答案，但太大了。）
我刚刚运行了这个程序：
＃！/ usr / bin / perl -w
使用警告；
使用严格；
使用Devel :: Peek（）;
使用Digest :: SHA（）;
我的$ x = flurbe；
Devel :: Peek :: Dump $ x;
print Digest :: SHA :: sha512_hex（$ x）， \n\n;
utf8 ::升级$ x;
Devel :: Peek :: Dump $ x;
print Digest :: SHA :: sha512_hex（$ x）， \n;
__END__
它给出了以下输出：
SV = PV（0x10441040）at 0x10491638
REFCNT = 1
标志=（PADMY，POK，pPOK）
PV = 0x10449ca0 flurbe \0
CUR = 6
LEN = 8
1cd2e71e55653caeb6c9bffa47a66ff1c9b526bbb732dcff28412090601e9b5e34d36be6a0267527347cd94039b383d4bc45b1b104b104b7b104b7b104b7b10b7b0bb7b0bb0b1b0b1b0b0b0b0b2e0b0b0b0b0b0b1b0b0b0b0b0b0b0b0b0b0b0b0b0b0b0b0b0b0b0b0b0b0b0b0b0b0b0b0b0b0b0b0b 1
标志=（PADMY，POK，pPOK，UTF8）
PV = 0x10449ca0 flurbe \0 [UTF8 flurbe]
CUR = 6
LEN = 8
1cd2e71e55653caeb6c9bffa47a66ff1c9b526bbb732dcff28412090601e9b5e34d36be6a0267527347cd94039b383d4bc45653d786d1041debe7faa0716bdf1
正确标识该字符串已升级为UTF-8，但这不会影响 Digest :: SHA 。
编辑后添加：在上面的评论中，你提到你的哈希是随机加盐的。这些盐是否可以包含ASCII范围以外的字节？如果是这样，则使用UTF-8升级的字符串进行连接会影响其内容。我只是运行了这个修改过的程序：
＃！/ usr / bin / perl -w
use警告；
使用严格；
使用Devel :: Peek（）;
使用Digest :: SHA（）;
我的$ x = flurbe；
my $ y = \xA0; ＃单个字节，十六进制00A0
my $ z = \xC2\xA0; ＃U + 00A0的UTF-8表示形式，为字节字符串
Devel :: Peek :: Dump $ x $ y;
print Digest :: SHA :: sha512_hex（ $ x $ y）， \n\n;
Devel :: Peek :: Dump $ x $ z;
print Digest :: SHA :: sha512_hex（ $ x $ z）， \n\n;
utf8 ::升级$ x;
Devel :: Peek :: Dump $ x $ y;
print Digest :: SHA :: sha512_hex（ $ x $ y）， \n;
__END__
它给出了以下输出：
SV = PV（0x104410e8）at 0x104d68d8
REFCNT = 1
标志=（PADTMP，POK，pPOK）
PV = 0x10449ca0 flurbe\240 \0
CUR = 7
LEN = 8
1901f989ed76143697ecc6683fd03ec793bc126d51cdbee0a72241933136c144f2e602828abddc7e4843df5542a099be92313fa5874bc1bffc4b1c104bfc4b1c0fb4e1b1e0bdc0bdc0bd0c0b0e0b0e0b0e0c0b0b0b0b0b0b0c0b0b0b0b0b0b0b0b0b0c0b0b0b0b0b0b0b0b0b0b0b0b0b0b0b0b0b0b0b0c b REFCNT = 1
标志=（PADTMP，POK，pPOK）
PV = 0x10489170 flurbe\302\240 \0
CUR = 8
LEN = 12
072f7b54c80fa8062ca1d17727a88c9ff4815f83c1166471331c6398b9140a06812eff341c98453f4c51356926dbe9694cbcbebfe4cda7e77cf68008ab838c6d
SV = PV（0x104d80a8）在0x104f0f98
REFCNT = 1个
FLAGS =（PADTMP，POK，pPOK，UTF8）
PV = 0x104896c8 flurbe\302\240 \0 [UTF8 flurbe\x {a0}]
CUR = 8
LEN = 12
072f7b54c80fa8062ca1d17727a88c9ff4815f83c1166471331c6398b9140a06812eff341c98453f4c51356926 94cbcbebfe4cda7e77cf68008ab838c6d
如您所见， $ x的SHA-512哈希$ y 取决于 $ x 是否已UTF-8升级。 $ x $ y 具有UTF-8升级的 $ x 可以提供与SHA-512相同的散列 $ x $ z 和 non -UTF-8升级的 $ x 。这是因为SHA-512是对字节而不是字符进行操作，并且UTF-8升级的字符串与字节字符串的连接导致字节字符串被UTF-8升级。
This is a follow up to my previous question on showing unicode string differences. As it turns out the strings appear to be the same, however in one of them the UTF8 flag is on.
SV = PVMG(0x4cca750) at 0x4b3fc90
REFCNT = 1
FLAGS = (PADMY,POK,pPOK,UTF8)
IV = 0
NV = 0
PV = 0x1eda410 "flurbe"\0 [UTF8 "flurbe"]
CUR = 6
LEN = 16
vs
SV = PV(0xf28090) at 0xf4b6a0
REFCNT = 1
FLAGS = (PADMY,POK,pPOK)
PV = 0xf37b90 "flurbe"\0
CUR = 6
LEN = 16
This appears to make a difference between the resulting sha512 hashes when I encrypt the string. Dancer is what is causing the first result to have utf8 as far as I can tell, my other script is simply a command line one, without using dancer in that how can I force it to behave in the same way?
解决方案 (This is more of a comment than an answer, but it's too big.)
I just ran this program:
#!/usr/bin/perl -w
use warnings;
use strict;
use Devel::Peek ();
use Digest::SHA ();
my $x = 'flurbe';
Devel::Peek::Dump $x;
print Digest::SHA::sha512_hex($x), "\n\n";
utf8::upgrade $x;
Devel::Peek::Dump $x;
print Digest::SHA::sha512_hex($x), "\n";
__END__
and it gave this output:
SV = PV(0x10441040) at 0x10491638
REFCNT = 1
FLAGS = (PADMY,POK,pPOK)
PV = 0x10449ca0 "flurbe"\0
CUR = 6
LEN = 8
1cd2e71e55653caeb6c9bffa47a66ff1c9b526bbb732dcff28412090601e9b5e34d36be6a0267527347cd94039b383d4bc45653d786d1041debe7faa0716bdf1
SV = PV(0x10441040) at 0x10491638
REFCNT = 1
FLAGS = (PADMY,POK,pPOK,UTF8)
PV = 0x10449ca0 "flurbe"\0 [UTF8 "flurbe"]
CUR = 6
LEN = 8
1cd2e71e55653caeb6c9bffa47a66ff1c9b526bbb732dcff28412090601e9b5e34d36be6a0267527347cd94039b383d4bc45653d786d1041debe7faa0716bdf1
As you can see, Devel::Peek::Dump correctly identifies that the string has been upgraded to UTF-8, but this doesn't affect the SHA-512 hash computed by Digest::SHA.
Edited to add: In a comment above, you mention that your "hashes are random salted". Can these salts include bytes outside the ASCII range? If so, concatenation with a UTF-8-upgraded string can affect their contents. I just ran this modified program:
#!/usr/bin/perl -w
use warnings;
use strict;
use Devel::Peek ();
use Digest::SHA ();
my $x = 'flurbe';
my $y = "\xA0"; # a single byte, hex 00A0
my $z = "\xC2\xA0"; # UTF-8 representation of U+00A0, as a byte-string
Devel::Peek::Dump "$x$y";
print Digest::SHA::sha512_hex("$x$y"), "\n\n";
Devel::Peek::Dump "$x$z";
print Digest::SHA::sha512_hex("$x$z"), "\n\n";
utf8::upgrade $x;
Devel::Peek::Dump "$x$y";
print Digest::SHA::sha512_hex("$x$y"), "\n";
__END__
and it gave this output:
SV = PV(0x104410e8) at 0x104d68d8
REFCNT = 1
FLAGS = (PADTMP,POK,pPOK)
PV = 0x10449ca0 "flurbe\240"\0
CUR = 7
LEN = 8
1901f989ed76143697ecc6683fd03ec793bc126d51cdbee0a72241933136c144f2e602828abddc7e4843df5542a099be92313fa5874d1d2dc54ecdd1ff308c5e
SV = PV(0x104d80b8) at 0x104ec098
REFCNT = 1
FLAGS = (PADTMP,POK,pPOK)
PV = 0x10489170 "flurbe\302\240"\0
CUR = 8
LEN = 12
072f7b54c80fa8062ca1d17727a88c9ff4815f83c1166471331c6398b9140a06812eff341c98453f4c51356926dbe9694cbcbebfe4cda7e77cf68008ab838c6d
SV = PV(0x104d80a8) at 0x104f0f98
REFCNT = 1
FLAGS = (PADTMP,POK,pPOK,UTF8)
PV = 0x104896c8 "flurbe\302\240"\0 [UTF8 "flurbe\x{a0}"]
CUR = 8
LEN = 12
072f7b54c80fa8062ca1d17727a88c9ff4815f83c1166471331c6398b9140a06812eff341c98453f4c51356926dbe9694cbcbebfe4cda7e77cf68008ab838c6d
As you can see, the SHA-512 hash of "$x$y" depends on whether $x was UTF-8-upgraded. "$x$y" with a UTF-8-upgraded $x gives the same SHA-512 hash as does "$x$z" with a non-UTF-8-upgraded $x. This is because SHA-512 operates on bytes, not characters, and the concatenation of a UTF-8-upgraded string with a byte-string causes the byte-string to be UTF-8-upgraded.
这篇关于在舞者中启用utf8，但不启用脚本的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！
