我怎么忽略UTF-8字节顺序标记的字符串比较？ [英] How do I ignore the UTF-8 Byte Order Marker in String comparisons?

问题描述

我在使用Visual Studio 2010中同样的测试案例正常工作在Visual Studio 2008（用C＃3.5），一股股比较字符串的问题测试在C＃4.0。

I'm having a problem comparing strings in a Unit Test in C# 4.0 using Visual Studio 2010. This same test case works properly in Visual Studio 2008 (with C# 3.5).

下面是相关的代码片段：

Here's the relevant code snippet:

byte[] rawData = GetData();
string data = Encoding.UTF8.GetString(rawData);

Assert.AreEqual("Constant", data, false, CultureInfo.InvariantCulture);

在调试这个测试中，数据字符串出现肉眼包含完全相同的字符串字面。当我打电话 data.ToCharArray（），我注意到字符串数据的第一个字节是值 65279 这是UTF-8字节顺序标记。我不明白的是为什么 Encoding.UTF8.GetString（）围绕保持这个字节。

While debugging this test, the data string appears to the naked eye to contain exactly the same string as the literal. When I called data.ToCharArray(), I noticed that the first byte of the string data is the value 65279 which is the UTF-8 Byte Order Marker. What I don't understand is why Encoding.UTF8.GetString() keeps this byte around.

怎么办我得到 Encoding.UTF8.GetString（）来的不的把字节顺序标记得到​​的字符串中？

How do I get Encoding.UTF8.GetString() to not put the Byte Order Marker in the resulting string?

更新：的问题是，的GetData（），其内容从磁盘上的文件，使用从文件中读取数据 FileStream.readbytes（）。我通过纠正这个的StreamReader ，并使用字符串转换为字节 Encoding.UTF8.GetBytes（），这是它应该已经摆在首位一直在做！感谢所有帮助。

Update: The problem was that GetData(), which reads a file from disk, reads the data from the file using FileStream.readbytes(). I corrected this by using a StreamReader and converting the string to bytes using Encoding.UTF8.GetBytes(), which is what it should've been doing in the first place! Thanks for all the help.

推荐答案

好吧，我想这是因为原始二进制数据包括BOM。你总是可以自己删除BOM解码后，如果你不希望它 - 但你应该考虑的字节数组是否应该考虑BOM入手

Well, I assume it's because the raw binary data includes the BOM. You could always remove the BOM yourself after decoding, if you don't want it - but you should consider whether the byte array should consider the BOM to start with.

编辑：另外，您也可以使用的StreamReader 来进行解码。下面是一个例子，显示被转换成使用 Encoding.GetString 两个字符，或通过一个字符相同的字节数组的StreamReader

Alternatively, you could use a StreamReader to perform the decoding. Here's an example, showing the same byte array being converted into two characters using Encoding.GetString or one character via a StreamReader:

using System;
using System.IO;
using System.Text;

class Test
{
    static void Main()
    {
        byte[] withBom = { 0xef, 0xbb, 0xbf, 0x41 };
        string viaEncoding = Encoding.UTF8.GetString(withBom);
        Console.WriteLine(viaEncoding.Length);

        string viaStreamReader;
        using (StreamReader reader = new StreamReader
               (new MemoryStream(withBom), Encoding.UTF8))
        {
            viaStreamReader = reader.ReadToEnd();           
        }
        Console.WriteLine(viaStreamReader.Length);
    }
}

这篇关于我怎么忽略UTF-8字节顺序标记的字符串比较？的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！