将字节数组解压缩为整数 [英] Unpacking a byte array into integers
问题描述
大家好。我有一个 someArray（240）As Byte 我需要解压缩到 Int32 和 Int16 整数。在我打开包装之前我知道：
1. someArray 中的所有数据都是整数值
2.开始位置是什么每个整数都在 someArray
3.每个整数的长度是多少： Int32 或 Int16
目前我有一个以下列方式解压缩的功能：
功能解包（someArray（）作为 字节）作为 整数（）
Dim i32（ 4 ） As 字节
Dim i16（ 2 ）作为 字节
Dim icount 甲s 整数 = 75
Dim ivalues（icount） As 整数
' field1：pos = 0 len = 4
Array.Copy（someArray， 0 ，i32， 0 ， 4 ）
ivalues（ 0 ）= Int32FromByteArray（i32，bigEndian）
' field2：pos = 4 len = 2
Array.Copy（someArray， 4 ，i16， 0 ， 2 ）
ivalues（ 1 ）= Int16FromByteArray（i16，bigEndian）
' field3：pos = 6 len = 4
Array.Copy（someArray ， 6 ，i32， 0 ， 4 ）
ivalues（ 2 ）= Int32FromByteArray（i32，bigEndian）
' etc ...直到所有75个整数都从someArray解包
返回 ivalues
结束 功能
函数 Int32FromByteArray 和 Int16FromByteArray 是使用 BigEndian 重新排序到 LittleEndian 。
现在我意识到这是一种非常硬编码的解包方式，但我能够做到这一点因为位置和 someArray 中的整数长度是标准化的。
我的问题：有没有办法将字节数组 someArray 直接解压缩到整数数组 ivalues ？类似 ivalues = someArray.SplitToInteger（someMask）？
显然 SplitToInteger 函数应该做一些比我现在的 Unpack 函数更聪明的东西。显然它应该加快拆包过程。
C＃中的答案当然也是受欢迎的。提前感谢您的想法...
后续1 ：
我选择了解决方案1， 2和4，但感谢大家的反应。我测试了phil.o的代码（解决方案1），它肯定适用于C＃。我现在正在努力转换到VB.NET这不是一件容易的事，因为这两种语言的位移操作符的工作方式略有不同（对于那些感兴趣的人：.net - VB.NET和C＃之间的二进制移位差异 - 堆栈溢出）。我将在解决方案4中将phil.o扩展与F.Xaver中的Unpack函数结合使用，但出于Endianess的原因不使用BitConverter类。一旦我得到VB bitshi（f）t，我将在VB.NET中发布我的回合。 Tnx !!
后续2 ：
这里是VB.NET。使用bitshift的VB中的技巧是首先使用 CInt 来转换字节，然后 然后 应用转移（这已经是在上面引用的文章中提到过，但是昨晚我没有提到它（很晚）。
VB.NET中的大小和小的Int32扩展名都是（顺便说一下名字 someArray 已更改为 traceHeader ）：
< Extension（ ）>
公共 功能 GetInt32BigEndian（traceHeader（） As 字节，pos 作为 整数）作为 整数
Dim result 作为 整数 = CInt （traceHeader （pos + 0））<< 24 或 _
CInt （traceHeader（pos + 1））<< 16 或 _
CInt （traceHeader（pos + 2））<< 8 或 _
CInt （traceHeader（pos + 3））
返回结果
结束 功能
< Extension（）>
公共 功能 GetInt32LittleEndian（traceHeader（） As 字节，pos 作为 整数）作为 整数
Dim result 作为 整数 = CInt （traceHeader （pos + 3））<< 24 或 _
CInt （traceHeader（pos + 2））<< 16 或 _
CInt （traceHeader（pos + 1））<< 8 或 _
CInt （traceHeader（pos））
返回结果
结束 功能
我将使用Linq自行发布解决方案。我想知道你有什么要说的......？
后续3并关闭：
我测试了处理6GB文件的答案，该文件包含上述3,506,820个字节数组（称为 someArray 和 traceHeader ）。结果是：
1.使用我原始代码的3,506,820个traceHeaders：48.04秒
2.对于相同的使用解决方案1（位移）：25.28秒
3.同样使用我的解决方案5使用linq：185.31秒
因此，解决方案1是明显的赢家。我使用linq的'解决方案5'不是解决方案。
Rgds并感谢
我认为两种扩展方法分别提取Int32和Int16，使用二进制移位操作得到结果：
public static int GetInt32（ this < span class =code-keyword> byte [] array， int pos）{
int result = 0 ;
result | = array [pos ++]<< 24 ;
result | = array [pos ++]<< 16 ;
result | = array [pos ++]<< 8 ;
result | = array [pos];
返回结果;
}
public static 短 GetInt16（此 byte []数组， int pos）{
short result = 0 ;
result | = array [pos ++]<< 8 ;
result | = array [pos];
返回结果;
}
用法：
int field1 = myArray.GetInt32（ 0 ）;
short field2 = myArray.GetInt16（ 4 ）;
int field3 = myArray.GetInt32（ 6 ）;
// 等
当然，这些扩展方法必须在静态类中声明。
希望这会有所帮助。问候。
PS：我没有包括数组位置验证。
那么......
int len = bytes.Length / 4 ;
int [] ints = new int < /跨度> [长度];
int inp = 0 ;
for （ int i = 0 ; i < len; i ++）
{
ints [i] =（bytes [inp ++]<< 24 ）+（bytes [inp ++]<< 16 ）+（bytes [inp ++]<< < span class =code-digit> 8 ）+ bytes [inp ++];
}
你可以通过使用不安全的代码和指针避免数组索引来加快速度，但它可能不会加速它 - 取决于优化器，你需要时间确定。
我可能会（在C＃意义上）
a ）使用BitConverter（和你一样）
但不同，...
b）定义了一个类表示解包的'目标结构'（就在您咬住我之前，请继续阅读）
c）在（b）类中使用自定义属性来定义从中提取数据的位置目标类的每个项目
d）然后使用反射来运行循环来解压缩数据
这会比你的方法更好 - 我不知道 - 我必须看看其他人对BitConverter的看法，只要考虑到'Endian-ness'我不认为这是一个问题
<块quote class =quote>Quote：我的问题：有没有办法将字节数组someArray直接解压缩到整数数组ivalues？有什么像ivalues = someArray.SplitToInteger（someMask）？
不是我见过的
Quote：显然，SplitToInteger函数应该做一些比我当前的Unpack函数更聪明的东西。显然它应该加快拆包过程。
更聪明？为什么？如何坚持KISS？ - 我认为BitConverter足够快，除非你要去看看Bit Twiddling - 事实上，看看这个，在那里有一个速度比较解析 - 从一个字节解包整数的惯用c＃是什么阵列？ - 堆栈溢出 [ ^ ]
可能不是你想要的
Hi all. I've got an someArray(240) As Byte which I need to unpack into Int32 and Int16 integers. I know before I unpack:
1. that all data in someArray are integer values
2. what the startposition of each integer is in someArray
3. what the length of each integer is: Int32 or Int16
Currently I have a function which unpacks in the following way:
Function Unpack(someArray() As Byte) As Integer()
Dim i32(4) As Byte
Dim i16(2) As Byte
Dim icount As Integer = 75
Dim ivalues(icount) As Integer
'field1: pos=0 len=4
Array.Copy(someArray,0,i32,0,4)
ivalues(0) = Int32FromByteArray(i32,bigEndian)
'field2: pos=4 len=2
Array.Copy(someArray,4,i16,0,2)
ivalues(1) = Int16FromByteArray(i16,bigEndian)
'field3: pos=6 len=4
Array.Copy(someArray,6,i32,0,4)
ivalues(2) = Int32FromByteArray(i32,bigEndian)
'etc... until all 75 integers are unpacked from someArray
Return ivalues
End Function
Functions Int32FromByteArray and Int16FromByteArray are functions which use the BitConverter class to convert from byte to integer taking into account that the number has to be re-ordered from BigEndian to LittleEndian.
Now I realize this is a very 'hard coded' way to unpack but I am able to do this because the positions and lengths of the integers in someArray are standardized.
My question: is there a way to unpack the byte array someArray directly into the integer array ivalues? Something like ivalues = someArray.SplitToInteger(someMask)?
Obviously the SplitToInteger function should do something more clever that my current Unpack function does. And obviously it should speed up the unpacking process.
Answers in C# are welcome too of course. Thanks in advance for your thoughts...
Follow-up 1:
I picked solutions 1, 2 and 4 but thanks for the reactions of everyone. I tested out the code of phil.o (solution 1) and it surely works in C#. I'm working now on the conversion to VB.NET which is not trivial because the bit shift operators work a bit differently for the two languages (for those interested: .net - Binary Shift Differences between VB.NET and C# - Stack Overflow). I will combine phil.o extensions with the Unpack function from F.Xaver in solution 4 but not use the BitConverter class for reasons of Endianess. I will post my wrap up in VB.NET here once I got the VB bitshi(f)t going. Tnx!!
Follow-up 2:
And here it is in VB.NET. The trick in VB with the bitshift is to cast byte first with CInt and then apply the shift (this was already mentioned in the article referenced above, but I didn't pick it up (late) last night).
The big and little Int32 extensions in VB.NET are (btw the name someArray has changed to traceHeader):
<Extension()>
Public Function GetInt32BigEndian(traceHeader() As Byte, pos As Integer) As Integer
Dim result As Integer = CInt(traceHeader(pos+0)) << 24 Or _
CInt(traceHeader(pos+1)) << 16 Or _
CInt(traceHeader(pos+2)) << 8 Or _
CInt(traceHeader(pos+3))
Return result
End Function
<Extension()>
Public Function GetInt32LittleEndian(traceHeader() As Byte, pos As Integer) As Integer
Dim result As Integer = CInt(traceHeader(pos+3)) << 24 Or _
CInt(traceHeader(pos+2)) << 16 Or _
CInt(traceHeader(pos+1)) << 8 Or _
CInt(traceHeader(pos))
Return result
End Function
I'm going to post a solution myself which I picked up, using Linq. I wonder what you have to say about that...?
Follow-up 3 and close out:
I have tested the answers processing a 6GB file containing 3,506,820 of byte arrays described above (called someArray and traceHeader). The results are:
1. for 3,506,820 traceHeaders using my original code: 48.04 seconds
2. for the same using solution 1 (bit shifting): 25.28 seconds
3. for the same using my solution 5 using linq: 185.31 seconds
So, solution 1 is the clear winner. My 'solution 5' with linq is not a solution.
Rgds and thanks
I think of two extension methods which would extract Int32 and Int16 respectively, using binary shift operations to get the results:
public static int GetInt32(this byte[] array, int pos) { int result = 0; result |= array[pos++] << 24; result |= array[pos++] << 16; result |= array[pos++] << 8; result |= array[pos]; return result; } public static short GetInt16(this byte[] array, int pos) { short result = 0; result |= array[pos++] << 8; result |= array[pos]; return result; }
Usage:
int field1 = myArray.GetInt32(0); short field2 = myArray.GetInt16(4); int field3 = myArray.GetInt32(6); // etc.
Of course, these extension methods must be declared in a static class.
Hope this helps. Regards.
PS: I did not include the array position validation.
Well...
int len = bytes.Length / 4; int[] ints = new int[len]; int inp = 0; for (int i = 0; i < len; i++) { ints[i] = (bytes[inp++] << 24) + (bytes[inp++] << 16) + (bytes[inp++] << 8) + bytes[inp++]; }
You could get it faster by using unsafe code and pointers to avoid the array indexing, but it might not speed it up that much - depends on the optimiser and you'd need to time it to be sure.
I probably would have (in a C# sense)
a) used BitConverter (like you)
but the differences, ...
b) Defined a Class representing the 'target structure' for unpacking into (just before you bite my head off, read on)
c) Used Custom Attributes on the Class from (b) to define the positions to extract the data from into each item of the target class
d) Then used reflection to run a loop to unpack the data
would this be any better than your method - I dont know - I'd have to see what other people say about BitConverter, as long as one takes into account 'Endian-ness' I dont see it as an issue
Quote:My question: is there a way to unpack the byte array someArray directly into the integer array ivalues? Something like ivalues = someArray.SplitToInteger(someMask)?
not that I've seen
Quote:Obviously the SplitToInteger function should do something more clever that my current Unpack function does. And obviously it should speed up the unpacking process.
more Clever ? why ?? how about adhering to KISS ? - and I think BitConverter is 'fast enough' unless you're going to go and NEED to look at Bit Twiddling - in fact, have a look at this, somewhere there there's a speed comparison parsing - What is the idiomatic c# for unpacking an integer from a byte array? - Stack Overflow[^]
Probably not what you were looking for
这篇关于将字节数组解压缩为整数的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！
