如何处理二进制文件格式的可移植性问题 [英] How to handle portability issues in a binary file format
问题描述
i。处理小/大端系统的最好方法是什么？
i.a会把所有东西都转换成网络字节顺序，然后用ntohl（）/ htonl（）工作吗？
<在x86，x64和arm上，打包结构的大小是否相同？
iii。这种方法是否有其固有的弱点？
struct __attribute __（（packed））Header {
uint8_t magic;
uint8_t标志;
};
$ b $ struct __attribute __（（packed））记录{
uint64_t length;
uint32_t crc;
uint16_t year;
uint8_t day;
uint8_t month;
uint8_t小时;
uint8_t分钟;
uint8_t second;
uint8_t type;
};
测试者代码我使用develop的格式：
#include
#include< unistd.h>
#include< stdio.h>
#include< limits.h>
#include< strings.h>
#include< stdint.h>
#include< sys / stat.h>
#include< fcntl.h>
#include< string.h>
struct __attribute __（（packed））Header {
uint8_t magic;
uint8_t标志;
};
$ b $ struct __attribute __（（packed））记录{
uint64_t length;
uint32_t crc;
uint16_t year;
uint8_t day;
uint8_t month;
uint8_t小时;
uint8_t分钟;
uint8_t second;
uint8_t type;
};
int main（void）
{
int fd = open（test.dat，O_RDWR | O_APPEND | O_CREAT，444）;
struct Header header = {1,0};
write（fd，& header，sizeof（header））;
char msg [] = {BINARY};
struct record record = {strlen（msg），0，0，0，0，0，0};
write（fd，& record，sizeof（record））;
write（fd，msg，record.length）;
close（fd）;
fd = open（test.dat，O_RDWR | O_APPEND | O_CREAT，444）;
read（fd，& header，sizeof（struct Header））;
read（fd，& record，sizeof（struct Record））;
int len = record.length;
char c;
while（len！= 0）{
read（fd，& c，1）;
len--;
printf（％c，c）;
}
close（fd）;
将文件定义为一个顺序，并根据需要转换为内部顺序，如果有必要的话，在阅读/写作（也许用ntohl等）时，在我看来，这是最好的方法。
。b $ b ii。我不信任打包结构。他们可能会为这些平台的工作，但没有保证。
三。在整个结构中使用fread和fwrite读写二进制文件（在我看来）是一个固有的弱方法。你最大的可能性，你会被字节大小问题，填充和对齐问题，字节顺序问题咬。
我喜欢做的是写一点功能，如get16 （）和put32（），它们一次读写一个字节，因此对字大小和字节顺序的困难本质上是不敏感的。然后我写了一个简单的putHeader和getRecord函数（就像这样）。
$ $ $ $ $ $ $ $ $> unsigned int get16（FILE * fp）
{
unsigned int r;
r = getc（fp）;
r =（r <8）| GETC（FP）;
return r;
}
void put32（unsigned long int x，FILE * fp）
{
putc（（int）（（x>> 24）& 0xff），fp）;
putc（（int）（（x>> 16）& 0xff），fp）;
putc（（int）（（x>> 8）& 0xff），fp）;
putc（（int）（x& 0xff），fp）;
}
正如@Olaf在其中一个注释中正确指出的那样，在生产代码中，您需要处理EOF和这些函数中的错误。我把这些留给了简单的介绍。]
I'm designing a binary file format to store strings[without terminating null to save space] and binary data.
i. What is the best way to deal with little/big endian systems? i.a Would converting everything to network byte order and back with ntohl()/htonl() work?
ii. Will the packed structures be the same size on x86, x64 and arm?
iii. Are their any inherent weakness with this approach?
struct __attribute__((packed)) Header {
uint8_t magic;
uint8_t flags;
};
struct __attribute__((packed)) Record {
uint64_t length;
uint32_t crc;
uint16_t year;
uint8_t day;
uint8_t month;
uint8_t hour;
uint8_t minute;
uint8_t second;
uint8_t type;
};
Tester code I'm using the develop the format:
#include <stdlib.h>
#include <unistd.h>
#include <stdio.h>
#include <limits.h>
#include <strings.h>
#include <stdint.h>
#include <sys/stat.h>
#include <fcntl.h>
#include <string.h>
struct __attribute__((packed)) Header {
uint8_t magic;
uint8_t flags;
};
struct __attribute__((packed)) Record {
uint64_t length;
uint32_t crc;
uint16_t year;
uint8_t day;
uint8_t month;
uint8_t hour;
uint8_t minute;
uint8_t second;
uint8_t type;
};
int main(void)
{
int fd = open("test.dat", O_RDWR|O_APPEND|O_CREAT, 444);
struct Header header = {1, 0};
write(fd, &header, sizeof(header));
char msg[] = {"BINARY"};
struct Record record = {strlen(msg), 0, 0, 0, 0, 0, 0, 0};
write(fd, &record, sizeof(record));
write(fd, msg, record.length);
close(fd);
fd = open("test.dat", O_RDWR|O_APPEND|O_CREAT, 444);
read(fd, &header, sizeof(struct Header));
read(fd, &record, sizeof(struct Record));
int len = record.length;
char c;
while (len != 0) {
read(fd, &c, 1);
len--;
printf("%c", c);
}
close(fd);
}
i. Defining the file to be in one order and converting to and from "internal" order, if necessary, when reading/writing (perhaps with ntohl and the like) is, in my opinion, the best approach.
ii. I do not trust packed structures. They might work for this approach for those platforms, but there are no guarantees.
iii. Reading and writing binary files using fread and fwrite on whole structs is (again in my opinion) an inherently weak approach. You maximize the likelihood that you will be bitten by word size problems, padding and alignment problems, and byte order problems.
What I like to do is write little functions like get16() and put32() that read and write a byte at a time and so are inherently insensitive to word size and byte order difficulties. Then I write straightforward putHeader and getRecord functions (and the like) in terms of these.
unsigned int get16(FILE *fp)
{
unsigned int r;
r = getc(fp);
r = (r << 8) | getc(fp);
return r;
}
void put32(unsigned long int x, FILE *fp)
{
putc((int)((x >> 24) & 0xff), fp);
putc((int)((x >> 16) & 0xff), fp);
putc((int)((x >> 8) & 0xff), fp);
putc((int)(x & 0xff), fp);
}
[P.S. As @Olaf correctly points out in one of the comments, in production code you'd need handling for EOF and error in these functions. I've left those out for simplicity of presentation.]
这篇关于如何处理二进制文件格式的可移植性问题的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！
