我如何分配 Vec u8 ?与缓存行的大小对齐? [英] How do I allocate a Vec<u8> that is aligned to the size of the cache line?

问题描述

我需要分配一个缓冲区来读取 File，但是这个缓冲区必须与缓存行的大小(64 字节)对齐.我正在为 Vec 寻找类似这样的函数:

I need to allocate a buffer for reading from a File, but this buffer must be aligned to the size of the cache line (64 bytes). I am looking for a function somewhat like this for Vec:

pub fn with_capacity_and_aligned(capacity: usize, alignment: u8) -> Vec<T>

这会给我我需要的 64 字节对齐.这显然不存在，但可能有一些我不知道的等价物(即黑客").

which would give me the 64 byte alignment that I need. This obviously doesn't exist, but there might be some equivalences (i.e. "hacks") that I don't know about.

所以，当我使用这个函数(它会给我想要的对齐)时，我可以安全地编写这段代码:

So, when I use this function (which will give me the desired alignment), I could write this code safely:

#[repr(C)]
struct Header {
    magic: u32,
    some_data1: u32,
    some_data2: u64,
}

let cache_line_size = 64; // bytes
let buffer: Vec<u8> = Vec::<u8>::with_capacity_and_alignment(some_size, cache_line_size);
match file.read_to_end(&mut buffer) {
    Ok(_) => {
        let header: Header = {
            // and since the buffer is aligned to 64 bytes, I wont get any SEGFAULT
            unsafe { transmute(buffer[0..(size_of::<Header>())]) }
        };
    }
}

并且不会因为对齐问题而出现任何恐慌(就像启动指令).

and not get any panics because of alignment issues (like launching an instruction).

推荐答案

您可以使用 #[repr(align(...))] 强制将类型对齐到特定大小.我们还使用 repr(C) 来确保该类型具有与字节数组相同的内存布局.

You can enforce the alignment of a type to a certain size using #[repr(align(...))]. We also use repr(C) to ensure that this type has the same memory layout as an array of bytes.

然后您可以创建一个对齐类型的向量并将其转换为适当类型的向量:

You can then create a vector of the aligned type and transform it to a vector of appropriate type:

use std::mem;

#[repr(C, align(64))]
struct AlignToSixtyFour([u8; 64]);

unsafe fn aligned_vec(n_bytes: usize) -> Vec<u8> {
    // Lazy math to ensure we always have enough.
    let n_units = (n_bytes / mem::size_of::<AlignToSixtyFour>()) + 1;

    let mut aligned: Vec<AlignToSixtyFour> = Vec::with_capacity(n_units);

    let ptr = aligned.as_mut_ptr();
    let len_units = aligned.len();
    let cap_units = aligned.capacity();

    mem::forget(aligned);

    Vec::from_raw_parts(
        ptr as *mut u8,
        len_units * mem::size_of::<AlignToSixtyFour>(),
        cap_units * mem::size_of::<AlignToSixtyFour>(),
    )
}

如果您重新分配数据，则无法保证 Vec 将保持对齐.这意味着您不能重新分配，因此您需要知道预先分配的大小.

There are no guarantees that the Vec<u8> will remain aligned if you reallocate the data. This means that you cannot reallocate so you will need to know how big to allocate up front.

出于同样的原因，该函数不安全.当类型被删除时，内存必须回到其原始分配，但该函数无法控制.

The function is unsafe for the same reason. When the type is dropped, the memory must be back to its original allocation, but this function cannot control that.

感谢 BurntSushi5 更正和补充.

Thanks to BurntSushi5 for corrections and additions.

另见:

• 如何将结构与指定的字节边界对齐?
• 将结构与 Rust 中的缓存行对齐
• 如何转换 Vec到 Vec U不复制向量?

由于上述限制和不安全性，另一个潜在的想法是分配一个足够大的缓冲区(可能还有一些回旋余地)，然后使用 align_to 以获得正确对齐的块.您可以使用与上述相同的 AlignToSixtyFour 类型，然后将 &[AlignToSixtyFour] 转换为具有类似逻辑的 &[u8].

Because of the limitations and unsafety above, another potential idea would be to allocate a big-enough buffer (maybe with some wiggle room), and then use align_to to get a properly aligned chunk. You could use the same AlignToSixtyFour type as above, and then convert the &[AlignToSixtyFour] into a &[u8] with similar logic.

此技术可用于给出对齐的(可选可变的)切片.由于它们是切片，因此您不必担心用户重新分配或删除它们.这将允许您将其包装成更好的类型.

This technique could be used to give out (optionally mutable) slices that are aligned. Since they are slices, you don't have to worry about the user reallocating or dropping them. This would allow you to wrap it up in a nicer type.

综上所述，我认为这里依赖对齐方式不适合您从文件中读取结构的实际目标.只需读取字节(u32、u32、u64)并构建结构:

All that being said, I think that relying on alignment here is inappropriate for your actual goal of reading a struct from a file. Simply read the bytes (u32, u32, u64) and build the struct:

use byteorder::{LittleEndian, ReadBytesExt}; // 1.3.4
use std::{fs::File, io};

#[derive(Debug)]
struct Header {
    magic: u32,
    some_data1: u32,
    some_data2: u64,
}

impl Header {
    fn from_reader(mut reader: impl io::Read) -> Result<Self, Box<dyn std::error::Error>> {
        let magic = reader.read_u32::<LittleEndian>()?;
        let some_data1 = reader.read_u32::<LittleEndian>()?;
        let some_data2 = reader.read_u64::<LittleEndian>()?;

        Ok(Self {
            magic,
            some_data1,
            some_data2,
        })
    }
}

fn main() -> Result<(), Box<dyn std::error::Error>> {
    let mut f = File::open("/etc/hosts")?;

    let header = Header::from_reader(&mut f)?;

    println!("{:?}", header);

    Ok(())
}

另见:

• 如何在 Rust 中从文件中读取结构体?
• 这是从二进制文件中读取结构最自然的方式吗?
• 我可以将字节数组反序列化为结构体吗?
• 在 Rust 中将 u8 缓冲区转换为结构体

这篇关于我如何分配 Vec u8 ?与缓存行的大小对齐?的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！