Linux内核设备驱动程序将DMA从设备插入用户空间内存 [英] Linux kernel device driver to DMA from a device into user-space memory
问题描述
我想尽快从启用了DMA的PCIe硬件设备中获取数据到用户空间.
I want to get data from a DMA enabled, PCIe hardware device into user-space as quickly as possible.
问:如何将直接I/O通过DMA传输和/或通过DMA传输结合到用户空间"
Q: How do I combine "direct I/O to user-space with/and/via a DMA transfer"
-
通读LDD3，看来我需要执行几种不同类型的IO操作！
Reading through LDD3, it seems that I need to perform a few different types of IO operations!?
dma_alloc_coherent给了我可以传递给硬件设备的物理地址.
但是将需要设置get_user_pages并在传输完成后执行copy_to_user类型的调用.这似乎很浪费，要求设备将DMA存入内核内存(充当缓冲区)，然后将其再次传输到用户空间.
LDD3 p453:/* Only now is it safe to access the buffer, copy to user, etc. */
dma_alloc_coherent gives me the physical address that I can pass to the hardware device.
But would need to have setup get_user_pages and perform a copy_to_user type call when the transfer completes. This seems a waste, asking the Device to DMA into kernel memory (acting as buffer) then transferring it again to user-space.
LDD3 p453: /* Only now is it safe to access the buffer, copy to user, etc. */
我理想中想要的是一些内存:
What I ideally want is some memory that:
- 我可以在用户空间中使用(也许通过ioctl调用请求驱动程序来创建DMA'able内存/缓冲区?)
- 我可以从中获取物理地址以传递给设备，以便所有用户空间要做的就是在驱动程序上执行读取
- 读取方法将激活DMA传输，阻塞等待DMA的完整中断，然后释放随后读取的用户空间(用户空间现在可以安全地使用/读取内存).
我是否需要用get_user_pages dma_map_page映射的单页流映射，设置映射和用户空间缓冲区?
Do I need single-page streaming mappings, setup mapping and user-space buffers mapped with get_user_pages dma_map_page?
到目前为止，我的代码在用户空间的给定地址处设置了get_user_pages(我将其称为直接I/O部分).然后，在dma_map_page中显示get_user_pages中的页面.我将dma_map_page的返回值作为DMA物理传输地址提供给设备.
My code so far sets up get_user_pages at the given address from user-space (I call this the Direct I/O part). Then, dma_map_page with a page from get_user_pages. I give the device the return value from dma_map_page as the DMA physical transfer address.
我正在使用一些内核模块作为参考:drivers_scsi_st.c和drivers-net-sh_eth.c.我会看infiniband代码，但找不到最基本的代码！
I am using some kernel modules as reference: drivers_scsi_st.c and drivers-net-sh_eth.c. I would look at infiniband code, but cant find which one is the most basic!
非常感谢.
推荐答案
我现在实际上正在做完全相同的事情，而我正在使用ioctl()路线.总体思路是让用户空间分配用于DMA传输的缓冲区，并使用ioctl()将缓冲区的大小和地址传递给设备驱动程序.然后，驱动程序将使用分散收集列表以及流DMA API来直接与设备和用户空间缓冲区之间来回传输数据.
I'm actually working on exactly the same thing right now and I'm going the ioctl() route. The general idea is for user space to allocate the buffer which will be used for the DMA transfer and an ioctl() will be used to pass the size and address of this buffer to the device driver. The driver will then use scatter-gather lists along with the streaming DMA API to transfer data directly to and from the device and user-space buffer.
我正在使用的实现策略是，驱动程序中的ioctl()进入一个循环，该循环以256k的块(这是硬件对其可处理的分散/聚集条目的硬性限制)的形式对DMA的用户空间缓冲区进行循环.这是隔离在一个函数内部的，该函数在每次传输完成之前都会阻塞(请参阅下文).当所有字节传输完毕或增量传输函数返回错误时，ioctl()退出并返回用户空间
The implementation strategy I'm using is that the ioctl() in the driver enters a loop that DMA's the userspace buffer in chunks of 256k (which is the hardware imposed limit for how many scatter/gather entries it can handle). This is isolated inside a function that blocks until each transfer is complete (see below). When all bytes are transfered or the incremental transfer function returns an error the ioctl() exits and returns to userspace
ioctl()
/*serialize all DMA transfers to/from the device*/
if (mutex_lock_interruptible( &device_ptr->mtx ) )
return -EINTR;
chunk_data = (unsigned long) user_space_addr;
while( *transferred < total_bytes && !ret ) {
chunk_bytes = total_bytes - *transferred;
if (chunk_bytes > HW_DMA_MAX)
chunk_bytes = HW_DMA_MAX; /* 256kb limit imposed by my device */
ret = transfer_chunk(device_ptr, chunk_data, chunk_bytes, transferred);
chunk_data += chunk_bytes;
chunk_offset += chunk_bytes;
}
mutex_unlock(&device_ptr->mtx);
用于增量传递函数的伪代码:
Pseudo code for incremental transfer function:
/*Assuming the userspace pointer is passed as an unsigned long, */
/*calculate the first,last, and number of pages being transferred via*/
first_page = (udata & PAGE_MASK) >> PAGE_SHIFT;
last_page = ((udata+nbytes-1) & PAGE_MASK) >> PAGE_SHIFT;
first_page_offset = udata & PAGE_MASK;
npages = last_page - first_page + 1;
/* Ensure that all userspace pages are locked in memory for the */
/* duration of the DMA transfer */
down_read(¤t->mm->mmap_sem);
ret = get_user_pages(current,
current->mm,
udata,
npages,
is_writing_to_userspace,
0,
&pages_array,
NULL);
up_read(¤t->mm->mmap_sem);
/* Map a scatter-gather list to point at the userspace pages */
/*first*/
sg_set_page(&sglist[0], pages_array[0], PAGE_SIZE - fp_offset, fp_offset);
/*middle*/
for(i=1; i < npages-1; i++)
sg_set_page(&sglist[i], pages_array[i], PAGE_SIZE, 0);
/*last*/
if (npages > 1) {
sg_set_page(&sglist[npages-1], pages_array[npages-1],
nbytes - (PAGE_SIZE - fp_offset) - ((npages-2)*PAGE_SIZE), 0);
}
/* Do the hardware specific thing to give it the scatter-gather list
and tell it to start the DMA transfer */
/* Wait for the DMA transfer to complete */
ret = wait_event_interruptible_timeout( &device_ptr->dma_wait,
&device_ptr->flag_dma_done, HZ*2 );
if (ret == 0)
/* DMA operation timed out */
else if (ret == -ERESTARTSYS )
/* DMA operation interrupted by signal */
else {
/* DMA success */
*transferred += nbytes;
return 0;
}
中断处理程序非常简短:
The interrupt handler is exceptionally brief:
/* Do hardware specific thing to make the device happy */
/* Wake the thread waiting for this DMA operation to complete */
device_ptr->flag_dma_done = 1;
wake_up_interruptible(device_ptr->dma_wait);
请注意，这只是一般方法，最近几周我一直在研究此驱动程序，但尚未进行实际测试...因此，请不要将此伪代码视为福音，请务必仔细检查所有逻辑和参数;-).
Please note that this is just a general approach, I've been working on this driver for the last few weeks and have yet to actually test it... So please, don't treat this pseudo code as gospel and be sure to double check all logic and parameters ;-).
这篇关于Linux内核设备驱动程序将DMA从设备插入用户空间内存的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！
