linux 内核 aio 功能 [英] linux kernel aio functionality
问题描述
我正在测试内核异步 io 函数(不是 posix aio)并试图弄清楚它是如何工作的.下面的代码是一个完整的程序，我只是简单地将一个数组重复写入使用 O_DIRECT 打开的文件中.我在回调函数中收到错误write missing bytes expect 1024 got 0"(请参阅 work_done() 中的 fprintf 语句).
I am testing kernel asynchronous io functions (not posix aio) and am trying to figure out how it works. The code below is a complete program where I simply write an array repeatedly to a file opened using O_DIRECT. I get an error in the callback function "write missed bytes expect 1024 got 0" (see the fprintf statement in work_done()).
对于不熟悉内核 aio 的人，以下代码执行以下操作:
For those not familiar with kernel aio, the code below does the following:
- 初始化一些结构
- 准备 aio (io_prep_pwrite)
- 提交 io 请求 (io_submit)
- 检查事件完成情况 (io_getevents)
- 调用回调函数查看是否一切正常.
我在第 5 步遇到错误.如果我不使用 O_DIRECT 打开文件，一切正常，但它超出了异步写入的目的.有人可以告诉我我做错了什么吗?这是内核 aio 的正确用法吗，例如，我对回调的使用是否正确?O_DIRECT 的使用有什么限制吗?
I get an error at step 5. If I do not open the file using O_DIRECT, things work fine, but it beats the purpose of having async writes. Can someone tell me what I am doing wrong? Is this the correct usage of kernel aio, for example, is my use of callbacks correct? Are there any restrictions on the usage of O_DIRECT?
我使用'gcc -Wall test.c -laio'编译
I compile using 'gcc -Wall test.c -laio'
提前致谢.
/*
* File: myaiocp.c
* Author: kmehta
*
* Created on July 11, 2011, 12:50 PM
*
*
* Testing kernel aio.
* Program creates a 2D matrix and writes it multiple times to create a file of desired size.
* Writes are performed using kernel aio functions (io_prep_pwrite, io_submit, etc.)
*/
#define _GNU_SOURCE
#define _XOPEN_SOURCE 600
#include <stdio.h>
#include <stdlib.h>
#include <getopt.h>
#include <pthread.h>
#include <fcntl.h>
#include <string.h>
#include <sys/uio.h>
#include <sys/time.h>
#include <omp.h>
#include <unistd.h>
#include <sys/types.h>
#include <sys/stat.h>
#include <errno.h>
#include <libaio.h>
char ** buf;
long seg_size;
int seg_rows;
double total_size;
char * filename;
static int wait_count = 0;
void io_task();
void cleanup();
void allocate_2D_matrix(int[]);
int file_open(char *);
void wr_done(io_context_t ctx, struct iocb* iocb, long res, long res2);
int main(int argc, char **argv) {
total_size = 1048576; //1MB
seg_size = 1024; //1kB
seg_rows = 1024;
filename = "aio.out";
int dims[] = {seg_rows, seg_size};
allocate_2D_matrix(dims); //Creates 2D matrix
io_task();
cleanup();
return 0;
}
/*
* Create a 2D matrix
*/
void allocate_2D_matrix(int dims[2]) {
int i;
char *data;
//create the matrix
data = (char *) calloc(1, dims[0] * dims[1] * sizeof (char));
if (data == NULL) {
printf("
Could not allocate memory for matrix.
");
exit(1);
}
buf = (char **) malloc(dims[0] * sizeof (char *));
if (buf == NULL) {
printf("
Could not allocate memory for matrix.
");
exit(1);
}
for (i = 0; i < dims[0]; i++) {
buf[i] = &(data[i * dims[1]]);
}
}
static void io_error(const char *func, int rc)
{
if (rc == -ENOSYS)
fprintf(stderr, "AIO not in this kernel
");
else if (rc < 0)
fprintf(stderr, "%s: %s
", func, strerror(-rc));
else
fprintf(stderr, "%s: error %d
", func, rc);
exit(1);
}
/*
* Callback function
*/
static void work_done(io_context_t ctx, struct iocb *iocb, long res, long res2)
{
if (res2 != 0) {
io_error("aio write", res2);
}
if (res != iocb->u.c.nbytes) {
fprintf(stderr, "write missed bytes expect %lu got %ld
",
iocb->u.c.nbytes, res2);
exit(1);
}
wait_count --;
printf("%d ", wait_count);
}
/*
* Wait routine. Get events and call the callback function work_done()
*/
int io_wait_run(io_context_t ctx, long iter)
{
struct io_event events[iter];
struct io_event *ep;
int ret, n;
/*
* get up to aio_maxio events at a time.
*/
ret = n = io_getevents(ctx, iter, iter, events, NULL);
printf("got %d events
", n);
/*
* Call the callback functions for each event.
*/
for (ep = events ; n-- > 0 ; ep++) {
io_callback_t cb = (io_callback_t)ep->data ; struct iocb *iocb = ep->obj ; cb(ctx, iocb, ep->res, ep->res2);
}
return ret;
}
void io_task() {
long offset = 0;
int bufIndex = 0;
//Open file
int fd = file_open(filename);
//Initialize structures
long i;
long iter = total_size / seg_size; //No. of iterations to reach desired file size (total_size)
io_context_t myctx;
if(0 != io_queue_init(iter, &myctx))
{
perror("Could not initialize io queue");
exit(EXIT_FAILURE);
}
struct iocb * ioq[iter];
//loop through iter times to reach desired file size
for (i = 0; i < iter; i++) {
struct iocb *io = (struct iocb*) malloc(sizeof (struct iocb));
io_prep_pwrite(io, fd, buf[bufIndex], seg_size, offset);
io_set_callback(io, work_done);
ioq[i] = io;
offset += seg_size;
bufIndex ++;
if (bufIndex > seg_rows - 1) //If entire matrix written, start again from index 0
bufIndex = 0;
}
printf("done preparing. Now submitting..
");
if(iter != io_submit(myctx, iter, ioq))
{
perror("Failure on submit");
exit(EXIT_FAILURE);
}
printf("now awaiting completion..
");
wait_count = iter;
int res;
while (wait_count) {
res = io_wait_run(myctx, iter);
if (res < 0)
io_error("io_wait_run", res);
}
close(fd);
}
void cleanup() {
free(buf[0]);
free(buf);
}
int file_open(char *filename) {
int fd;
if (-1 == (fd = open(filename, O_DIRECT | O_CREAT | O_WRONLY | O_TRUNC, 0666))) {
printf("
Error opening file.
");
exit(-1);
}
return fd;
}
推荐答案
首先，使用 libaio 而不是 POSIX aio 做得很好.
First of all, good job using libaio instead of POSIX aio.
O_DIRECT 的使用有什么限制吗?
Are there any restrictions on the usage of O_DIRECT ?
我不是 100% 确定这是真正的问题，但是 O_DIRECT 有一些要求(主要引用自 TLPI):
I'm not 100% sure this is the real problem, but O_DIRECT has some requirements (quoting mostly from TLPI):
- 正在传输的数据缓冲区必须在内存边界对齐，该边界是块大小的倍数(使用
posix_memalign) - 数据传输开始的文件或设备偏移量必须是块大小的倍数
- 要传输的数据长度必须是块大小的倍数
乍一看，我可以看出您没有采取任何预防措施来对齐 allocate_2D_matrix 中的内存.
At a glance, I can see you are not taking aby precautions to align memory in allocate_2D_matrix.
如果我不使用 O_DIRECT 打开文件，一切正常，但它超过了异步写入的目的.
If I do not open the file using O_DIRECT, things work fine, but it beats the purpose of having async writes.
事实并非如此.异步 I/O 在没有 O_DIRECT 的情况下运行良好(例如考虑系统调用的数量被削减).
This happens not to be the case. Asynchronous I/O works well without O_DIRECT (for instance think of the number of system calls slashed).
这篇关于linux 内核 aio 功能的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！
