我们是否需要互斥锁来执行多线程文件IO [英] Do we need mutex to perform multithreading file IO

问题描述

我正在尝试使用多个线程(pthread)对文件进行随机写入(基准测试).如果我注释掉mutex lock，看起来好像创建的文件大小小于实际大小，好像有些写入丢失了(总是块大小的某些倍数).但是，如果我保留互斥锁，则大小总是精确的.

I'm trying to do random write (Benchmark test) to a file using multiple threads (pthread). Looks like if I comment out mutex lock the created file size is less than actual as if Some writes are getting lost (always in some multiple of chunk size). But if I keep the mutex it's always exact size.

我的代码是否在其他地方有问题，并且互斥量并不是真正必需的(如@evan所建议)或这里需要互斥锁

Is my code have a problem in other place and mutex is not really required (as suggested by @evan ) or mutex is necessary here

void *DiskWorker(void *threadarg) {

FILE *theFile = fopen(fileToWrite, "a+");
....
for (long i = 0; i < noOfWrites; ++i) {
            //pthread_mutex_lock (&mutexsum);
            // For Random access

            fseek ( theFile , randomArray[i] * chunkSize  , SEEK_SET );
            fputs ( data , theFile );

            //Or for sequential access (in this case above 2 lines would not be here)

            fprintf(theFile, "%s", data);
            //sequential access end

            fflush (theFile);
            //pthread_mutex_unlock(&mutexsum);
        }
.....
}

推荐答案

您肯定需要一个互斥锁，因为您要发出几个不同的文件命令.基础文件子系统可能无法知道要调用多少文件命令来完成整个操作.

You definitely need a mutex because you are issuing several different file commands. The underlying file subsystem can't possibly know how many file commands you are going to call to complete your whole operation.

所以您需要互斥锁.

在您遇到的情况下，您可能会发现将互斥锁置于循环之外会获得更好的性能.原因是，否则，线程之间的切换可能会导致磁盘不同部分之间的过度跳过.硬盘大约需要10ms来移动读/写头，这样可能会使事情变慢很多.

In your situation you may find you get better performance putting the mutex outside the loop. The reason being that, otherwise, switching between threads may cause excessive skipping between different parts of the disk. Hard disks take about 10ms to move the read/write head so that could potentially slow things down a lot.

因此对它进行基准测试可能是一个好主意.

So it might be a good idea to benchmark that.

这篇关于我们是否需要互斥锁来执行多线程文件IO的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！