是否存在同步文件流？ [英] Is there a limiltation of simultaneous filestreams?

问题描述

我对使用同时打开的 fstream s有一个战略问题。
我必须编写一个程序，该程序也读取了大量文件。每个文件中都有一堆标识符的信息，但是只有一次。我必须计算这些信息，然后将每个标识符的信息保存在单独的文件中。每个标识符都出现在多个文件中，并且每次应保存在同一文件中（一个标识符多次）。
我期望有数百个标识符，所以我怀疑我是否应该同时打开数百个文件组。

I have a strategic question to the use of simultaneously opened fstreams. I have to write a program which has too read a large amount of files. In each file there are information to a bunch of identifiers, but one time. I have to compute this information and than save it for each identifier in a separate file. Every identifier appears in several files and should be saved every time in the same file (One identifier with many times). I expect some hundred identifiers so I doubt I should have several hundred filesteams open simultaneously.

那么同时文件流是否有限制？
还是您提出另一种方法？

So is there a limitation of simultaneous filestreams? Or do you propose another way of doing this?

该程序将计算大量数据（大约10GB或更大），并且可能计算几个小时。

The program will compute a massive amount of data (about 10GB or larger) and perhaps computes several hours.

谢谢

推荐答案

最终，一切都会受到限制。文件是操作系统管理的一个很好的例子，并且您必须参考OS文档中的特定限制。我相信在Linux中，它是可在内核中配置的。可能还会有用户和进程配额。

There's ultimately a limit to anything. Files are a perfect example of something managed by the operating system, and you will have to consult your OS documentation for the specific limit. In Linux, I believe it is configurable in the kernel. There may additionally be user and process quotas.

我认为200个要求不高。

I don't think 200 is too many to ask.

尝试查看非常简单。只需编写一个程序即可打开更多文件，直到出现错误为止。

It's quite simple to try and see. Just write a program that keeps opening more files until you get an error.

在Mac上OS X 10.8，此程序

On Mac OS X 10.8, this program

#include <iostream>
#include <fstream>
#include <iomanip>
#include <string>

int main() {
    int i = 0;
    std::ofstream *f;
    do {
        f = new std::ofstream( std::to_string( i ++ ) );
    } while ( * f << "hello" << std::flush );
    -- i; // Don't count last iteration, which failed to open anything.

    std::cout << i << '\n';
}

产生输出 253 。因此，如果您使用的是Mac，那么您会很高兴:)。

Produces the output 253. So if you're on a Mac, you're golden :) .

这篇关于是否存在同步文件流？的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！