OSX上的Java文件限制低于bash [英] Java file limit on OSX lower than in bash
问题描述
我已经增加了我的macbook pro上的最大文件限制，以便Elasticsearch可以使用更多的文件，但它不起作用。
I've increased the max files limit on my macbook pro so that Elasticsearch can work with more files, but it isn't working.
我运行命令'ulimit -a'，它说打开的文件是100,000。我可以运行这样一个简单的shell脚本：
I run the command 'ulimit -a' and it says "open files" is 100,000. I can run a simple shell script like this:
export counter=0
while (true) ; do touch "/tmp/foo${counter}" ; export counter=`expr $counter + 1` ; done
我可以创建大量文件（在我杀死脚本之前超过60,000）。
And I'm able to create lots of files (over 60,000 before I killed the script).
但是，使用Java代码在/ tmp目录的空子目录中创建RandomAccessFiles，在我收到错误之前，我只能创建10,232个文件：java .io.FileNotFoundException（打开的文件太多）。这是我的Java代码：
However, using Java code to create RandomAccessFiles in an empty sub-directory of the "/tmp" directory, I can only make 10,232 files before I get the error: java.io.FileNotFoundException (Too many open files). Here's my Java code:
import java.io.*;
import java.util.*;
public class max_open_files {
public static void main(String ... args) throws Exception {
File testDir = new File("/tmp/tempsubdir");
testDir.mkdirs();
List<File> files = new LinkedList<File>();
List<RandomAccessFile> fileHandles = new LinkedList<RandomAccessFile>();
try {
while (true) {
File f = new File(testDir, "tmp" + fileHandles.size());
RandomAccessFile raf = new RandomAccessFile(f, "rw");
files.add(f);
fileHandles.add(raf);
}
} catch (Exception ex) {
System.out.println(ex.getClass() + " " + ex.getMessage());
}
for (RandomAccessFile raf : fileHandles) raf.close()
for (File f : files) f.delete();
System.out.println("max open files: " + fileHandles.size());
}
}
这个java代码类似于Elasticsearch中的代码测试文件数量的限制（在MaxOpenFiles方法的FileSystemUtils类中）。所以Elasticsearch有与我的Java程序相同的问题。
This java code is similar to the code in Elasticsearch that tests the limit on the number of files (in the FileSystemUtils class in the maxOpenFiles method). So Elasticsearch has the same problem my Java program has.
为什么shell脚本比Java代码（我的和Elasticsearch的）更多的文件？为什么文件数量的高系统限制不能被Java代码识别？
Why can the shell script make so many more files than the Java code (mine and Elasticsearch's)? Why does the high system limit on the number of files not get recognized by the Java code?
更新5月13日下午4:50 CDT：我创建了一个C版本的测试程序来查看限制是否是特定于Java的，并且似乎是这样。下面的C版本可以打开32,765个文件，而Java代码限制为10,232个文件。
Update May 13 4:50pm CDT: I created a C version of the test program to see if the limit was Java-specific and it appears to be so. The C version below can open 32,765 files while the Java code is limited to 10,232 files.
#include <stdio.h>
#include <stdlib.h>
int main(int argc, const char *argv) {
char name[100];
FILE *fp;
int ndx;
for (ndx = 0; ndx < 50000; ndx++) {
sprintf(name, "/tmp/foo%d.txt", ndx);
fp = fopen(name, "w");
if (fp == NULL) {
fprintf(stdout, "Can not create file %d\n", ndx);
return 1;
}
fprintf(fp, "hello %d", ndx);
}
return 0;
}
推荐答案
-XX： - MaxFDLimit
指示VM禁止将文件描述符限制设置为默认最大值。默认行为是将限制设置为由OPEN_MAX指定的值，即10240.通常，这是进程可能打开的最大文件数。但是，可以使用sysctl实用程序将此限制增加到用户指定的值。在这种情况下，您可能希望通过-XX：-MaxFDLimit来停止Java VM将打开的文件数限制为10240。
Directs the VM to refrain from setting the file descriptor limit to the default maximum. The default behavior is to set the limit to the value specified by OPEN_MAX, which is 10240. Normally, this is the maximum number of files that a process may have open. It is possible, however, to increase this limit to a user-specified value with the sysctl utility. Under such circumstances, you may want to pass -XX:-MaxFDLimit to stop the Java VM from restricting the number of open files to 10240.
这篇关于OSX上的Java文件限制低于bash的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！
