持久化时，Ubuntu VM中会发生异常:打开的文件过多 [英] Ignite occur exception in Ubuntu VM when persistent: Too many open files

问题描述

我在Java代码包xxx.jar中运行一个服务器节点，但发生了如下异常:

I run a server node in java code package xxx.jar, but occur exception like this:

Caused by: java.nio.file.FileSystemException: /home/ranger/EIIP/tools/work/db/ServerNode/cache-TOFTableCache/part-942.bin: Too many open files
at sun.nio.fs.UnixException.translateToIOException(UnixException.java:91)
at sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:102)
at sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:107)
at sun.nio.fs.UnixFileSystemProvider.newAsynchronousFileChannel(UnixFileSystemProvider.java:196)
at java.nio.channels.AsynchronousFileChannel.open(AsynchronousFileChannel.java:248)
at java.nio.channels.AsynchronousFileChannel.open(AsynchronousFileChannel.java:301)
at org.apache.ignite.internal.processors.cache.persistence.file.AsyncFileIO.<init>(AsyncFileIO.java:66)
at org.apache.ignite.internal.processors.cache.persistence.file.AsyncFileIOFactory.create(AsyncFileIOFactory.java:44)
at org.apache.ignite.internal.processors.cache.persistence.file.FilePageStore.init(FilePageStore.java:523)
... 31 more

但仅在Windows的ubuntu VM中发生，并且在纯ubuntu系统上运行时没有此异常，我尝试了以下方法，但仍然存在相同的问题:

but only occured in ubuntu VM at windows, and no this exception when run at pure ubuntu system, I tried the following methods, but still the same problem：

vim /etc/security/limits.conf
root soft nofile 10240
root hard nofile 20480

vim /etc/sysctl.conf
fs.inotify.max_user_watches=524288
ulimit -n 4096

这是我的代码:

        IgniteConfiguration igniteCfg = new IgniteConfiguration();
        igniteCfg.setConsistentId("ServerNode"); //Set Consistent ID

        // Ignite Persistence
        DataStorageConfiguration storageCfg = new DataStorageConfiguration();
        DataRegionConfiguration regionCfg = new DataRegionConfiguration();
        regionCfg.setName("TableCache_Region");
        regionCfg.setInitialSize(100L * 1024 * 1024);
        regionCfg.setMaxSize(8L * 1024 * 1024 * 1024);
        regionCfg.setPersistenceEnabled(true);

        storageCfg.setDataRegionConfigurations(regionCfg);
        storageCfg.setPageSize(4096); // Changing the page size to 4 KB.
        storageCfg.setWriteThrottlingEnabled(true); // Enabling the writes throttling.

        igniteCfg.setDataStorageConfiguration(storageCfg);
        igniteCfg.setWorkDirectory(System.getProperty("user.dir") + "/work"); // System.getProperty("java.class.path")

        Ignite ignite = Ignition.start(igniteCfg);
        ignite.cluster().baselineAutoAdjustEnabled(false);

        // Activate a cluster automatically once all the nodes of the baseline topology have joined after a cluster restart.
        ignite.cluster().active(true);

        // Manually setting Baseline Topology
        Collection<ClusterNode> nodes = ignite.cluster().forServers().nodes();
        // Set all server nodes to baseline topology
        ignite.cluster().setBaselineTopology(nodes);

任何想法如何解决此问题?谢谢.在此处输入图片描述

Any idea how to resolve this issue? Thanks. enter image description here

推荐答案

据我所知，重新启动后会持续保留ulimits值，您应该在配置文件中进行设置:

As I know to persists the ulimits values across reboots you should set it in the configuration file:

/etc/security/limits.conf

/etc/security/limits.conf

它包含软" 和硬" 选项.root的硬选项，其他人的软选项.

It contains "soft" and "hard" options. Hard options for root, soft for others.

使用 ulimit 命令，您可以覆盖当前用户和会话的"soft" 值.可能没有存储您的限制，或者您设置了"soft" 选项，但是使用sudo命令启动GridGain，而您的"hard" 选项不正确.

Using ulimit command you can overwrite the "soft" values for current user and session. Probably your limits weren't stored or you set "soft" options but start the GridGain using sudo command and your "hard" options were incorrect.

能否请您再次检查并提供以下信息:

Could you please double-check and provide the next information:

1)您使用哪种操作系统?

1)What operation system is used by you?

2)您的环境中是否有/etc/security/limits.conf文件?

2)Do you have /etc/security/limits.conf file in your environment?

3)您是否有将启动Ignite的用户正确的值.如果您是在root用户下启动的，请检查硬"选项

3)Do you have correct values for user that will start the Ignite. In case if you started it under root then check the "hard" options

但是，我建议在此处设置以下选项:

However, I suggest to set the following options there:

ignite soft nofile 65536
ignite hard nofile 65536
ignite soft nproc 65536
ignite hard nproc 65536

其中 ignite 是用于启动Ignite的用户名.

Where ignite is the username that was used for Ignite starting.

这篇关于持久化时，Ubuntu VM中会发生异常:打开的文件过多的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！