给我的 jar 文件更多的内存 [英] Give more memory to my jar file

问题描述

我有一个多线程爬虫.在这个程序中，如果我加载了很多种子，就会出现错误.我看到 java.lang.OutOfMemoryError 并认为可能内存不够.我尝试使用以下参数运行 crawler.jar 文件:java -Xms512m -Xmx3G -jar crawler.jar 但到目前为止，没有运气.

I have a multithreaded crawler. In this program, if I load a lot of seeds, I get an error. I saw the java.lang.OutOfMemoryError and thought maybe the memory is not enough. I tried running the crawler.jar file with these arguments: java -Xms512m -Xmx3G -jar crawler.jar but so far, no luck.

这是程序的堆栈跟踪:

Exception in thread "main" java.lang.reflect.InvocationTargetException
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
        at java.lang.reflect.Method.invoke(Method.java:597)
        at org.eclipse.jdt.internal.jarinjarloader.JarRsrcLoader.main(JarRsrcLoader.java:58)
Caused by: java.lang.OutOfMemoryError: unable to create new native thread
        at java.lang.Thread.start0(Native Method)
        at java.lang.Thread.start(Thread.java:597)
        at com.sleepycat.je.utilint.DaemonThread.runOrPause(DaemonThread.java:99)
        at com.sleepycat.je.dbi.EnvironmentImpl.runOrPauseDaemons(EnvironmentImpl.java:772)
        at com.sleepycat.je.dbi.EnvironmentImpl.envConfigUpdate(EnvironmentImpl.java:717)
        at com.sleepycat.je.dbi.EnvironmentImpl.finishInit(EnvironmentImpl.java:579)
        at com.sleepycat.je.dbi.DbEnvPool.getEnvironment(DbEnvPool.java:204)
        at com.sleepycat.je.Environment.makeEnvironmentImpl(Environment.java:230)
        at com.sleepycat.je.Environment.<init>(Environment.java:212)
        at com.sleepycat.je.Environment.<init>(Environment.java:166)
        ...

这与我怀疑的记忆有关吗?当我使用 java -jar 运行 jar 文件时，添加 -Xms512m -Xmx3G 是否有效?

Is this related to memory as I suspected? Does adding the -Xms512m -Xmx3G work when I'm running the jar file using java -jar?

我运行了任务管理器(我在 Windows Server 上运行)，但在运行应用程序后，内存并没有变得更高！我错了吗?

I ran task manager (I'm running on Windows Server) but after running the app, the memory doesn't go that far higher! Am I wrong?

推荐答案

-Xms512m -Xmx3G 选项只会影响 JVM 的堆大小，并不能解决您的问题.

The -Xms512m -Xmx3G option only effect the heap size of JVM and would not solve your problem.

但是，默认线程数限制在大多数情况下应该足够了.您可以通过调整 JVM/系统选项来增加限制，但是无论您创建多少线程，您的系统容量都受限于您的计算机资源.例如cpu、内存、网络等

However, the default thread number limits should be enough in most scenarios. You can increase the limit by tweaking JVM/System options, but no matter how many thread you create, the capacity of your system is bounded to your computer resource. e.g. cpu, memory, network, etc.

我建议换个角度解决这个问题:

I suggest to solve this problem from a different view:

• 尝试在工作线程之间共享 Environment 实例.
• 尝试将并发级别控制为系统繁忙且不触及线程数限制的速率.

这篇关于给我的 jar 文件更多的内存的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！