物理内存使用率过高 [英] Physical memory usage is too high

问题描述

当我尝试使用

mvn compile exec:java -Dexec.classpathScope=compile -Dexec.mainClass=my.Topology

错误是

ERROR backtype.storm.util - Async loop died!
java.lang.OutOfMemoryError: Physical memory usage is too high: physicalBytes = 3G > maxPhysicalBytes = 3G

我该如何解决?我不知道我应该增加哪个物理内存！如果我在生产模式下运行拓扑，这个错误会消失吗?

How can I solve it? I don't know which Physical memory I should increase ! and if I run the topology in production mode, will this error be disappeared?

更新

Physical Memory Array
Location: System Board Or Motherboard
Use: System Memory
Error Correction Type: None
Maximum Capacity: 32 GB
Error Information Handle: 0x0019
Number Of Devices: 4

推荐答案

我也在使用 Apache Storm 和 JavaCV (OpenCV).我有两个拓扑，过程中的第二个拓扑有两个螺栓，一个用于将视频拆分为帧，另一个用于检测人脸.

I'm also using Apache Storm with JavaCV (OpenCV). I have two Topologies, and the second Topology on the process has two bolts, one for split a video into frames, and another bolt for detecting faces.

我遇到了同样的问题:

2017-08-02 11:19:18.578 o.a.s.util Thread-5-OpenCVBolt-executor[3 3] [ERROR] 
Async loop died!
java.lang.OutOfMemoryError: Physical memory usage is too high: physicalBytes = 1G > maxPhysicalBytes = 1G
at org.bytedeco.javacpp.Pointer.deallocator(Pointer.java:562) ~[stormjar.jar:?]
at org.bytedeco.javacpp.helper.opencv_core$AbstractCvMemStorage.create(opencv_core.java:1649) ~[stormjar.jar:?]
at org.bytedeco.javacpp.helper.opencv_core$AbstractCvMemStorage.create(opencv_core.java:1658) ~[stormjar.jar:?]
at OpenCVBolt.detect(OpenCVBolt.java:30) ~[stormjar.jar:?]
at OpenCVBolt.execute(OpenCVBolt.java:104) ~[stormjar.jar:?]
at org.apache.storm.daemon.executor$fn__4973$tuple_action_fn__4975.invoke(executor.clj:727) ~[storm-core-1.0.3.jar:1.0.3]
at org.apache.storm.daemon.executor$mk_task_receiver$fn__4894.invoke(executor.clj:459) ~[storm-core-1.0.3.jar:1.0.3]
at org.apache.storm.disruptor$clojure_handler$reify__4409.onEvent(disruptor.clj:40) ~[storm-core-1.0.3.jar:1.0.3]
at org.apache.storm.utils.DisruptorQueue.consumeBatchToCursor(DisruptorQueue.java:453) ~[storm-core-1.0.3.jar:1.0.3]

我能够解决它.我不知道您是否使用 JavaCV 来处理视频和图像.如果是这样，并且您正在使用 Maven，请确保您在 pom.xml 中使用 JavaCV 1.3.2 版:

I was able to solve it. I don't know if you are using JavaCV to work with video and images. If so, and you are working with Maven, make sure you work with JavaCV version 1.3.2 in your pom.xml:

<dependency>
  <groupId>org.bytedeco</groupId>
  <artifactId>javacv</artifactId>
  <version>1.3.2</version>
</dependency>

然后，您需要在 Bolt 的 prepare() 方法中应用以下几行来更改 maxPhysicalBytes 的属性.

Then, you need to apply the following lines in the prepare() method in your Bolt to change the property of maxPhysicalBytes.

System.setProperty("org.bytedeco.javacpp.maxphysicalbytes", "0"); 
System.setProperty("org.bytedeco.javacpp.maxbytes", "0");

这对我有用.错误已经消失.我希望这对您有所帮助.

That's work for me. The error has disappeared. I hope this helps you.

更新

@Override
public void prepare(Map conf, TopologyContext context, OutputCollector collector) {

    System.setProperty("org.bytedeco.javacpp.maxphysicalbytes", "0");
    System.setProperty("org.bytedeco.javacpp.maxbytes", "0");

    _collector = collector;
}

这篇关于物理内存使用率过高的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！