在Stormcrawler中向元数据添加一些键时出现java.util.ConcurrentModificationException [英] java.util.ConcurrentModificationException when adding some key to metadata in stormcrawler
本文介绍了在Stormcrawler中向元数据添加一些键时出现java.util.ConcurrentModificationException的处理方法，对大家解决问题具有一定的参考价值，需要的朋友们下面随着小编来一起学习吧！
问题描述
我已经在元数据中添加了一个字段，用于传输和保留状态索引.该字段为字符串列表，名称为 input_keywords .在Strom群集中运行拓扑之后，拓扑停止，并显示以下日志:
java.lang.RuntimeException:com.esotericsoftware.kryo.KryoException:java.util.ConcurrentModificationException序列化跟踪:md(com.digitalpebble.stormcrawler.Metadata)在org.apache.storm.utils.DisruptorQueue.consumeBatchToCursor(DisruptorQueue.java:522)〜[storm-core-1.2.1.3.1.4.0-315.jar:1.2.1.3.1.4.0-315]在org.apache.storm.utils.DisruptorQueue.consumeBatchWhenAvailable(DisruptorQueue.java:487)〜[storm-core-1.2.1.3.1.4.0-315.jar:1.2.1.3.1.4.0-315]在org.apache.storm.disruptor $ consume_batch_when_available.invoke(disruptor.clj:74)〜[storm-core-1.2.1.3.1.4.0-315.jar:1.2.1.3.1.4.0-315]在org.apache.storm.disruptor $ consume_loop_STAR_ $ fn__4132.invoke(disruptor.clj:84)〜[storm-core-1.2.1.3.1.4.0-315.jar:1.2.1.3.1.4.0-315]在org.apache.storm.util $ async_loop $ fn__1221.invoke(util.clj:484)[storm-core-1.2.1.3.1.4.0-315.jar:1.2.1.3.1.4.0-315]在clojure.lang.AFn.run(AFn.java:22)[clojure-1.7.0.jar :?]在java.lang.Thread.run(Thread.java:745)[?:1.8.0_112]引起原因:com.esotericsoftware.kryo.KryoException:java.util.ConcurrentModificationException序列化跟踪:md(com.digitalpebble.stormcrawler.Metadata)在com.esotericsoftware.kryo.serializers.ObjectField.write(ObjectField.java:101)〜[kryo-3.0.3.jar :?]在com.esotericsoftware.kryo.serializers.FieldSerializer.write(FieldSerializer.java:518)〜[kryo-3.0.3.jar :?]在com.esotericsoftware.kryo.Kryo.writeClassAndObject(Kryo.java:628)〜[kryo-3.0.3.jar :?]在com.esotericsoftware.kryo.serializers.CollectionSerializer.write(CollectionSerializer.java:100)〜[kryo-3.0.3.jar :?]在com.esotericsoftware.kryo.serializers.CollectionSerializer.write(CollectionSerializer.java:40)〜[kryo-3.0.3.jar :?]在com.esotericsoftware.kryo.Kryo.writeObject(Kryo.java:534)〜[kryo-3.0.3.jar :?]在org.apache.storm.serialization.KryoValuesSerializer.serializeInto(KryoValuesSerializer.java:44)〜[storm-core-1.2.1.3.1.4.0-315.jar:1.2.1.3.1.4.0-315]在org.apache.storm.serialization.KryoTupleSerializer.serialize(KryoTupleSerializer.java:44)〜[storm-core-1.2.1.3.1.4.0-315.jar:1.2.1.3.1.4.0-315]在org.apache.storm.daemon.worker $ mk_transfer_fn $ transfer_fn__10378.invoke(worker.clj:203)〜[storm-core-1.2.1.3.1.4.0-315.jar:1.2.1.3.1.4.0-315]在org.apache.storm.daemon.executor $ start_batch_transfer_GT_worker_handler_BANG $ fn__10056.invoke(executor.clj:314)〜[storm-core-1.2.1.3.1.4.0-315.jar:1.2.1.3.1.4.0-315]在org.apache.storm.disruptor $ clojure_handler $ reify__4115.onEvent(disruptor.clj:41)〜[storm-core-1.2.1.3.1.4.0-315.jar:1.2.1.3.1.4.0-315]在org.apache.storm.utils.DisruptorQueue.consumeBatchToCursor(DisruptorQueue.java:509)〜[storm-core-1.2.1.3.1.4.0-315.jar:1.2.1.3.1.4.0-315]...另外6个引起原因:java.util.ConcurrentModificationException在java.util.HashMap $ HashIterator.nextNode(HashMap.java:1437)〜[?:1.8.0_112]在java.util.HashMap $ EntryIterator.next(HashMap.java:1471)〜[?:1.8.0_112]在java.util.HashMap $ EntryIterator.next(HashMap.java:1469)〜[?:1.8.0_112]在com.esotericsoftware.kryo.serializers.MapSerializer.write(MapSerializer.java:99)〜[kryo-3.0.3.jar :?]在com.esotericsoftware.kryo.serializers.MapSerializer.write(MapSerializer.java:39)〜[kryo-3.0.3.jar :?]在com.esotericsoftware.kryo.Kryo.writeObject(Kryo.java:552)〜[kryo-3.0.3.jar :?]在com.esotericsoftware.kryo.serializers.ObjectField.write(ObjectField.java:80)〜[kryo-3.0.3.jar :?]在com.esotericsoftware.kryo.serializers.FieldSerializer.write(FieldSerializer.java:518)〜[kryo-3.0.3.jar :?]在com.esotericsoftware.kryo.Kryo.writeClassAndObject(Kryo.java:628)〜[kryo-3.0.3.jar :?]在com.esotericsoftware.kryo.serializers.CollectionSerializer.write(CollectionSerializer.java:100)〜[kryo-3.0.3.jar :?]在com.esotericsoftware.kryo.serializers.CollectionSerializer.write(CollectionSerializer.java:40)〜[kryo-3.0.3.jar :?]在com.esotericsoftware.kryo.Kryo.writeObject(Kryo.java:534)〜[kryo-3.0.3.jar :?]在org.apache.storm.serialization.KryoValuesSerializer.serializeInto(KryoValuesSerializer.java:44)〜[storm-core-1.2.1.3.1.4.0-315.jar:1.2.1.3.1.4.0-315]在org.apache.storm.serialization.KryoTupleSerializer.serialize(KryoTupleSerializer.java:44)〜[storm-core-1.2.1.3.1.4.0-315.jar:1.2.1.3.1.4.0-315]在org.apache.storm.daemon.worker $ mk_transfer_fn $ transfer_fn__10378.invoke(worker.clj:203)〜[storm-core-1.2.1.3.1.4.0-315.jar:1.2.1.3.1.4.0-315]在org.apache.storm.daemon.executor $ start_batch_transfer_GT_worker_handler_BANG $ fn__10056.invoke(executor.clj:314)〜[storm-core-1.2.1.3.1.4.0-315.jar:1.2.1.3.1.4.0-315]在org.apache.storm.disruptor $ clojure_handler $ reify__4115.onEvent(disruptor.clj:41)〜[storm-core-1.2.1.3.1.4.0-315.jar:1.2.1.3.1.4.0-315]在org.apache.storm.utils.DisruptorQueue.consumeBatchToCursor(DisruptorQueue.java:509)〜[storm-core-1.2.1.3.1.4.0-315.jar:1.2.1.3.1.4.0-315]...另外6个2021-02-27 08:03:34.276 o.a.s.d.executor线程20-disruptor-executor [45 45]-发送队列[ERROR]java.lang.RuntimeException:com.esotericsoftware.kryo.KryoException:java.util.ConcurrentModificationException序列化跟踪:md(com.digitalpebble.stormcrawler.Metadata)在org.apache.storm.utils.DisruptorQueue.consumeBatchToCursor(DisruptorQueue.java:522)〜[storm-core-1.2.1.3.1.4.0-315.jar:1.2.1.3.1.4.0-315]在org.apache.storm.utils.DisruptorQueue.consumeBatchWhenAvailable(DisruptorQueue.java:487)〜[storm-core-1.2.1.3.1.4.0-315.jar:1.2.1.3.1.4.0-315]在org.apache.storm.disruptor $ consume_batch_when_available.invoke(disruptor.clj:74)〜[storm-core-1.2.1.3.1.4.0-315.jar:1.2.1.3.1.4.0-315]在org.apache.storm.disruptor $ consume_loop_STAR_ $ fn__4132.invoke(disruptor.clj:84)〜[storm-core-1.2.1.3.1.4.0-315.jar:1.2.1.3.1.4.0-315]在org.apache.storm.util $ async_loop $ fn__1221.invoke(util.clj:484)[storm-core-1.2.1.3.1.4.0-315.jar:1.2.1.3.1.4.0-315]在clojure.lang.AFn.run(AFn.java:22)[clojure-1.7.0.jar :?]在java.lang.Thread.run(Thread.java:745)[?:1.8.0_112]引起原因:com.esotericsoftware.kryo.KryoException:java.util.ConcurrentModificationException序列化跟踪:md(com.digitalpebble.stormcrawler.Metadata)在com.esotericsoftware.kryo.serializers.ObjectField.write(ObjectField.java:101)〜[kryo-3.0.3.jar :?]在com.esotericsoftware.kryo.serializers.FieldSerializer.write(FieldSerializer.java:518)〜[kryo-3.0.3.jar :?]在com.esotericsoftware.kryo.Kryo.writeClassAndObject(Kryo.java:628)〜[kryo-3.0.3.jar :?]在com.esotericsoftware.kryo.serializers.CollectionSerializer.write(CollectionSerializer.java:100)〜[kryo-3.0.3.jar :?]在com.esotericsoftware.kryo.serializers.CollectionSerializer.write(CollectionSerializer.java:40)〜[kryo-3.0.3.jar :?]在com.esotericsoftware.kryo.Kryo.writeObject(Kryo.java:534)〜[kryo-3.0.3.jar :?]在org.apache.storm.serialization.KryoValuesSerializer.serializeInto(KryoValuesSerializer.java:44)〜[storm-core-1.2.1.3.1.4.0-315.jar:1.2.1.3.1.4.0-315]在org.apache.storm.serialization.KryoTupleSerializer.serialize(KryoTupleSerializer.java:44)〜[storm-core-1.2.1.3.1.4.0-315.jar:1.2.1.3.1.4.0-315]在org.apache.storm.daemon.worker $ mk_transfer_fn $ transfer_fn__10378.invoke(worker.clj:203)〜[storm-core-1.2.1.3.1.4.0-315.jar:1.2.1.3.1.4.0-315]在org.apache.storm.daemon.executor $ start_batch_transfer_GT_worker_handler_BANG $ fn__10056.invoke(executor.clj:314)〜[storm-core-1.2.1.3.1.4.0-315.jar:1.2.1.3.1.4.0-315]在org.apache.storm.disruptor $ clojure_handler $ reify__4115.onEvent(disruptor.clj:41)〜[storm-core-1.2.1.3.1.4.0-315.jar:1.2.1.3.1.4.0-315]在org.apache.storm.utils.DisruptorQueue.consumeBatchToCursor(DisruptorQueue.java:509)〜[storm-core-1.2.1.3.1.4.0-315.jar:1.2.1.3.1.4.0-315]...另外6个引起原因:java.util.ConcurrentModificationException在java.util.HashMap $ HashIterator.nextNode(HashMap.java:1437)〜[?:1.8.0_112]在java.util.HashMap $ EntryIterator.next(HashMap.java:1471)〜[?:1.8.0_112]在java.util.HashMap $ EntryIterator.next(HashMap.java:1469)〜[?:1.8.0_112]在com.esotericsoftware.kryo.serializers.MapSerializer.write(MapSerializer.java:99)〜[kryo-3.0.3.jar :?]在com.esotericsoftware.kryo.serializers.MapSerializer.write(MapSerializer.java:39)〜[kryo-3.0.3.jar :?]在com.esotericsoftware.kryo.Kryo.writeObject(Kryo.java:552)〜[kryo-3.0.3.jar :?]在com.esotericsoftware.kryo.serializers.ObjectField.write(ObjectField.java:80)〜[kryo-3.0.3.jar :?]在com.esotericsoftware.kryo.serializers.FieldSerializer.write(FieldSerializer.java:518)〜[kryo-3.0.3.jar :?]在com.esotericsoftware.kryo.Kryo.writeClassAndObject(Kryo.java:628)〜[kryo-3.0.3.jar :?]在com.esotericsoftware.kryo.serializers.CollectionSerializer.write(CollectionSerializer.java:100)〜[kryo-3.0.3.jar :?]在com.esotericsoftware.kryo.serializers.CollectionSerializer.write(CollectionSerializer.java:40)〜[kryo-3.0.3.jar :?]在com.esotericsoftware.kryo.Kryo.writeObject(Kryo.java:534)〜[kryo-3.0.3.jar :?]在org.apache.storm.serialization.KryoValuesSerializer.serializeInto(KryoValuesSerializer.java:44)〜[storm-core-1.2.1.3.1.4.0-315.jar:1.2.1.3.1.4.0-315]在org.apache.storm.serialization.KryoTupleSerializer.serialize(KryoTupleSerializer.java:44)〜[storm-core-1.2.1.3.1.4.0-315.jar:1.2.1.3.1.4.0-315]在org.apache.storm.daemon.worker $ mk_transfer_fn $ transfer_fn__10378.invoke(worker.clj:203)〜[storm-core-1.2.1.3.1.4.0-315.jar:1.2.1.3.1.4.0-315]在org.apache.storm.daemon.executor $ start_batch_transfer_GT_worker_handler_BANG $ fn__10056.invoke(executor.clj:314)〜[storm-core-1.2.1.3.1.4.0-315.jar:1.2.1.3.1.4.0-315]在org.apache.storm.disruptor $ clojure_handler $ reify__4115.onEvent(disruptor.clj:41)〜[storm-core-1.2.1.3.1.4.0-315.jar:1.2.1.3.1.4.0-315]在org.apache.storm.utils.DisruptorQueue.consumeBatchToCursor(DisruptorQueue.java:509)〜[storm-core-1.2.1.3.1.4.0-315.jar:1.2.1.3.1.4.0-315]...另外6个2021-02-27 08:03:34.327 o.a.s.util Thread-20-disruptor-executor [45 45] -send-queue [ERROR]暂停过程:(工人死亡")java.lang.RuntimeException :(工作人员死亡")在org.apache.storm.util $ exit_process_BANG_.doInvoke(util.clj:341)[storm-core-1.2.1.3.1.4.0-315.jar:1.2.1.3.1.4.0-315]在clojure.lang.RestFn.invoke(RestFn.java:423)[clojure-1.7.0.jar :?]在org.apache.storm.daemon.worker $ fn_10827 $ fn_10828.invoke(worker.clj:781)[storm-core-1.2.1.3.1.4.0-315.jar:1.2.1.3.1.4.0-315]在org.apache.storm.daemon.executor $ mk_executor_data $ fn_10034 $ fn_10035.invoke(executor.clj:281)[storm-core-1.2.1.3.1.4.0-315.jar:1.2.1.3.1.4.0-315]在org.apache.storm.util $ async_loop $ fn__1221.invoke(util.clj:494)[storm-core-1.2.1.3.1.4.0-315.jar:1.2.1.3.1.4.0-315]在clojure.lang.AFn.run(AFn.java:22)[clojure-1.7.0.jar :?]在java.lang.Thread.run(Thread.java:745)[?:1.8.0_112] 对于拓扑的每个组件，我们都有不同的并行提示.将 input_keywords 添加到元数据后，我们得到了错误.错误的主要原因是什么?
解决方案
您正在序列化Metadata实例时对其进行修改.您无法做到这一点，请参阅风暴故障排除页面.
如 1.16版本说明中所述，您可以锁定元数据.这不会解决问题，但会告诉您要将代码写入元数据的位置.
在我们的拓扑中，我们将相同的元数据同时发射到多个螺栓. 神秘解释.不要那样做.
I have added a field to metadata for transferring and persisting in the status index. The field is a List of String and its name is input_keywords. After running topology in the Strom cluster, The topology halted with the following logs:
java.lang.RuntimeException: com.esotericsoftware.kryo.KryoException: java.util.ConcurrentModificationException
Serialization trace:
md (com.digitalpebble.stormcrawler.Metadata)
at org.apache.storm.utils.DisruptorQueue.consumeBatchToCursor(DisruptorQueue.java:522) ~[storm-core-1.2.1.3.1.4.0-315.jar:1.2.1.3.1.4.0-315]
at org.apache.storm.utils.DisruptorQueue.consumeBatchWhenAvailable(DisruptorQueue.java:487) ~[storm-core-1.2.1.3.1.4.0-315.jar:1.2.1.3.1.4.0-315]
at org.apache.storm.disruptor$consume_batch_when_available.invoke(disruptor.clj:74) ~[storm-core-1.2.1.3.1.4.0-315.jar:1.2.1.3.1.4.0-315]
at org.apache.storm.disruptor$consume_loop_STAR_$fn__4132.invoke(disruptor.clj:84) ~[storm-core-1.2.1.3.1.4.0-315.jar:1.2.1.3.1.4.0-315]
at org.apache.storm.util$async_loop$fn__1221.invoke(util.clj:484) [storm-core-1.2.1.3.1.4.0-315.jar:1.2.1.3.1.4.0-315]
at clojure.lang.AFn.run(AFn.java:22) [clojure-1.7.0.jar:?]
at java.lang.Thread.run(Thread.java:745) [?:1.8.0_112]
Caused by: com.esotericsoftware.kryo.KryoException: java.util.ConcurrentModificationException
Serialization trace:
md (com.digitalpebble.stormcrawler.Metadata)
at com.esotericsoftware.kryo.serializers.ObjectField.write(ObjectField.java:101) ~[kryo-3.0.3.jar:?]
at com.esotericsoftware.kryo.serializers.FieldSerializer.write(FieldSerializer.java:518) ~[kryo-3.0.3.jar:?]
at com.esotericsoftware.kryo.Kryo.writeClassAndObject(Kryo.java:628) ~[kryo-3.0.3.jar:?]
at com.esotericsoftware.kryo.serializers.CollectionSerializer.write(CollectionSerializer.java:100) ~[kryo-3.0.3.jar:?]
at com.esotericsoftware.kryo.serializers.CollectionSerializer.write(CollectionSerializer.java:40) ~[kryo-3.0.3.jar:?]
at com.esotericsoftware.kryo.Kryo.writeObject(Kryo.java:534) ~[kryo-3.0.3.jar:?]
at org.apache.storm.serialization.KryoValuesSerializer.serializeInto(KryoValuesSerializer.java:44) ~[storm-core-1.2.1.3.1.4.0-315.jar:1.2.1.3.1.4.0-315]
at org.apache.storm.serialization.KryoTupleSerializer.serialize(KryoTupleSerializer.java:44) ~[storm-core-1.2.1.3.1.4.0-315.jar:1.2.1.3.1.4.0-315]
at org.apache.storm.daemon.worker$mk_transfer_fn$transfer_fn__10378.invoke(worker.clj:203) ~[storm-core-1.2.1.3.1.4.0-315.jar:1.2.1.3.1.4.0-315]
at org.apache.storm.daemon.executor$start_batch_transfer_GT_worker_handler_BANG$fn__10056.invoke(executor.clj:314) ~[storm-core-1.2.1.3.1.4.0-315.jar:1.2.1.3.1.4.0-315]
at org.apache.storm.disruptor$clojure_handler$reify__4115.onEvent(disruptor.clj:41) ~[storm-core-1.2.1.3.1.4.0-315.jar:1.2.1.3.1.4.0-315]
at org.apache.storm.utils.DisruptorQueue.consumeBatchToCursor(DisruptorQueue.java:509) ~[storm-core-1.2.1.3.1.4.0-315.jar:1.2.1.3.1.4.0-315]
... 6 more
Caused by: java.util.ConcurrentModificationException
at java.util.HashMap$HashIterator.nextNode(HashMap.java:1437) ~[?:1.8.0_112]
at java.util.HashMap$EntryIterator.next(HashMap.java:1471) ~[?:1.8.0_112]
at java.util.HashMap$EntryIterator.next(HashMap.java:1469) ~[?:1.8.0_112]
at com.esotericsoftware.kryo.serializers.MapSerializer.write(MapSerializer.java:99) ~[kryo-3.0.3.jar:?]
at com.esotericsoftware.kryo.serializers.MapSerializer.write(MapSerializer.java:39) ~[kryo-3.0.3.jar:?]
at com.esotericsoftware.kryo.Kryo.writeObject(Kryo.java:552) ~[kryo-3.0.3.jar:?]
at com.esotericsoftware.kryo.serializers.ObjectField.write(ObjectField.java:80) ~[kryo-3.0.3.jar:?]
at com.esotericsoftware.kryo.serializers.FieldSerializer.write(FieldSerializer.java:518) ~[kryo-3.0.3.jar:?]
at com.esotericsoftware.kryo.Kryo.writeClassAndObject(Kryo.java:628) ~[kryo-3.0.3.jar:?]
at com.esotericsoftware.kryo.serializers.CollectionSerializer.write(CollectionSerializer.java:100) ~[kryo-3.0.3.jar:?]
at com.esotericsoftware.kryo.serializers.CollectionSerializer.write(CollectionSerializer.java:40) ~[kryo-3.0.3.jar:?]
at com.esotericsoftware.kryo.Kryo.writeObject(Kryo.java:534) ~[kryo-3.0.3.jar:?]
at org.apache.storm.serialization.KryoValuesSerializer.serializeInto(KryoValuesSerializer.java:44) ~[storm-core-1.2.1.3.1.4.0-315.jar:1.2.1.3.1.4.0-315]
at org.apache.storm.serialization.KryoTupleSerializer.serialize(KryoTupleSerializer.java:44) ~[storm-core-1.2.1.3.1.4.0-315.jar:1.2.1.3.1.4.0-315]
at org.apache.storm.daemon.worker$mk_transfer_fn$transfer_fn__10378.invoke(worker.clj:203) ~[storm-core-1.2.1.3.1.4.0-315.jar:1.2.1.3.1.4.0-315]
at org.apache.storm.daemon.executor$start_batch_transfer_GT_worker_handler_BANG$fn__10056.invoke(executor.clj:314) ~[storm-core-1.2.1.3.1.4.0-315.jar:1.2.1.3.1.4.0-315]
at org.apache.storm.disruptor$clojure_handler$reify__4115.onEvent(disruptor.clj:41) ~[storm-core-1.2.1.3.1.4.0-315.jar:1.2.1.3.1.4.0-315]
at org.apache.storm.utils.DisruptorQueue.consumeBatchToCursor(DisruptorQueue.java:509) ~[storm-core-1.2.1.3.1.4.0-315.jar:1.2.1.3.1.4.0-315]
... 6 more
2021-02-27 08:03:34.276 o.a.s.d.executor Thread-20-disruptor-executor[45 45]-send-queue [ERROR]
java.lang.RuntimeException: com.esotericsoftware.kryo.KryoException: java.util.ConcurrentModificationException
Serialization trace:
md (com.digitalpebble.stormcrawler.Metadata)
at org.apache.storm.utils.DisruptorQueue.consumeBatchToCursor(DisruptorQueue.java:522) ~[storm-core-1.2.1.3.1.4.0-315.jar:1.2.1.3.1.4.0-315]
at org.apache.storm.utils.DisruptorQueue.consumeBatchWhenAvailable(DisruptorQueue.java:487) ~[storm-core-1.2.1.3.1.4.0-315.jar:1.2.1.3.1.4.0-315]
at org.apache.storm.disruptor$consume_batch_when_available.invoke(disruptor.clj:74) ~[storm-core-1.2.1.3.1.4.0-315.jar:1.2.1.3.1.4.0-315]
at org.apache.storm.disruptor$consume_loop_STAR_$fn__4132.invoke(disruptor.clj:84) ~[storm-core-1.2.1.3.1.4.0-315.jar:1.2.1.3.1.4.0-315]
at org.apache.storm.util$async_loop$fn__1221.invoke(util.clj:484) [storm-core-1.2.1.3.1.4.0-315.jar:1.2.1.3.1.4.0-315]
at clojure.lang.AFn.run(AFn.java:22) [clojure-1.7.0.jar:?]
at java.lang.Thread.run(Thread.java:745) [?:1.8.0_112]
Caused by: com.esotericsoftware.kryo.KryoException: java.util.ConcurrentModificationException
Serialization trace:
md (com.digitalpebble.stormcrawler.Metadata)
at com.esotericsoftware.kryo.serializers.ObjectField.write(ObjectField.java:101) ~[kryo-3.0.3.jar:?]
at com.esotericsoftware.kryo.serializers.FieldSerializer.write(FieldSerializer.java:518) ~[kryo-3.0.3.jar:?]
at com.esotericsoftware.kryo.Kryo.writeClassAndObject(Kryo.java:628) ~[kryo-3.0.3.jar:?]
at com.esotericsoftware.kryo.serializers.CollectionSerializer.write(CollectionSerializer.java:100) ~[kryo-3.0.3.jar:?]
at com.esotericsoftware.kryo.serializers.CollectionSerializer.write(CollectionSerializer.java:40) ~[kryo-3.0.3.jar:?]
at com.esotericsoftware.kryo.Kryo.writeObject(Kryo.java:534) ~[kryo-3.0.3.jar:?]
at org.apache.storm.serialization.KryoValuesSerializer.serializeInto(KryoValuesSerializer.java:44) ~[storm-core-1.2.1.3.1.4.0-315.jar:1.2.1.3.1.4.0-315]
at org.apache.storm.serialization.KryoTupleSerializer.serialize(KryoTupleSerializer.java:44) ~[storm-core-1.2.1.3.1.4.0-315.jar:1.2.1.3.1.4.0-315]
at org.apache.storm.daemon.worker$mk_transfer_fn$transfer_fn__10378.invoke(worker.clj:203) ~[storm-core-1.2.1.3.1.4.0-315.jar:1.2.1.3.1.4.0-315]
at org.apache.storm.daemon.executor$start_batch_transfer_GT_worker_handler_BANG$fn__10056.invoke(executor.clj:314) ~[storm-core-1.2.1.3.1.4.0-315.jar:1.2.1.3.1.4.0-315]
at org.apache.storm.disruptor$clojure_handler$reify__4115.onEvent(disruptor.clj:41) ~[storm-core-1.2.1.3.1.4.0-315.jar:1.2.1.3.1.4.0-315]
at org.apache.storm.utils.DisruptorQueue.consumeBatchToCursor(DisruptorQueue.java:509) ~[storm-core-1.2.1.3.1.4.0-315.jar:1.2.1.3.1.4.0-315]
... 6 more
Caused by: java.util.ConcurrentModificationException
at java.util.HashMap$HashIterator.nextNode(HashMap.java:1437) ~[?:1.8.0_112]
at java.util.HashMap$EntryIterator.next(HashMap.java:1471) ~[?:1.8.0_112]
at java.util.HashMap$EntryIterator.next(HashMap.java:1469) ~[?:1.8.0_112]
at com.esotericsoftware.kryo.serializers.MapSerializer.write(MapSerializer.java:99) ~[kryo-3.0.3.jar:?]
at com.esotericsoftware.kryo.serializers.MapSerializer.write(MapSerializer.java:39) ~[kryo-3.0.3.jar:?]
at com.esotericsoftware.kryo.Kryo.writeObject(Kryo.java:552) ~[kryo-3.0.3.jar:?]
at com.esotericsoftware.kryo.serializers.ObjectField.write(ObjectField.java:80) ~[kryo-3.0.3.jar:?]
at com.esotericsoftware.kryo.serializers.FieldSerializer.write(FieldSerializer.java:518) ~[kryo-3.0.3.jar:?]
at com.esotericsoftware.kryo.Kryo.writeClassAndObject(Kryo.java:628) ~[kryo-3.0.3.jar:?]
at com.esotericsoftware.kryo.serializers.CollectionSerializer.write(CollectionSerializer.java:100) ~[kryo-3.0.3.jar:?]
at com.esotericsoftware.kryo.serializers.CollectionSerializer.write(CollectionSerializer.java:40) ~[kryo-3.0.3.jar:?]
at com.esotericsoftware.kryo.Kryo.writeObject(Kryo.java:534) ~[kryo-3.0.3.jar:?]
at org.apache.storm.serialization.KryoValuesSerializer.serializeInto(KryoValuesSerializer.java:44) ~[storm-core-1.2.1.3.1.4.0-315.jar:1.2.1.3.1.4.0-315]
at org.apache.storm.serialization.KryoTupleSerializer.serialize(KryoTupleSerializer.java:44) ~[storm-core-1.2.1.3.1.4.0-315.jar:1.2.1.3.1.4.0-315]
at org.apache.storm.daemon.worker$mk_transfer_fn$transfer_fn__10378.invoke(worker.clj:203) ~[storm-core-1.2.1.3.1.4.0-315.jar:1.2.1.3.1.4.0-315]
at org.apache.storm.daemon.executor$start_batch_transfer_GT_worker_handler_BANG$fn__10056.invoke(executor.clj:314) ~[storm-core-1.2.1.3.1.4.0-315.jar:1.2.1.3.1.4.0-315]
at org.apache.storm.disruptor$clojure_handler$reify__4115.onEvent(disruptor.clj:41) ~[storm-core-1.2.1.3.1.4.0-315.jar:1.2.1.3.1.4.0-315]
at org.apache.storm.utils.DisruptorQueue.consumeBatchToCursor(DisruptorQueue.java:509) ~[storm-core-1.2.1.3.1.4.0-315.jar:1.2.1.3.1.4.0-315]
... 6 more
2021-02-27 08:03:34.327 o.a.s.util Thread-20-disruptor-executor[45 45]-send-queue [ERROR] Halting process: ("Worker died")
java.lang.RuntimeException: ("Worker died")
at org.apache.storm.util$exit_process_BANG_.doInvoke(util.clj:341) [storm-core-1.2.1.3.1.4.0-315.jar:1.2.1.3.1.4.0-315]
at clojure.lang.RestFn.invoke(RestFn.java:423) [clojure-1.7.0.jar:?]
at org.apache.storm.daemon.worker$fn_10827$fn_10828.invoke(worker.clj:781) [storm-core-1.2.1.3.1.4.0-315.jar:1.2.1.3.1.4.0-315]
at org.apache.storm.daemon.executor$mk_executor_data$fn_10034$fn_10035.invoke(executor.clj:281) [storm-core-1.2.1.3.1.4.0-315.jar:1.2.1.3.1.4.0-315]
at org.apache.storm.util$async_loop$fn__1221.invoke(util.clj:494) [storm-core-1.2.1.3.1.4.0-315.jar:1.2.1.3.1.4.0-315]
at clojure.lang.AFn.run(AFn.java:22) [clojure-1.7.0.jar:?]
at java.lang.Thread.run(Thread.java:745) [?:1.8.0_112]
We have different parallelism hints for each components of topology. After adding the input_keywords to metadata we got the error. What is the main reason of the Error?
解决方案
You are modifying a Metadata instance while it is being serialized. You can't do that, see Storm troubleshooting page.
As explained in the release notes of 1.16, you can lock the metadata. This won't fix the issue but will tell you where in your code you are writing into the metadata.
In our topology, we emitted the same metadata to multiple bolts at the same time.
mystery explained. don't do that.
这篇关于在Stormcrawler中向元数据添加一些键时出现java.util.ConcurrentModificationException的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！
查看全文
