找到了多个defaults.yaml资源 [英] Found multiple defaults.yaml resources

问题描述

当我尝试提交拓扑时，我发现这个

when i tried to submit the topology i found this

Exception in thread "main" java.lang.RuntimeException: Found multiple defaults.yaml resources. You're probably bundling the Storm jars with your topology jar.
at backtype.storm.utils.Utils.findAndReadConfigFile(Utils.java:115)
at backtype.storm.utils.Utils.readDefaultConfig(Utils.java:135)
at backtype.storm.utils.Utils.readStormConfig(Utils.java:155)
at backtype.storm.StormSubmitter.submitTopology(StormSubmitter.java:61)
at backtype.storm.StormSubmitter.submitTopology(StormSubmitter.java:40)
at trident.myproject.main(myproject.java:288)

但是这个错误在pom.xml中更新后出现

But this error appeared after updated in pom.xml by

< scope> compile< / scope>而不是< scope>提供< / scope>

因为我是错误

An exception occured while executing the Java class. storm/trident/state/StateFactory

这里是pom文件

<plugins>
        <plugin>
            <artifactId>maven-assembly-plugin</artifactId>
            <configuration>
                <descriptorRefs>
                    <descriptorRef>jar-with-dependencies</descriptorRef>
                </descriptorRefs>
                <archive>
                    <manifest>
                        <mainClass>trident.myproject</mainClass>
                        <!-- <mainClass>crawler.Crawler</mainClass> -->
                    </manifest>
                </archive>
            </configuration>

pom文件的第2部分

part 2 of pom file

<executions>
                <execution>
                    <id>make-assembly</id>
                    <phase>package</phase>
                    <goals>
                        <goal>single</goal>
                    </goals>
                </execution>
            </executions>
        </plugin>

pom文件的第3部分

part 3 of pom file

<plugin>
            <groupId>org.apache.maven.plugins</groupId>
            <artifactId>maven-compiler-plugin</artifactId>
            <configuration>
                <source>1.6</source>
                <target>1.6</target>
            </configuration>
        </plugin>
    </plugins>

推荐答案

在中运行拓扑有根本区别code> LocalCluster 或远程通过 StormSubmitter （这是项目中的默认设置）。

There is a fundamental difference in running a topology in LocalCluster or remotely via StormSubmitter (which is the default setting in the project).

storm-core 的范围设置为< scope>提供< / scope> 是默认值，因为无论如何这些类文件在集群中都可用。 提供告诉maven，这些类不能包含在汇编的 jar 文件中，从而减少了罐子。此外，如果多次提供文件，这可以避免冲突 - 如果您将范围更改为 compile default.yaml 会发生这种情况。 C $ C>。对于这些情况，来自 storm-core 的所有文件都打包到 jar 中并提交给集群。 Storm找到文件 defaults.yaml 本地（即，在群集中的工作机器上本地）和 jar 。因此，Storm不知道使用哪一个并引发错误。

The scope of storm-core is set to <scope>provided</scope> be default because those class files are available in the cluster anyway. provided tells maven, that those classes must not be included into the jar file that is assembled, thus reducing the size of the jar. Furthermore, this avoids conflicts if files are provided multiple times -- that is what happens with default.yaml if you change the scope to compile. For those case, all files from storm-core are packaged into you jar and submitted to the cluster. Storm finds the file defaults.yaml "locally" (ie, locally on the worker machine in the cluster) and in your jar. Thus, Storm does not know which one to use and raises an error.

然而，提供排除那些类文件如果你也在本地运行。当然，本地这些文件不能自动使用，但在启动本地JVM时必须包含在 CLASSPATH 中。由于提供从 storm-core 中排除文件，您将获得 ClassNotFound exception。

However, provided excludes those class files if you run locally, too. Of course, locally those files are not available automatically but must be included in CLASSPATH when starting up the local JVM. As provided excludes the files from storm-core you get the ClassNotFound exception.

作为每次要提交到其他环境时更改范围的替代方法，可以将范围设置为 compile 并在 maven-jar-plugin 设置中明确包含拓扑Main / Bolt / Spout类。这个显式包含自动从jar中排除所有其他文件，即来自 storm-core 的所有文件。

As an alternative to change the scope each time you want to submit to a different environment, you can set the scope to compile and include your topology Main/Bolt/Spout classes explicitly in your maven-jar-plugin settings. This explicit inclusion automatically excludes all other files from the jar, ie, all files from storm-core.

<plugin>
  <groupId>org.apache.maven.plugins</groupId>
  <artifactId>maven-jar-plugin</artifactId>
  <version>2.6</version>

  <executions>
    <execution>
      <id>MyTopology</id>
      <phase>package</phase>
      <goals>
        <goal>jar</goal>
      </goals>
      <configuration>
        <includes>
          <include>my/topology/package/**/*.class</include>
        </includes>
      </configuration>
    </execution>
  </executions>
</plugin>

这篇关于找到了多个defaults.yaml资源的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！