java.lang.Runtime "main"线程中的异常异常：在执行 Storm 时发现多个 defaults.yaml 资源

Question

我正在尝试执行此处的教程http://virajonlinetutor.blogspot.in/

卡夫卡风暴拓扑代码为：

package com.storm;
import storm.kafka.KafkaSpout;
import storm.kafka.SpoutConfig;
import storm.kafka.StringScheme;
import storm.kafka.ZkHosts;
import backtype.storm.Config;
import backtype.storm.LocalCluster;
import backtype.storm.spout.SchemeAsMultiScheme;
import backtype.storm.topology.TopologyBuilder;
public class KafkaStormTopology {
public static void main(String[] args) {
  ZkHosts zk = new ZkHosts("10.25.3.208:2181");
  SpoutConfig config = new SpoutConfig(zk,"deepthy","",
"KafkaStorm");
 config.scheme = new SchemeAsMultiScheme(new StringScheme());
  config.forceFromStart = true;
  TopologyBuilder builder = new TopologyBuilder();
  builder.setSpout("KafkaSpout", new KafkaSpout(config), 1);
  builder.setBolt("Bolt", new FileBolt(), 1).globalGrouping("KafkaSpout");
  LocalCluster cluster = new LocalCluster();
  Config conf = new Config();
  conf.setDebug(true);
  cluster.submitTopology("SampleTopology", conf, builder.createTopology());
 }
}

我的pom.xml：

<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
<modelVersion>4.0.0</modelVersion>
<groupId>com.deepthy.storm</groupId>
<artifactId>Storm</artifactId>
<version>0.0.1-SNAPSHOT</version>
<name>Storm</name>
<url>http://maven.apache.org</url>
<properties>
    <project.build.sourceEncoding>UTF-8</project.build.sourceEncoding>
</properties>
<repositories>
    <repository>
        <id>clojars.org</id>
        <url>http://clojars.org/repo</url>
    </repository>
    <repository>
        <id>central</id>
        <url>http://repo1.maven.org/maven2/</url>
    </repository>
</repositories>
<dependencies>
    <dependency>
        <groupId>junit</groupId>
        <artifactId>junit</artifactId>
        <version>3.8.1</version>
        <scope>test</scope>
    </dependency>
      <dependency>
        <groupId>org.apache.storm</groupId>
        <artifactId>storm-core</artifactId>
        <version>0.9.2-incubating</version>
        <scope>provided</scope>
    </dependency>
    <dependency>
        <groupId>org.apache.storm</groupId>
        <artifactId>storm-kafka</artifactId>
        <version>0.9.2-incubating</version>
        <scope>provided</scope>
    </dependency>
    <dependency>
<groupId>org.apache.kafka</groupId>
<artifactId>kafka_2.10</artifactId>
<version>0.8.2.2</version>
<scope>provided</scope>
</dependency>
</dependencies>
<build>
<plugins>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-assembly-plugin</artifactId>
<version>2.5.3</version>
<configuration>
<descriptorRefs>
<descriptorRef>jar-with-dependencies</descriptorRef>
</descriptorRefs>
<archive>
<manifest>
<mainClass>com.storm.KafkaStormTopology</mainClass>
</manifest>
 </archive>
</configuration>
<executions>
<execution>
<phase>package</phase>
<goals>
<goal>single</goal>
 </goals>
 </execution>
</executions>
</plugin>
</plugins>
</build>
</project>

我得到以下错误：

Exception in thread "main" java.lang.RuntimeException: Found multiple defaults.yaml resources. You're probably bundling the Storm jars with your topology jar. [jar:file:/usr/hdp/2.3.0.0-2557/storm/lib/storm-core-0.10.0.2.3.0.0-2557.jar!/defaults.yaml, jar:file:/usr/hdp/2.3.0.0-2557/storm/S1.jar!/defaults.yaml]
    at backtype.storm.utils.Utils.findAndReadConfigFile(Utils.java:140)
    at backtype.storm.utils.Utils.readDefaultConfig(Utils.java:167)
    at backtype.storm.utils.Utils.readStormConfig(Utils.java:191)
    at backtype.storm.config$read_storm_config.invoke(config.clj:121)
    at backtype.storm.command.config_value$_main.invoke(config_value.clj:22)
    at clojure.lang.AFn.applyToHelper(AFn.java:161)
    at clojure.lang.AFn.applyTo(AFn.java:151)
    at backtype.storm.command.config_value.main(Unknown Source)

我尝试将pom.xml中的作用域从"compile"更改为"provided"，但出现了相同的错误。

关于如何解决这个问题的任何建议。

Answer 1

您的jar包含不允许的文件defaults.yaml。您需要从jar组装步骤中排除该文件。我建议使用maven-jar-plugin而不是maven-assembly-plugin，并"手动"包含所有需要的文件——这也有助于"最小化"你的罐子，只包含你需要的东西：

<plugin>
        <groupId>org.apache.maven.plugins</groupId>
        <artifactId>maven-jar-plugin</artifactId>
        <executions>
                <execution>
                    <id>YOUR-ID</id>
                    <phase>package</phase>
                    <goals>
                        <goal>jar</goal>
                    </goals>
                    <configuration>
                        <finalName>JAR-FILE-NAME-PREFIX</finalName>
                        <classifier>JAR-FILE-NAME-SUFFIX</classifier>
                        <archive>
                            <manifestEntries>
                                <mainClass>com.storm.KafkaStormTopology</mainClass>
                            </manifestEntries>
                        </archive>
                        <includes>
                            <include>packages/to/include/**/*.java</include>
                            <!-- more include here -->
                        <includes>
                    </configuration>
                </execution>
        </executions>
</plugin>

Answer 2

排除pom文件中的default.yaml文件。

在您的shade插件中添加：META-INF/.SFMETA-INF/.DSAMETA-INF/.RSA.yaml