Apache Flink:java.lang.NoClassDefFoundError [英] Apache Flink: java.lang.NoClassDefFoundError

问题描述

我正在尝试遵循此

可能有什么问题?任何依赖项，版本?

注意:如果我注释有问题的行，程序运行没有问题.另请注意，此错误也出现在某些 Kafka (aws-kafka) 示例中；我在 amazonaws.kafka.samples.CustomMM2ReplicationPolicy 类中看到了同样的错误.

POM 文件

<project xmlns="http://maven.apache.org/POM/4.0.0";xmlns:xsi=http://www.w3.org/2001/XMLSchema-instance"xsi:schemaLocation=http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd"><modelVersion>4.0.0</modelVersion><groupId>as</groupId><artifactId>a</artifactId><版本>1</版本><包装>罐</包装><属性><project.build.sourceEncoding>UTF-8</project.build.sourceEncoding><geotools.version>21-SNAPSHOT</geotools.version><java.version>1.8</java.version><scala.binary.version>2.11</scala.binary.version><flink.version>1.6.2</flink.version><kda.version>1.0.1</kda.version></属性><构建><插件><插件><groupId>org.apache.maven.plugins</groupId><artifactId>maven-compiler-plugin</artifactId><version>3.1</version><配置><source>${java.version}</source><target>${java.version}</target></配置></插件><插件><groupId>org.apache.maven.plugins</groupId><artifactId>maven-shade-plugin</artifactId><version>3.0.0</version><执行><!-- 在包装阶段运行阴影目标--><执行><phase>包</phase><目标><目标>阴影</目标></目标><配置><工件集><排除><exclude>org.apache.flink:force-shading</exclude><exclude>com.google.code.findbugs:jsr305</exclude><exclude>org.slf4j:*</exclude><exclude>log4j:*</exclude></排除></artifactSet><过滤器><过滤器><!-- 不要复制 META-INF 文件夹中的签名.否则，这可能会在使用 JAR 时导致 SecurityExceptions.--><神器>*:*</神器><排除><exclude>META-INF/*.SF</exclude><exclude>META-INF/*.DSA</exclude><exclude>META-INF/*.RSA</exclude></排除></过滤器></过滤器><变形金刚><transformer implementation="org.apache.maven.plugins.shade.resource.ManifestResourceTransformer"><mainClass>com.amazonaws.services.kinesisanalytics.aws</mainClass></变压器></变形金刚></配置></执行></执行></插件></plugins></build><依赖项><依赖><groupId>org.locationtech.jts</groupId><artifactId>jts-core</artifactId><version>1.16.0</version></依赖><依赖><groupId>org.geotools</groupId><artifactId>gt-geojson</artifactId><version>${geotools.version}</version></依赖><依赖><groupId>org.geotools</groupId><artifactId>-geojsondatastore</artifactId><version>${geotools.version}</version></依赖><依赖><groupId>org.geotools</groupId><artifactId>gt-main</artifactId><version>${geotools.version}</version></依赖><依赖><groupId>com.amazonaws</groupId><artifactId>aws-kinesisanalytics-runtime</artifactId><version>${kda.version}</version></依赖><依赖><groupId>org.apache.flink</groupId><artifactId>flink-connector-kinesis_${scala.binary.version}</artifactId><version>${flink.version}</version></依赖><依赖><groupId>org.apache.flink</groupId><artifactId>flink-streaming-java_${scala.binary.version}</artifactId><version>${flink.version}</version><范围>提供</范围></依赖><依赖><groupId>com.amazonaws</groupId><artifactId>aws-kinesisanalytics-flink</artifactId><version>${kda.version}</version></依赖></依赖项></项目>

解决方案

你的错误在这里:

<依赖><groupId>org.apache.flink</groupId><artifactId>flink-streaming-java_${scala.binary.version}</artifactId><version>${flink.version}</version><范围>提供</范围></依赖>

范围是提供的.这意味着，您的程序希望在运行时可以访问该库，但在任何地方都找不到它.

NoClassDefFoundError 表示在编译时可以访问，但在执行期间不能访问.

---

将范围更改为 compile 以解决问题:

<依赖><groupId>org.apache.flink</groupId><artifactId>flink-streaming-java_${scala.binary.version}</artifactId><version>${flink.version}</version><scope>编译</scope></依赖>

另外，你可以看看此处了解更多技术信息.

I'm trying to follow this example but when I try to compile it, I have this error:

Error: Unable to initialize main class com.amazonaws.services.kinesisanalytics.aws
Caused by: java.lang.NoClassDefFoundError: org/apache/flink/streaming/api/functions/source/SourceFunction

The error is due this code:

    private static DataStream<String> createSourceFromStaticConfig(StreamExecutionEnvironment env) {
        Properties inputProperties = new Properties();
        inputProperties.setProperty(ConsumerConfigConstants.AWS_REGION, region);
        inputProperties.setProperty(ConsumerConfigConstants.STREAM_INITIAL_POSITION, "LATEST");

        return env.addSource(new FlinkKinesisConsumer<>(inputStreamName, new SimpleStringSchema(), inputProperties));
    }

And I suppose this is the problematic line:

  return env.addSource(new FlinkKinesisConsumer<>(inputStreamName, new SimpleStringSchema(), inputProperties));

This are my maven dependencies:

What could be wrong? any dependencies, version?

Note: If I comment the problematic line, the program run without problem. Also to note, this error also appears on certain Kafka (aws-kafka) samples; I was shown the same error with the amazonaws.kafka.samples.CustomMM2ReplicationPolicy class.

POM file

<?xml version="1.0" encoding="UTF-8"?>
<project xmlns="http://maven.apache.org/POM/4.0.0"
         xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
         xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
    <modelVersion>4.0.0</modelVersion>
    <groupId>as</groupId>
    <artifactId>a</artifactId>
    <version>1</version>
    <packaging>jar</packaging>
    <properties>
        <project.build.sourceEncoding>UTF-8</project.build.sourceEncoding>
        <geotools.version>21-SNAPSHOT</geotools.version>
        <java.version>1.8</java.version>
        <scala.binary.version>2.11</scala.binary.version>
        <flink.version>1.6.2</flink.version>
        <kda.version>1.0.1</kda.version>
    </properties>
    <build>
        <plugins>
            <plugin>
                <groupId>org.apache.maven.plugins</groupId>
                <artifactId>maven-compiler-plugin</artifactId>
                <version>3.1</version>
                <configuration>
                    <source>${java.version}</source>
                    <target>${java.version}</target>
                </configuration>
            </plugin>

            <plugin>
                <groupId>org.apache.maven.plugins</groupId>
                <artifactId>maven-shade-plugin</artifactId>
                <version>3.0.0</version>
                <executions>
                    <!-- Run shade goal on package phase -->
                    <execution>
                        <phase>package</phase>
                        <goals>
                            <goal>shade</goal>
                        </goals>
                        <configuration>
                            <artifactSet>
                                <excludes>
                                    <exclude>org.apache.flink:force-shading</exclude>
                                    <exclude>com.google.code.findbugs:jsr305</exclude>
                                    <exclude>org.slf4j:*</exclude>
                                    <exclude>log4j:*</exclude>
                                </excludes>
                            </artifactSet>
                            <filters>
                                <filter>
                                    <!-- Do not copy the signatures in the META-INF folder.
                                    Otherwise, this might cause SecurityExceptions when using the JAR. -->
                                    <artifact>*:*</artifact>
                                    <excludes>
                                        <exclude>META-INF/*.SF</exclude>
                                        <exclude>META-INF/*.DSA</exclude>
                                        <exclude>META-INF/*.RSA</exclude>
                                    </excludes>
                                </filter>
                            </filters>
                            <transformers>
                                <transformer implementation="org.apache.maven.plugins.shade.resource.ManifestResourceTransformer">
                                    <mainClass>com.amazonaws.services.kinesisanalytics.aws</mainClass>
                                </transformer>
                            </transformers>
                        </configuration>
                    </execution>
                </executions>
            </plugin>
        </plugins>

    </build>
    <dependencies>
        <dependency>
            <groupId>org.locationtech.jts</groupId>
            <artifactId>jts-core</artifactId>
            <version>1.16.0</version>
        </dependency>
        <dependency>
            <groupId>org.geotools</groupId>
            <artifactId>gt-geojson</artifactId>
            <version>${geotools.version}</version>
        </dependency>
        <dependency>
            <groupId>org.geotools</groupId>
            <artifactId>gt-geojsondatastore</artifactId>
            <version>${geotools.version}</version>
        </dependency>
        <dependency>
            <groupId>org.geotools</groupId>
            <artifactId>gt-main</artifactId>
            <version>${geotools.version}</version>
        </dependency>
        <dependency>
            <groupId>com.amazonaws</groupId>
            <artifactId>aws-kinesisanalytics-runtime</artifactId>
            <version>${kda.version}</version>
        </dependency>
        <dependency>
            <groupId>org.apache.flink</groupId>
            <artifactId>flink-connector-kinesis_${scala.binary.version}</artifactId>
            <version>${flink.version}</version>
        </dependency>
        <dependency>
            <groupId>org.apache.flink</groupId>
            <artifactId>flink-streaming-java_${scala.binary.version}</artifactId>
            <version>${flink.version}</version>
            <scope>provided</scope>
        </dependency>
        <dependency>
            <groupId>com.amazonaws</groupId>
            <artifactId>aws-kinesisanalytics-flink</artifactId>
            <version>${kda.version}</version>
        </dependency>
    </dependencies>
</project>

解决方案

Your error is here:

<dependency>
    <groupId>org.apache.flink</groupId>
    <artifactId>flink-streaming-java_${scala.binary.version}</artifactId>
    <version>${flink.version}</version>
    <scope>provided</scope>
</dependency>

The scope is provided. This means, that your program expects the library to be accessible during the runtime, but it can't find it anywhere.

NoClassDefFoundError means it was accessible during the compile-time, but not during the execution.

---

Change the scope to compile in order to resolve the issue:

<dependency>
    <groupId>org.apache.flink</groupId>
    <artifactId>flink-streaming-java_${scala.binary.version}</artifactId>
    <version>${flink.version}</version>
    <scope>compile</scope>
</dependency>

Also, you can take a look here for more technical info.

这篇关于Apache Flink:java.lang.NoClassDefFoundError的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！