Apache Nifi 属性描述符的 dynamicModifiableClasspath() 的 Hello world 程序不起作用 [英] Hello world program of dynamicallyModifiesClasspath() of Apache Nifi properties descriptor not working
问题描述
我有一个处理器，它依赖于某个 jar 作为依赖项.但是，对依赖项 jar 的更改非常频繁.所以我希望能够将 jar 路径指定为处理器的属性，并让 nifi 每次修改属性中的路径并重新启动处理器时加载 jar.这应该可以使用
处理器配置:
我收到以下异常:
2019-05-27 17:01:54,536 ERROR [Timer-Driven Process Thread-1] com.mycompany.djl.MyDjlProcessor MyDjlProcessor[id=f8fa5750-016a-1000-ecc3-c19jl322]=f8fa5750-016a-1000-ecc3-c19732119332] 由于 java.lang.NoClassDefFoundError 无法处理:com/mycompany/djldependency/DjlDependencyClass；回滚会话:{}java.lang.NoClassDefFoundError: com/mycompany/djldependency/DjlDependencyClass在 com.mycompany.djl.MyDjlProcessor.onTrigger(MyDjlProcessor.java:76)在 org.apache.nifi.processor.AbstractProcessor.onTrigger(AbstractProcessor.java:27)在 org.apache.nifi.controller.StandardProcessorNode.onTrigger(StandardProcessorNode.java:1122)在 org.apache.nifi.controller.tasks.ContinuallyRunProcessorTask.call(ContinuallyRunProcessorTask.java:147)在 org.apache.nifi.controller.tasks.ContinuallyRunProcessorTask.call(ContinuallyRunProcessorTask.java:47)在 org.apache.nifi.controller.scheduling.TimerDrivenSchedulingAgent$1.run(TimerDrivenSchedulingAgent.java:128)在 java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)在 java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308)在 java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180)在 java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294)在 java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)在 java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)在 java.lang.Thread.run(Thread.java:748)我在这里做了什么蠢事吗?
请在此处下载这两个项目(nifi 和依赖项 jar).
PropertyDescriptor 的 API 文档中提到了以下注释:
如果组件包含一个 PropertyDescriptor，其中 dynamicModifiableClasspath 设置为 true，则该组件还必须使用 @RequiresInstanceClassloading 进行注释，否则该组件将被视为无效.
我不确定那个无效"部分是否仍然成立，但是是的，在更新您的处理器代码以使用 @RequiresInstanceClassloading 对其进行注释后，我能够让它正常工作.
I have a processor which depends on some jar as a dependency. However changes are made to the dependency jar quite frequenctly. So I want to be able to specify the jar path as a property of processor and let the nifi load the jar each time I modify the path in the property and restart the processor. This is supposed to be doable using dynamicallyModifiesClasspath as explained in this article. However I am unable to do this. Below is my code of hello world program using dynamicallyModifiesClasspath property:
(Below, Djl stands for "dynamic jar loading", a random prefix I guessed to name this hello world program.)
DjlDependencyClass.java
This is the class on which my nifi processor depends and I want to dynamically change its jar path in my nifi processor.
public class DjlDependencyClass {
public static String getMessage()
{
return "DJL-DEPENDENCY VERSION-1";
}
}
MyDjlProcessor.java
This is the nifi processor which depends on DjlDependencyClass.
import org.apache.nifi.annotation.lifecycle.OnScheduled;
import org.apache.nifi.components.PropertyDescriptor;
import org.apache.nifi.processor.exception.ProcessException;
import org.apache.nifi.processor.AbstractProcessor;
import org.apache.nifi.processor.ProcessContext;
import org.apache.nifi.processor.ProcessSession;
import org.apache.nifi.processor.ProcessorInitializationContext;
import org.apache.nifi.processor.Relationship;
import org.apache.nifi.processor.util.StandardValidators;
import com.mycompany.djldependency.DjlDependencyClass;
import java.util.ArrayList;
import java.util.Collections;
import java.util.HashSet;
import java.util.List;
import java.util.Set;
public class MyDjlProcessor extends AbstractProcessor {
public static final Relationship MY_RELATIONSHIP = new Relationship.Builder()
.name("MY_RELATIONSHIP")
.description("Example relationship")
.build();
public static final PropertyDescriptor pathToDjlDependencyJar = new PropertyDescriptor.Builder()
.name("Djl Dependency JAR")
.description("Djl Dependency JAR")
.required(true)
.addValidator(StandardValidators.NON_EMPTY_VALIDATOR)
.dynamicallyModifiesClasspath(true)
.expressionLanguageSupported(true)
.build();
private List<PropertyDescriptor> descriptors;
private Set<Relationship> relationships;
@Override
protected void init(final ProcessorInitializationContext context) {
final List<PropertyDescriptor> descriptors = new ArrayList<PropertyDescriptor>();
descriptors.add(pathToDjlDependencyJar);
this.descriptors = Collections.unmodifiableList(descriptors);
final Set<Relationship> relationships = new HashSet<Relationship>();
relationships.add(MY_RELATIONSHIP);
this.relationships = Collections.unmodifiableSet(relationships);
}
@Override
public Set<Relationship> getRelationships() {
return this.relationships;
}
@Override
public final List<PropertyDescriptor> getSupportedPropertyDescriptors() {
return descriptors;
}
@OnScheduled
public void onScheduled(final ProcessContext context) {
}
@Override
public void onTrigger(final ProcessContext context, final ProcessSession session) throws ProcessException {
System.out.println(DjlDependencyClass.getMessage());
}
}
This is how dependency inside pom is marked to have provided scope, so that the dependency jar will not be embedded inside nar:
<dependency>
<groupId>com.mycompany</groupId>
<artifactId>djl-dependency</artifactId>
<version>0.0.1-SNAPSHOT</version>
<scope>provided </scope>
</dependency>
I am testing this processor as follows:
Configuration of the processor:
I am getting following exception:
2019-05-27 17:01:54,536 ERROR [Timer-Driven Process Thread-1] com.mycompany.djl.MyDjlProcessor MyDjlProcessor[id=f8fa5750-016a-1000-ecc3-c19732119332] MyDjlProcessor[id=f8fa5750-016a-1000-ecc3-c19732119332] failed to process due to java.lang.NoClassDefFoundError: com/mycompany/djldependency/DjlDependencyClass; rolling back session: {}
java.lang.NoClassDefFoundError: com/mycompany/djldependency/DjlDependencyClass
at com.mycompany.djl.MyDjlProcessor.onTrigger(MyDjlProcessor.java:76)
at org.apache.nifi.processor.AbstractProcessor.onTrigger(AbstractProcessor.java:27)
at org.apache.nifi.controller.StandardProcessorNode.onTrigger(StandardProcessorNode.java:1122)
at org.apache.nifi.controller.tasks.ContinuallyRunProcessorTask.call(ContinuallyRunProcessorTask.java:147)
at org.apache.nifi.controller.tasks.ContinuallyRunProcessorTask.call(ContinuallyRunProcessorTask.java:47)
at org.apache.nifi.controller.scheduling.TimerDrivenSchedulingAgent$1.run(TimerDrivenSchedulingAgent.java:128)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308)
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180)
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
Am I doing something stupid here?
Please download both projects (nifi and dependency jar) here.
The API document for PropertyDescriptor mentions the following note:
If a component contains a PropertyDescriptor where dynamicallyModifiesClasspath is set to true, the component must also be annotated with @RequiresInstanceClassloading, otherwise the component will be considered invalid.
I'm not sure whether that 'invalid' part still holds true but yeah, after updating your processor code to have it annotated with @RequiresInstanceClassloading, I was able to get it working.
这篇关于Apache Nifi 属性描述符的 dynamicModifiableClasspath() 的 Hello world 程序不起作用的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！
