如何使用直接运行器在Eclipse中调试Dataflow/Apache Beam管道DoFn函数 [英] How to debug Dataflow/Apache Beam pipeline DoFn functions in eclipse using direct runner
问题描述
我想在月食中使用直接运行器来运行管道， DoFn函数中的断点并调试执行.我尝试通过以下步骤设置直接跑步:
I want to run my pipeline using direct runner in eclipse and put a break point in my DoFn functions and debug execution. I tried to setup direct runner with following steps:
- 添加直接运行程序Maven包
- 在pom.xml中设置直接运行程序的maven配置文件.我的pom.xml具有此个人资料
<profiles>
<profile>
<id>direct-runner</id>
<activation>
<activeByDefault>true</activeByDefault>
</activation>
<dependencies>
<dependency>
<groupId>org.apache.beam</groupId>
<artifactId>beam-runners-direct-java</artifactId>
<version>0.2.0-incubating</version>
</dependency>
</dependencies>
</profile>
</profiles>
<profiles>
<profile>
<id>direct-runner</id>
<activation>
<activeByDefault>true</activeByDefault>
</activation>
<dependencies>
<dependency>
<groupId>org.apache.beam</groupId>
<artifactId>beam-runners-direct-java</artifactId>
<version>0.2.0-incubating</version>
</dependency>
</dependencies>
</profile>
</profiles>
- 我在pom.xml中的插件管理下有此 maven插件
- I have this maven plugin under plugin management in my pom.xml
<pluginManagement>
<plugins>
<plugin>
<groupId>org.codehaus.mojo</groupId>
<artifactId>exec-maven-plugin</artifactId>
<version>1.4.0</version>
<executions>
<execution>
<goals>
<goal>java</goal>
</goals>
</execution>
</executions>
<configuration>
<cleanupDaemonThreads>false</cleanupDaemonThreads>
<mainClass>com.MyMainClass</mainClass>
</configuration>
</plugin>
</plugins>
</pluginManagement>
<pluginManagement>
<plugins>
<plugin>
<groupId>org.codehaus.mojo</groupId>
<artifactId>exec-maven-plugin</artifactId>
<version>1.4.0</version>
<executions>
<execution>
<goals>
<goal>java</goal>
</goals>
</execution>
</executions>
<configuration>
<cleanupDaemonThreads>false</cleanupDaemonThreads>
<mainClass>com.MyMainClass</mainClass>
</configuration>
</plugin>
</plugins>
</pluginManagement>
- 以下是我的Eclipse调试配置的屏幕截图 当我使用上述调试配置运行时，作业开始于GCP数据流而不是本地JVM线程，并且我的断点从未被击中.
- Below is a screen shot of my eclipse debug configuration When I run using above debug configuration job starts in GCP dataflow instead of local JVM threads and my breakpoints are never hit.
推荐答案
可能是在测试方法中创建管道的方式.尝试使用像这样的TestPipeline util类创建管道
Probably is the way how you are creating your pipeline in your test methods. Try to create the pipeline using the TestPipeline util class like this
public TestPipeline p = TestPipeline.create();
这篇关于如何使用直接运行器在Eclipse中调试Dataflow/Apache Beam管道DoFn函数的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！
