如何使用直接运行器在 Eclipse 中调试 Dataflow/Apache Beam 管道 DoFn 功能 [英] How to debug Dataflow/Apache Beam pipeline DoFn functions in eclipse using direct runner

问题描述

我想在 Eclipse 中使用

在 pom.xml 中为直接运行器设置 maven 配置文件.我的 pom.xml 有 当我使用上述调试配置作业在 GCP 数据流而不是本地 JVM 线程中运行时，我的断点永远不会被命中.

解决方案

可能是您在测试方法中创建管道的方式.尝试像这样使用 TestPipeline util 类创建管道

public TestPipeline p = TestPipeline.create();

I want to run my pipeline using direct runner in eclipse and put a break point in my DoFn functions and debug execution. I tried to setup direct runner with following steps:

1. Add direct runner maven package
2. Setup maven profile for direct runner in pom.xml. My pom.xml has this profile

<profiles> <profile> <id>direct-runner</id> <activation> <activeByDefault>true</activeByDefault> </activation> <dependencies> <dependency> <groupId>org.apache.beam</groupId> <artifactId>beam-runners-direct-java</artifactId> <version>0.2.0-incubating</version> </dependency> </dependencies> </profile> </profiles>

3. I have this maven plugin under plugin management in my pom.xml

<pluginManagement> <plugins> <plugin> <groupId>org.codehaus.mojo</groupId> <artifactId>exec-maven-plugin</artifactId> <version>1.4.0</version> <executions> <execution> <goals> <goal>java</goal> </goals> </execution> </executions> <configuration> <cleanupDaemonThreads>false</cleanupDaemonThreads> <mainClass>com.MyMainClass</mainClass> </configuration> </plugin> </plugins> </pluginManagement>

4. Below is a screen shot of my eclipse debug configuration When I run using above debug configuration job starts in GCP dataflow instead of local JVM threads and my breakpoints are never hit.

解决方案

Probably is the way how you are creating your pipeline in your test methods. Try to create the pipeline using the TestPipeline util class like this

public TestPipeline p = TestPipeline.create();

这篇关于如何使用直接运行器在 Eclipse 中调试 Dataflow/Apache Beam 管道 DoFn 功能的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！