在日食中调试hadoop [英] Debugging hadoop in eclipse

问题描述

是否可以在Eclipse中调试Hadoop的源代码？我不是要求地图缩小任务。我想看看哪个部分的Hadoop源代码负责安排地图缩减任务及其工作原理。是否有任何机制可以完成？

Is it possible to debug Hadoop's source code in Eclipse?I'm not asking about the map reduce tasks. I want to see which part of the Hadoop source code is responsible for scheduling the map reduce tasks and how it works. Is there any mechanism by which it can be done?

推荐答案

您可以下载Hadoop项目并将其集成到您的日食中，并使用F5或F6进行调试。
您在eclipse中有不同的调试方式：

You can download Hadoop project and integrate it to your eclipse, and use F5 or F6 to debug. You have different mode of debugging in eclipse:

1. F5：逐步调试

< F6：跳过循环和子程序

2. F7：跳过循环或子程序并返回到最后一个光标点。


3. F8：执行并进入调试

1. F5 : Step by Step debugging
2. F6 : Skips loops and Subroutines
3. F7 : Skips the loop or subroutine and returns to the last cursor point.
4. F8 : Execute and come out of debugging

或者你可以尝试一下一步来了解工作流程，你可以从你的 run（）方法在你的主要。

Or you can try yourself to understand the workflow by following step by step, you can begin from your run() method in your main.

回答你的问题：谁安排地图任务？

To answer your question: who does schedule the map task?

正如你所见此模式，文件由 InputFormat 类除以固定大小的片段称为InputSplits。每个拆分然后被赋予一个映射器，它是一个分配了地图任务的节点。

As you can see in this schema, files are divided by the InputFormat class into fixed-size pieces called InputSplits. Each split is then given to a mapper, which is a node that was assigned a map task.

相同的 InputFormat class还提供了一个 RecordReader ，负责解析拆分和提取记录。每个记录作为（键，值）对传递给map函数。因此， Mapper 类是调用地图方法的人。

The same InputFormat class also provides a RecordReader responsible for parsing the split and extracting records.Each record is passed to a map function as a (key, value) pair. So the Mapper class is the one who call map methods.

以下是文字数据示例的工作流程：

Here is the workflow of the wordcount example:

哪里 FileInputFormat 是一个抽象类，扩展抽象类 InputFormat ，以及 TextInputFormat 扩展了 FileInputFormat 类。

Where the FileInputFormat is an abstract class that extends the abstract class InputFormat, and the TextInputFormat extends the FileInputFormat class.

这篇关于在日食中调试hadoop的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！