Namenode高可用性客户端请求 [英] Namenode high availability client request

问题描述

任何人都可以告诉我，如果我正在使用java应用程序通过Namenode HA安装程序向HDFS请求一些文件上载/下载操作，那么这个请求首先放到哪里？我的意思是客户如何知道哪个namenode是活动的？

Can anyone please tell me that If I am using java application to request some file upload/download operations to HDFS with Namenode HA setup, Where this request go first? I mean how would client know that which namenode is active?

如果您提供一些工作流类型图或详细解释请求步骤（从开始到结束）。

It would be great if you provide some workflow type diagram or something that explains request steps in detail(start to end).

推荐答案

如果hadoop集群配置了HA，那么它将在hdfs-site.xml中具有namenode ID，如下所示：

If hadoop cluster is configured with HA, then it will have namenode IDs in hdfs-site.xml like this :

<property>
  <name>dfs.ha.namenodes.mycluster</name>
  <value>namenode1,namenode2</value>
</property>

NameNode首先启动的任何一个都会激活。您可以选择以特定顺序启动群集，以便首选节点首先启动。

Whichever NameNode is started first will become active. You may choose to start the cluster in a specific order such that your preferred node starts first.

如果要确定namenode的当前状态，可以使用 getServiceStatus（）命令：

If you want to determine the current status of namenode, you can use getServiceStatus() command :

hdfs haadmin -getServiceState <machine-name>

好的，编写驱动程序类时，需要在配置对象中设置以下属性：

Well, while writing the driver class, you need to set the following properties in configuration object:

 public static void main(String[] args) throws Exception {
    if (args.length != 2){
        System.out.println("Usage: pgm <hdfs:///path/to/copy> </local/path/to/copy/from>");
        System.exit(1);
    }
    Configuration conf = new Configuration(false);
    conf.set("fs.defaultFS", "hdfs://nameservice1");
    conf.set("fs.default.name", conf.get("fs.defaultFS"));
    conf.set("dfs.nameservices","nameservice1");
    conf.set("dfs.ha.namenodes.nameservice1", "namenode1,namenode2");
    conf.set("dfs.namenode.rpc-address.nameservice1.namenode1","hadoopnamenode01:8020");
    conf.set("dfs.namenode.rpc-address.nameservice1.namenode2", "hadoopnamenode02:8020");
    conf.set("dfs.client.failover.proxy.provider.nameservice1","org.apache.hadoop.hdfs.server.namenode.ha.ConfiguredFailoverProxyProvider");

    FileSystem fs =  FileSystem.get(URI.create(args[0]), conf);
    Path srcPath = new Path(args[1]);
    Path dstPath = new Path(args[0]);
    //in case the same file exists on remote location, it will be overwritten
    fs.copyFromLocalFile(false, true, srcPath, dstPath);
}

请求将转到nameservice1，并按照namenode进一步由Hadoop集群处理状态（主动/待机）。

Request will go to the nameservice1 and further handled by Hadoop cluster as per the namenode status(active/standby).

有关更多详细信息，请参阅 HDFS高可用性

For more details, please refer the HDFS High availability

这篇关于Namenode高可用性客户端请求的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！