使用SSH从Java应用程序连接到Hadoop [英] Connecting to Hadoop from Java app using SSH

问题描述

我正在尝试连接到远程hadoop集群，该集群无法仅通过HDFS进行访问.现在，它正以这种方式使用:用户通过SSH连接到Jump box(例如ssh [e-mail])，然后我们从跳转框服务器中也通过ssh连接到hadoop(例如ssh [e-mail]).我正在尝试使用HDFS客户端从我的Scala/Java应用程序访问文件.现在，我感觉就像在Matrix中一样-我必须更深入"并且不知道如何到达服务器.

I'm trying to connect to remote hadoop cluster, which isn't accessible just through HDFS. Right now it is being used in that way: user connects to Jump box through SSH (e.g. ssh [e-mail]), then from jump box server we do connect to hadoop also with ssh (e.g. ssh [e-mail]). What I'm trying to do is to access files from my Scala/Java application using HDFS client. Now I'm feeling like in Matrix - "I must go deeper" and don't know how to reach the server.

可能有人曾经有过类似的经历吗?现在，我正在尝试通过我的应用程序使用SSH客户端连接到第一台服务器，但是那时我不知道如何调用HDFS客户端.

May be someone had similar experience? Right now I'm trying to connect to first server with SSH client from my app, but then I don't know how to call the HDFS client.

任何想法都会受到赞赏，谢谢！

Any ideas will be appreciated, thanks!

推荐答案

我可以想到这样的东西.有"ganymed-ssh2" api，可帮助您使用ssh连接到某些服务器并从那里运行unix命令.使用此工具，您可以连接到jumo盒.

I can think of something like this . There is "ganymed-ssh2" api which helps you to connect to some server using ssh and run unix command from there. Using this you can connect to your jumo box.

然后从那里可以运行命令"ssh [e-mail] hadoop fs somthing"

And from there you can run command as " ssh [e-mail] hadoop fs somthing"

我们可以像这样用ssh运行命令.

As we can run commands with ssh like this.

在您的跳转框中，为您的hadoopcluster计算机设置一个密码为ssh的密码.或者您可以使用带有密码的sshpass.

From your jump box, setup a password less ssh to your hadoopcluster machine. or you can use sshpass with password.

您可以访问以下链接来检查如何使用此api:

You can visit following link to check how to use this api:

http ://souravgulati.webs.com/apps/forums/topics/show/8116298-how-to-execute-unix-command-from-java-

这篇关于使用SSH从Java应用程序连接到Hadoop的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！