星火/摇篮 - 中获取的build.gradle IP地址用于启动主及工人 [英] Spark/Gradle -- Getting IP Address in build.gradle to use for starting master and workers

问题描述

我明白在一个基本水平的build.gradle的各运动部件打造剧本，但我有麻烦绑到一起。

I understand at a basic level the various moving parts of build.gradle build scripts but am having trouble tying it all together.

在Apache的星火独立模式，只是想开始从的build.gradle相同的方块高手和工人。

In Apache Spark standalone mode, just trying to start a master and worker on the same box from build.gradle. (Later will extend with call with $SPARK_HOME/sbin/start-slaves with the proper argument for masterIP.)

问：我如何分配我的IP地址在Groovy /的build.gradle一个变量，所以我可以在Exec的任务，它传递的命令？我们希望这个在几个不同的开发机器上运行。

我们有一个（我觉得相当标准）/ etc / hosts中的FQDN并分配给主机127.0.1.1配置。该驱动程序来解决这个确定，但主开始与主机名奴隶是不是一种选择，我需要的IP地址。

We have a (I think fairly standard) /etc/hosts config with the FQDN and hostname assigned to 127.0.1.1. The driver gets around this OK but starting master and slaves with hostnames is not an option, I need the ip address.

我想：

task getMasterIP (type: Exec){
    // declare script scope variable using no def or
    executable "hostname"
    args += "-I"

    // need results of hostname call assigned to script scope variable
    sparkMasterIP = <resultsOfHostnameCall>
}

// added this because startSlave stops if Master is already running
task startSlaveOnly(dependsOn:'getMasterIP', type: Exec){
    executable "/usr/local/spark/sbin/start-slave.sh"
    args += "spark://$sparkMasterIP:7077"
    doLast {
        println "enslaved"
    }
}

// now make startSlave call startSlaveOnly after the initial startMaster
task startSlave(dependsOn:'startMaster', type: Exec) {
    finalizedBy 'startSlaveOnly'
}

当我尝试类似建议在文档中进行EXEC对Groovy呼吁：

task getMasterIP (type: Exec){
    // declare script scope variable using no def or
    sparkMasterIP = executable "hostname"
    args += "-I"
}

我得到的可执行文件不承认的警告。

I get a warning that executable is not recognized.

的关于我的想法多一点背景部分，不是主要的问题。

谷歌搜索的build.gradle脚本作用域变量看着前两个结果，在基本的文档我只看到一种类型的变量和扩展属性的情况下使用。

Googling "build.gradle script scope variables" and looking at the first two results, in the basic docs I only see one type of variable and ext properties to be used.

16.4。声明变量 - 有两种，可以在一个构建脚本中声明的变量：局部变量和额外的属性

16.4. Declaring variables -- There are two kinds of variables that can be declared in a build script: local variables and extra properties.

但在这个其他摇篮文档附录B.潜在的陷阱我看到两类变量从扩展属性作用域旁白：

But in this other Gradle doc Appendix B. Potential Traps I am seeing two kinds of variables scopes aside from the ext properties:

有关摇篮用户了解如何使用Groovy的交易是非常重要的
  脚本变量。 Groovy中有两种类型的脚本变量。一个具有
  局部范围，一个具有脚本范围广。

For Gradle users it is important to understand how Groovy deals with script variables. Groovy has two types of script variables. One with a local scope and one with a script-wide scope.

通过这个例子用法：

String localScope1 = 'localScope1'
def localScope2 = 'localScope2'
scriptScope = 'scriptScope'

我假设我应该使用脚本范围变量没有高清或类型声明。完全开放的完全不同的方法，谢谢。

I am assuming I should be using script-scope variables with no "def" or type declaration. Totally open to completely different approaches, thanks.

推荐答案

要获取本地IP地址：

//return all v4 addresses 
def getLocalIPv4() {
    def ip4s = []
    NetworkInterface.getNetworkInterfaces()
            .findAll { it.isUp() && !it.isLoopback() && !it.isVirtual() }
            .each {
        it.getInetAddresses()
                .findAll { !it.isLoopbackAddress() && it instanceof Inet4Address }
                .each { ip4s << it }
    }
    return ip4s
}

//optionally, return all ipv6 addresses
def getLocalIPv6() {
    def ip6s = []
    NetworkInterface.getNetworkInterfaces()
            .findAll { it.isUp() && !it.isLoopback() && !it.isVirtual() }
            .each {
        it.getInetAddresses()
                .findAll { !it.isLoopbackAddress() && it instanceof Inet6Address }
                .each { ip6s << it }
    }
    return ip6s
}



task printIP()<<{
    println getLocalIPv4()
    println getLocalIPv6()
}

这两个函数上述返回分别IPv4或IPv6地址的列表。您可能注意到，我跳过所有localhosts，所有不为接口，所有环回和虚拟接口。如果你想使用的第一个IPv4地址，您可以在任何地方使用它作为：

The two functions above return a list of ipv4 or ipv6 addresses respectively. You might notice that I'm skipping all localhosts, interfaces that are not up, all loopbacks and virtual interfaces. If you want to use the first ipv4 address, you can use it elsewhere as:

getLocalIPv4()[0]

或你的情况：

args += "spark:/"+ getLocalIPv4()[0] + ":7077"

这篇关于星火/摇篮 - 中获取的build.gradle IP地址用于启动主及工人的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！