用Ganglia进行火花监控 [英] Spark Monitoring with Ganglia

问题描述

我正在测试框架Apache Spark. 我需要监视有关群集的某些方面，例如网络和资源.

I'm testing the framework Apache Spark. I need monitoring some aspects about my cluster like network and resources.

对于我所需要的，ANGLIA似乎是一个不错的选择. 然后，我发现Spark支持Ganglia.

Ganglia looks like a good option for what I need. Then, I found out that Spark has support to Ganglia.

在Spark监视网页上有以下信息: 要安装GangliaSink，您需要执行自定义版本的Spark."

On the Spark monitoring webpage there is this information: "To install the GangliaSink you’ll need to perform a custom build of Spark."

我在Spark中找到目录:"/extras/spark-ganglia-lgpl".但是我不知道如何安装.

I found in my Spark the directory: "/extras/spark-ganglia-lgpl". But I don't know how to install it.

如何安装Ganglia来监视Spark集群? 我该如何进行自定义构建?

How can I install the Ganglia to monitoring Spark cluster? How I do this custom build?

谢谢！

推荐答案

Spark Ganglia支持是Spark项目的Maven配置文件之一，它是"spark-ganglia-lgpl".为了激活配置文件，在生成项目时，在mvn命令中放置了"-Pspark-ganglia-lgpl"选项.例如，使用Ganglia构建Apache Hadoop 2.4.X的方法是

Spark Ganglia support is one of Maven profiles of Spark project and it's "spark-ganglia-lgpl". In order to activate the profile, you put "-Pspark-ganglia-lgpl" option in mvn command when you build the project. For example, building Apache Hadoop 2.4.X with Ganglia is done by

mvn -Pspark-ganglia-lgpl -Pyarn -Phadoop-2.4 -Dhadoop.version=2.4.0 -DskipTests clean package

有关构建Spark项目的信息，请参阅使用以下命令构建Spark的文档Maven

For building the Spark project, please refer to documentation of Building Spark with Maven

这篇关于用Ganglia进行火花监控的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！