Spark Monitoring with Ganglia

Question

I need to monitor the network and the resources that are being used by Spark in the Cluster.

Is Gangalia a good option? If so how can I install and use it? Help Needed.

Thanks in Advance

Answer 1

Ganglia looks like a good option for what you need. Gangalia is a maven project , so to use it you need to put 

-Pspark-ganglia-lgpl option in mvn command when you build the project.

mvn -Pspark-ganglia-lgpl -Pyarn -Phadoop-2.4 -Dhadoop.version=2.4.0 -DskipTests clean package