Analyze Big Data Platforms For Security and Performance

Deploy Environment

Setup Environment

Apache Eagle (called Eagle in the following) as an analytics solution for identifying security and performance issues instantly, relies on streaming platform Storm1 + Kafka2 to meet the realtime criteria, and persistence storage to store metadata and some metrics. As for the persistence storage, it supports three types of database: HBase3, Derby4, and Mysql

To run monitoring applications, Eagle requires the following dependencies.

  • For streaming platform dependencies

    • Storm: 0.9.3 or later
    • Kafka: 0.8.x or later
    • Java: 1.7.x
    • NPM (On MAC OS try “brew install node”)
  • For database dependencies (Choose one of them)

    • HBase: 0.98 or later
      • Hadoop5: 2.6.x is required
    • Mysql
      • Installation is required
    • Derby
      • No installation

Setup Cluster in Sandbox

To make thing easier you can try Eagle with an all-in-one sandbox VM, like HDP sandbox(HDP 2.2.4 is recommended). Next we will go with Hortonworks Sandbox 2.2.4 to setup a minimal requirement cluster with Storm and Kafka.

  1. Launch Ambari6
    • Enable Ambari in sandbox http://127.0.0.1:8000 (Click on Enable Button)
    • Login to Ambari UI http://127.0.0.1:8080/ with user:admin and password:admin
  2. Start Storm and Kafka via Ambari. Showing Storm as an example below. Restart Services

  3. (Optional) Start HBase via Ambari with root as HBase superuser add superuser add superuser

  4. Add Eagle service port. If the NAT network is used in a virtual machine, add port 9099 to “Port Forwarding” Port Forwarding

Footnotes

  1. All mentions of “storm” on this page represent Apache Storm.

  2. All mentions of “kafka” on this page represent Apache Kafka.

  3. All mentions of “hbase” on this page represent Apache HBase.

  4. All mentions of “derby” on this page represent Apache Derby.

  5. Apache Hadoop.

  6. All mentions of “ambari” on this page represent Apache Ambari.

Copyright © 2015 The Apache Software Foundation, Licensed under the Apache License, Version 2.0.
Apache Eagle, Eagle, Apache Hadoop, Hadoop, Apache HBase, HBase, Apache Hive, Hive, Apache Ambari, Ambari, Apache Spark, Spark, Apache Kafka, Kafka, Apache Storm, Storm, Apache Maven, Maven, Apache Tomcat, Tomcat, Apache Derby, Derby, Apache Cassandra, Cassandra, Apache ZooKeeper, ZooKeeper, Apache, the Apache feather logo, and the Apache project logo are trademarks of The Apache Software Foundation.