Quick and Dirty: Install and setup Elasticsearch, Logstash, and Kibana

Quick and Dirty: Install and setup Elasticsearch, Logstash, and Kibana

First you obviously need to download all of the packages. You can get them from HERE. Its also a given that you have Apache webserver installed and running.

For me I like to use the TGZ files. Im kinda funny about not getting files spread out all over the place and I like to keep them all together so I put all the files into /opt and exploded them there…

tar -zxvf *.tar

I setup my symlinks…

ln -s /opt/elasticsearch-1.1.1 ES
ln -s /opt/logstash-1.4.0 LG
ln -s /opt/kibana-3.0.1 KB

From here you pretty much are customizing your install if you already know what you are wanting to log and track. In my example I was just logging all the log files found in /var/*

First I configured a conf file for LG and ES. Here im just telling LG to log all the *log files it finds under /var/* Im doing this as root and this is just on a local vm so im not particularly worried about permissions or security. Just know that you dont want to do it this way in a prod environment.

CONF FILE:  logstash-apache.conf     I created this and put it in /opt/LG/bin/

input {
file {
path => "/var/*/*log"
start_position => beginning
}
}
filter {
if [path] =~ "access" {
mutate { replace => { "type" => "apache_access" } }
grok {
match => { "message" => "%{COMBINEDAPACHELOG}" }
}
}
date {
match => [ "timestamp" , "dd/MMM/yyyy:HH:mm:ss Z" ]
}
}
output {
elasticsearch {
host => localhost
}
stdout { codec => rubydebug }
}

For more info on that conf file check out the Logstash tutorial here.

Now you want to copy everything that’s in the /opt/KB/ dir into a dir in your webserver. For me…

cp /opt/KB/* /var/www/html/

You will then need to edit a file. Open “config.js” and find the line that says “elasticsearch:” This is the location of your webserver. Since mine is locally hosted on a VM, mine reads:

elasticsearch: “http://localhost.localdomain:9200”,

From here you are ready to start things up. First start Elasticsearch, then Logstash and then we will fire up Kibana.

/opt/ES/bin/elasticsearch
/opt/LG/bin/logstash -f logstash-apache.conf

Then in your browser go to: http://localhost.localdomain:9200

You should now be looking at the default Kibana page. It will have some further info on getting started and how to change your default startup page in Kibana.