elasticsearch-logstash-kibana

ELK search SetUp

Get GPG Key and Repo file from link

Import the Elasticsearch GPG Key:

rpm --import https://artifacts.elastic.co/GPG-KEY-elasticsearch

Create Repo file at /etc/elasticsearch with the name of elasticsearch.repo with belo content

[elasticsearch]
name=Elasticsearch repository for 8.x packages
baseurl=https://artifacts.elastic.co/packages/8.x/yum
gpgcheck=1
gpgkey=https://artifacts.elastic.co/GPG-KEY-elasticsearch
enabled=1
autorefresh=1
type=rpm-md

Install Elasticsearch, Logstash and Kibana

yum install elasticsearch logstash kibana -y

If you want to see elastic search in UI, need to edit elasticsearch.yml shown as below

vi /etc/elasticsearch/elasticsearch.yml
--------------------------------------------------
network.host: 0.0.0.0
http.port: 9200
xpack.security.enabled: true         #also need to modify xpack.security.enabled as false
---------------------------------------------------

Start elasticsearch service

service elasticsearch start

Check status of Elastic Search

service elasticsearch status

Check elasticsearch in UI: Allow port=9200 in security group :9200

Kibana SetUp

Open "kibana.yml" and edit below details

vi /etc/kibana/kibana.yml
-------------------------------------------------
server.port: 5601
server.host: "0.0.0.0"
elasticsearch.hosts: ["http://localhost:9200"]
-------------------------------------------------

Start kibana service

service kibana start

Check status of kibana

service kibana status

Check Kibana in UI: Allow port=5601 in security group :5601

Clone application and send logs to Elasticsearch

Create log file with below commands

cd springboot-elk

mkdir -p .mvn
echo "--add-opens java.base/java.lang=ALL-UNNAMED" > .mvn/jvm.config

Run application for logs:
   
mvn spring-boot:run
  
Check log file at /root/logback

Run logstash command to send logs to elasticsearch

cp logback.conf /usr/share/logstash/logback.conf        # copy logback configuration file
/usr/share/logstash/bin/logstash -f logback.conf        # sending logs to elastic search

Click on dashboard

Click on create new view

Click on save data view to kibana

Click on discover and see logs