| ... | @@ -4,6 +4,80 @@ In this document we install the elk stack and connect it to elk-stack. We are ru |
... | @@ -4,6 +4,80 @@ In this document we install the elk stack and connect it to elk-stack. We are ru |
|
|
|
|
|
|
|
|
|
|
|
|
We are sending gitlab logs by using filebeat to logstash to a remote machine.
|
|
We are sending gitlab logs by using filebeat to logstash to a remote machine.
|
|
|
|
## Install-guide Version 2.0
|
|
|
|
|
|
|
|
Upgrading your elk-stack to version 6.2.4. All the configurations all the same but download a different packages.
|
|
|
|
|
|
|
|
### Elk-stack installed
|
|
|
|
| Version | Service |
|
|
|
|
|:----:|:----:|
|
|
|
|
| 6.3.0 | Elasticsearch |
|
|
|
|
| 6.3.0 | Kibana |
|
|
|
|
| 6.3.0 | Logstash |
|
|
|
|
| 6.3.0 | Filebeat|
|
|
|
|
|
|
|
|
You Install java as in installation guide 1.0.
|
|
|
|
|
|
|
|
### Installing elasticsearch
|
|
|
|
|
|
|
|
https://www.elastic.co/guide/en/elasticsearch/reference/6.3/deb.html
|
|
|
|
|
|
|
|
```
|
|
|
|
sudo apt-get install apt-transport-https
|
|
|
|
sudo apt-get update
|
|
|
|
wget https://artifacts.elastic.co/downloads/elasticsearch/elasticsearch-6.3.0.deb
|
|
|
|
wget https://artifacts.elastic.co/downloads/elasticsearch/elasticsearch-6.3.0.deb.sha512
|
|
|
|
shasum -a 512 -c elasticsearch-6.3.0.deb.sha512
|
|
|
|
sudo dpkg -i elasticsearch-6.3.0.deb
|
|
|
|
```
|
|
|
|
```
|
|
|
|
sudo systemctl enable elasticsearch \
|
|
|
|
sudo systemctl restart elasticsearch
|
|
|
|
```
|
|
|
|
### Installing kibana
|
|
|
|
|
|
|
|
https://www.elastic.co/guide/en/kibana/6.3/deb.html
|
|
|
|
```
|
|
|
|
echo "deb https://artifacts.elastic.co/packages/6.x/apt stable main" | sudo tee -a /etc/apt/sources.list.d/elastic-6.x.list
|
|
|
|
sudo apt-get update && sudo apt-get install kibana
|
|
|
|
|
|
|
|
```
|
|
|
|
```
|
|
|
|
sudo systemctl enable kibana \
|
|
|
|
sudo systemctl restart kibana
|
|
|
|
```
|
|
|
|
### Installing logstash
|
|
|
|
|
|
|
|
https://www.elastic.co/guide/en/logstash/6.3/installing-logstash.html
|
|
|
|
```
|
|
|
|
sudo apt-get update && sudo apt-get install logstash
|
|
|
|
```
|
|
|
|
```
|
|
|
|
sudo systemctl enable logstash \
|
|
|
|
sudo systemctl restart logstash
|
|
|
|
```
|
|
|
|
|
|
|
|
### Installing filebeat
|
|
|
|
|
|
|
|
https://www.elastic.co/guide/en/beats/filebeat/6.3/filebeat-installation.html
|
|
|
|
```
|
|
|
|
curl -L -O https://artifacts.elastic.co/downloads/beats/filebeat/filebeat-6.3.0-amd64.deb
|
|
|
|
sudo dpkg -i filebeat-6.3.0-amd64.deb
|
|
|
|
```
|
|
|
|
```
|
|
|
|
sudo systemctl enable filebeat \
|
|
|
|
sudo systemctl restart filebeat
|
|
|
|
```
|
|
|
|
|
|
|
|
### Troubleshooting
|
|
|
|
|
|
|
|
If you don't see any logs coming for your Kibana or it doesn't recognize index pattern you should check that the Filebeat is sending logs correctly to Logstash. The Filebeat log file is at /var/log/filebeat and it should look something like this when the logs are sent to the Logstash:
|
|
|
|
|
|
|
|
|
|
|
|
The Logstash log is located in /var/log/logstash and it should look something like this:
|
|
|
|
|
|
|
|
|
|
|
|
If you get a ERR Connecting error publishing events (retrying): dial tcp {remote.logstash.host.ip}:5044 in your Filebeat log it means that there is something wrong in the connection. You should check both Filebeat and Logstash config files from typos and check that the port 5044 is available in the firewall.
|
|
|
|
|
|
|
|
## Install-guide Version 1.0
|
|
## Install-guide Version 1.0
|
|
|
|
|
|
| ... | @@ -168,78 +242,3 @@ Now you can access Kibana in your browser's localhost. To get the logs you have |
... | @@ -168,78 +242,3 @@ Now you can access Kibana in your browser's localhost. To get the logs you have |
|
|
|
|
|
|
|
It is supposed to look something like this when you have logs coming from Filebeat to your Kibana.
|
|
It is supposed to look something like this when you have logs coming from Filebeat to your Kibana.
|
|
|
|
|
|
|
|
## Install-guide Version 2.0
|
|
|
|
|
|
|
|
|
|
Upgrading your elk-stack to version 6.2.4. All the configurations all the same but download a different packages.
|
|
|
|
|
|
|
|
|
|
### Elk-stack installed
|
|
|
|
|
| Version | Service |
|
|
|
|
|
|:----:|:----:|
|
|
|
|
|
| 6.3.0 | Elasticsearch |
|
|
|
|
|
| 6.3.0 | Kibana |
|
|
|
|
|
| 6.3.0 | Logstash |
|
|
|
|
|
| 6.3.0 | Filebeat|
|
|
|
|
|
|
|
|
|
|
You Install java as in installation guide 1.0.
|
|
|
|
|
|
|
|
|
|
### Installing elasticsearch
|
|
|
|
|
|
|
|
|
|
https://www.elastic.co/guide/en/elasticsearch/reference/6.3/deb.html
|
|
|
|
|
|
|
|
|
|
```
|
|
|
|
|
sudo apt-get install apt-transport-https
|
|
|
|
|
sudo apt-get update
|
|
|
|
|
wget https://artifacts.elastic.co/downloads/elasticsearch/elasticsearch-6.3.0.deb
|
|
|
|
|
wget https://artifacts.elastic.co/downloads/elasticsearch/elasticsearch-6.3.0.deb.sha512
|
|
|
|
|
shasum -a 512 -c elasticsearch-6.3.0.deb.sha512
|
|
|
|
|
sudo dpkg -i elasticsearch-6.3.0.deb
|
|
|
|
|
```
|
|
|
|
|
```
|
|
|
|
|
sudo systemctl enable elasticsearch \
|
|
|
|
|
sudo systemctl restart elasticsearch
|
|
|
|
|
```
|
|
|
|
|
### Installing kibana
|
|
|
|
|
|
|
|
|
|
https://www.elastic.co/guide/en/kibana/6.3/deb.html
|
|
|
|
|
```
|
|
|
|
|
echo "deb https://artifacts.elastic.co/packages/6.x/apt stable main" | sudo tee -a /etc/apt/sources.list.d/elastic-6.x.list
|
|
|
|
|
sudo apt-get update && sudo apt-get install kibana
|
|
|
|
|
|
|
|
|
|
```
|
|
|
|
|
```
|
|
|
|
|
sudo systemctl enable kibana \
|
|
|
|
|
sudo systemctl restart kibana
|
|
|
|
|
```
|
|
|
|
|
### Installing logstash
|
|
|
|
|
|
|
|
|
|
https://www.elastic.co/guide/en/logstash/6.3/installing-logstash.html
|
|
|
|
|
```
|
|
|
|
|
sudo apt-get update && sudo apt-get install logstash
|
|
|
|
|
```
|
|
|
|
|
```
|
|
|
|
|
sudo systemctl enable logstash \
|
|
|
|
|
sudo systemctl restart logstash
|
|
|
|
|
```
|
|
|
|
|
|
|
|
|
|
### Installing filebeat
|
|
|
|
|
|
|
|
|
|
https://www.elastic.co/guide/en/beats/filebeat/6.3/filebeat-installation.html
|
|
|
|
|
```
|
|
|
|
|
curl -L -O https://artifacts.elastic.co/downloads/beats/filebeat/filebeat-6.3.0-amd64.deb
|
|
|
|
|
sudo dpkg -i filebeat-6.3.0-amd64.deb
|
|
|
|
|
```
|
|
|
|
|
```
|
|
|
|
|
sudo systemctl enable filebeat \
|
|
|
|
|
sudo systemctl restart filebeat
|
|
|
|
|
```
|
|
|
|
|
|
|
|
|
|
### Troubleshooting
|
|
|
|
|
|
|
|
|
|
If you don't see any logs coming for your Kibana or it doesn't recognize index pattern you should check that the Filebeat is sending logs correctly to Logstash. The Filebeat log file is at /var/log/filebeat and it should look something like this when the logs are sent to the Logstash:
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
The Logstash log is located in /var/log/logstash and it should look something like this:
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
If you get a ERR Connecting error publishing events (retrying): dial tcp {remote.logstash.host.ip}:5044 in your Filebeat log it means that there is something wrong in the connection. You should check both Filebeat and Logstash config files from typos and check that the port 5044 is available in the firewall.
|
|
|
|
|
|
|
|
|
\ No newline at end of file |
|
|
