notfounduser's diary

## download elasticsearch & kibana [root@superset min]# wget https://artifacts.elastic.co/downloads/kibana/kibana-7.16.0-x86_64.rpm --2021-12-13 13:42:14-- https://artifacts.elastic.co/downloads/kibana/kibana-7.16.0-x86_64.rpm Resolving artifacts.elastic.co (artifacts.elastic.co)... 34.120.127.130, 2600:1901:0:1d7:: Connecting to artifacts.elastic.co (artifacts.elastic.co)|34.120.127.130|:443... connected. HTTP request sent, awaiting response... 200 OK Length: 286976720 (274M) [binary/octet-stream] Saving to: ‘kibana-7.16.0-x86_64.rpm’ 26% [================> ] 77,127,104 91.8MB/s ^C [root@superset min]# wget https://artifacts.elastic.co/downloads/elasticsearch/elasticsearch-7.16.0-x86_64.rpm --2021-12-13 13:42:39-- https://artifacts.elastic.co/downloads/elasticsearch/elasticsearch-7.16.0-x86_64.rpm Resolving artifacts.elastic.co (artifacts.elastic.co)... 34.120.127.130, 2600:1901:0:1d7:: Connecting to artifacts.elas

#set logstash [root@elk74 ~]# cd /etc/logstash/ [root@elk74 logstash]# ll 합계 44 drwxrwxr-x. 2 root root 6 3월 26 17:55 conf.d -rw-r--r--. 1 root root 2019 3월 26 17:55 jvm.options -rw-r--r--. 1 root root 8880 3월 26 17:55 log4j2.properties -rw-r--r--. 1 root root 959 5월 8 11:09 logs.yml -rw-r--r--. 1 root root 342 3월 26 17:55 logstash-sample.conf -rw-r--r--. 1 root root 8866 5월 8 08:58 logstash.yml -rw-r--r--. 1 root root 285 3월 26 17:55 pipelines.yml -rw-------. 1 root root 1696 3월 26 17:55 startup.options [root@elk74 logstash]# vi logs.yml input { jdbc { jdbc_driver_library => "/root/mysql-connector-java-8.0.16.jar" jdbc_driver_class => "com.mysql.jdbc.Driver" jdbc_connection_string => "jdbc:mysql://192.168.0.158:3306/es_db" jdbc_user => root jdbc_password => mypass123 jdbc_paging_enabled => true tracking_column => "unix_ts_in_secs" use_column_value => true tracking

#error message Access to XMLHttpRequest at 'http://192.168.0.102:9200/' from origin 'http://192.168.0.102' has been blocked by CORS policy: No 'Access-Control-Allow-Origin' header is present on the requested resource.  #solution [root@elkmaster ~]# vi /etc/elasticsearch/elasticsearch.yml [# # Make sure that the heap size is set to about half the memory available # on the system and that the owner of the process is allowed to use this # limit. # # Elasticsearch performs poorly when the system is swapping the memory. # # ---------------------------------- Network ----------------------------------- # # Set the bind address to a specific IP (IPv4 or IPv6): # #network.host: 192.168.0.1 network.host: 0.0.0.0 network.bind_host: 0.0.0.0 transport.host: 127.0.0.1 http.host: 0.0.0.0 http.cors.enabled: true http.cors.allow-origin: "*"

#make repo  [root@localhost java-se-8u41-ri]# cat /etc/yum.repos.d/elasticsearch.repo [elasticsearch] name=Elasticsearch repository for 7.x packages #download package  [root@localhost java-se-8u41-ri]# yum install --enablerepo=elasticsearch kibana --downloadonly --downloaddir=/root/elastic Loaded plugins: fastestmirror, langpacks base | 3.6 kB 00:00:00 elasticsearch | 1.3 kB 00:00:00 extras | 2.9 kB 00:00:00 updates

#download package  [root@localhost java-se-8u41-ri]# cat /etc/yum.repos.d/elasticsearch.repo [elasticsearch] name=Elasticsearch repository for 7.x packages baseurl=https://artifacts.elastic.co/packages/7.x/yum gpgcheck=1 gpgkey=https://artifacts.elastic.co/GPG-KEY-elasticsearch enabled=0 autorefresh=1 type=rpm-md [root@localhost java-se-8u41-ri]# [root@localhost java-se-8u41-ri]# yum install --enablerepo=elasticsearch elasticsearch --downloadonly --downloaddir=/root/elastic Loaded plugins: fastestmirror, langpacks elasticsearch | 1.3 kB 00:00:00 elasticsearch/primary | 115 kB 00:00:01 Loading mirror speeds from cached hostfile * base: mirror.kakao.com * extras: mirror.kaka

#show error [2020-03-27T00:51:26,356][INFO ][o.e.d.DiscoveryModule    ] [master-node1] using discovery type [zen] and host providers [settings] [2020-03-27T00:51:34,430][INFO ][o.e.n.Node               ] [master-node1] initialized [2020-03-27T00:51:34,430][INFO ][o.e.n.Node               ] [master-node1] starting ... [2020-03-27T00:51:36,061][INFO ][o.e.t.TransportService   ] [master-node1] publish_address {172.20.0.2:9300}, bound_addresses {[::]:9300} [2020-03-27T00:51:36,238][INFO ][o.e.b.BootstrapChecks    ] [master-node1] bound or publishing to a non-loopback address, enforcing bootstrap checks ERROR: [1] bootstrap checks failed [1]: max virtual memory areas vm.max_map_count [65530] is too low, increase to at least [262144] #fix error [root@localhost ~]# sysctl -w vm.max_map_count=262144 vm.max_map_count = 262144

install ELK on docker #install docker [root@minimacent ~]# yum install -y yum-utils device-mapper-persistent-data lvm2 #install docker [root@localhost ~]# yum-config-manager --add-repo https://download.docker.com/linux/centos/docker-ce.repo Loaded plugins: fastestmirror, langpacks adding repo from: https://download.docker.com/linux/centos/docker-ce.repo grabbing file https://download.docker.com/linux/centos/docker-ce.repo to /etc/yum.repos.d/docker-ce.repo repo saved to /etc/yum.repos.d/docker-ce.repo #install docker [root@localhost ~]# yum-config-manager --add-repo https://download.docker.com/linux/centos/docker-ce.repo Loaded plugins: fastestmirror, langpacks #install docker [root@minimacent ~]# yum install docker-ce docker-ce-cli containerd.io Loaded plugins: fastestmirror, langpacks Loading mirror speeds from cached hostfile * base: mirror.kakao.com * extras: mirror.kakao.com * updates: mirror.kakao.com #install docker [root@minimacent ~]# yum i

logstash로 data 수집 후, kibana index 작성중 이하의 에러가 발생 Error "forbidden" 권한 문제? #edit privileges PUT /_all/_settings {"index.blocks.read_only_allow_delete": null}

3개의 데이터 전용 노드, 1개의 마스터 전용 노드로 구성합니다. Master Node가 설치된 서버에는 Kibana, Logstash 을 설치합니다. 데이터는 Master Node 를 통해서만 색인됩니다. 서버스펙 운영 서버에 필요한 CPU, RAM, 저장장치는 총4대   – OS : Centos 7   – CPU : 4core   – RAM : 4GB   – JAVA : Openjdk 8 Cluster 설치 es-master는 마스터 es-client1는 데이터, es-client2는 데이터, es-client3는 데이터 cluster.name: es-demo node.name: node-1   (각각node-2, node-3으로 설정) node.master: true      (마스터 노드만 true로 설정, 데이터 노드는 false) node.data: false          (마스터 노드만  false 로 설정, 데이터 노드는 true) 기동 장애 발생 시 대응.. rm -rf /var/lib/elasticsearch/nodes/0 로 데이터 삭제 후, 재설정 실행 결과 es-client1, es-client2, es-client3에 각각 분산되어 데이터 수집 보나스 테스트(Metricbeat로 apache,system 모니터링) [root@localhost modules.d]# metricbeat modules list Enabled: apache system vi apache.yml # Module: apache # Docs: https://www.elastic.co/guide/en/beats/metricbeat/6.4/metricbeat-module-apache.html – modu