Elasticsearch集架构的部署和调优（⼀）
[root@es-node1 ~]# mkdir /usr/java
[root@es-node1 ~]# tar zxvf jdk1.8.0_ -C /usr/java/
[root@es-node1 ~]# cp /etc/profile /etc/profile.back
[root@es-node1 ~]# vim /etc/profile
export JAVA_HOME=/usr/java/jdk1.8.0_131/
export CLASSPATH=$CLASSPATH:$JAVA_HOME/lib:$JAVA_HOME/jre/lib
export PATH=$JAVA_HOME/bin:$JAVA_HOME/jre/bin:$PATH:$HOMR/bin
[root@es-node1 ~]# source /etc/profile
[root@es-node1 ~]# java -version
java version "1.8.0_131"
Java(TM) SE Runtime Environment (build 1.8.0_131-b11)
Java HotSpot(TM) 64-Bit Server VM (build 25.131-b11, mixed mode)
[Elasticsearch]
[root@es-node1 ~]# wget /downloads/elasticsearch/elasticsearch-6.3.
[root@es-node1 ~]# tar zxvf elasticsearch-6.3. -C /usr/src/
[root@es-node1 ~]# mv /usr/src/elasticsearch-6.3.2/ /usr/local/elasticsearch
[root@es-node1 ~]# useradd elasticsearch
[root@es-node1 ~]# chown -R elasticsearch:elasticsearch /usr/local/elasticsearch/
【Elasticsearch优化】
为了使得Elasticsearch获得⾼效稳定的性能，需要对系统和JVM两个⽅⾯进⾏优化
[root@es-node1 ~]# vim /f
fs.file-max=655360 #最⼤打开⽂件描述符数，建议修改为655360或者更⾼
vm.max_map_count=262144 #直接影响java线程数量，⽤来限制⼀个进程可以⽤于VM（虚拟内存区域⼤⼩）默认为65530，建议修改为262144或者更⾼[root@es-node1 ~]#vim /etc/f
* soft nproc 20480
* hard nproc 20480
* soft nofile 65536
* hard nofile 65536
* soft memlock unlimited
* hard memlock unlimited
[root@es-node1 ~]# sed -i '5s#4096#20480#g' /etc/security/limits.f
#JVM调优
JVM调优主要是针对Elasticsearch的JVM内存资源进⾏优化，elasticsearch的内存资源配置⽂件为jvm.options
[root@es-node1 ~]# vim /usr/local/elasticsearch/config/jvm.options #根据服务器内存⼤⼩，进⾏修改合适的值，建议修改服务器物理内存的⼀半最佳
-Xms1g
-Xmx1g
[root@es-node1 ~]# egrep -v "#|^$" /usr/local/elasticsearch/l
cluster.name: es_data #配置集名称
node.name: es_node1 #节点名称
node.master: true #指定了该节点是否有资格选举master，默认为true
node.data: true #指定该节点是否存储索引数据，默认为true，表⽰为数据存储
path.data: /usr/local/elasticsearch/data #设置索引数据存储的位置
path.logs: /usr/local/elasticsearch/logs #设置⽇志⽂件存储的位置
<_lock: true #设置true⽤来锁住物理内存
log4j与log4j2indices.fielddata.cache.size: 50mb #索引字段缓存⼤⼩
network.host: 0.0.0.0 #监听地址
http.port: 9200 #Elasticsearch对外提供的http端⼝
[root@es-node1 config]# vim /usr/local/elasticsearch/config/jvm.options
-Xms2g
-Xmx2g
-XX:+UseConcMarkSweepGC
-XX:CMSInitiatingOccupancyFraction=75
-XX:+UseCMSInitiatingOccupancyOnly
-XX:+DisableExplicitGC
-XX:+AlwaysPreTouch
-server
-Xss1m
-Djava.awt.headless=true
-ding=UTF-8
-sys=true
-
Djdk.io.permissionsUseCanonicalPath=true
-Unsafe=true
-KeySetOptimization=true
-ler.maxCapacityPerThread=0
-Dlog4j.shutdownHookEnabled=false
-Dlog4j2.disable.jmx=true
-Dlog4j.skipJansi=true
-XX:+HeapDumpOnOutOfMemoryError
[root@es-node1 ~]#mkdir /usr/local/elasticsearch/data/ -p
[root@es-node1 ~]#chown -R elasticsearch:elasticsearch /usr/local/elasticsearch/
[root@es-node1 ~]# su - elasticsearch
[elasticsearch@es-node1 ~]$ /usr/local/elasticsearch/bin/elasticsearch -d
p s：当你的elasticsearch启动的时候，什么错误都没有报，但是就是端⼝进程没起来·，此时查看⽇志信息，输出如下，
这是因为⽬录权限属主属组是root账号，我们只需chown elasticsearch：elasticsearch -R /usr/local/elasticsearch/即可
tail /usr/local/elasticsearch/logs/es_cluster.log
at java.nio.ateAndCheckIsDirectory(Files.java:781) ~[?:1.8.0_131]
at java.nio.ateDirectories(Files.java:767) ~[?:1.8.0_131]
at v.NodeEnvironment.<init>(NodeEnvironment.java:203) ~[elasticsearch-6.3.2.jar:6.3.2]
at de.Node.<init>(Node.java:270) ~[elasticsearch-6.3.2.jar:6.3.2]
at de.Node.<init>(Node.java:252) ~[elasticsearch-6.3.2.jar:6.3.2]
at org.elasticsearch.bootstrap.Bootstrap$5.<init>(Bootstrap.java:213) ~[elasticsearch-6.3.2.jar:6.3.2]
at org.elasticsearch.bootstrap.Bootstrap.setup(Bootstrap.java:213) ~[elasticsearch-6.3.2.jar:6.3.2]
at org.elasticsearch.bootstrap.Bootstrap.init(Bootstrap.java:326) ~[elasticsearch-6.3.2.jar:6.3.2]
at org.elasticsearch.bootstrap.Elasticsearch.init(Elasticsearch.java:136) ~[elasticsearch-6.3.2.jar:6.3.2]
... 6 more
【Elasticsearch图形化⼯具】
root@es-node1 ~]# yum install -y nodejs npm git （ps：如果yum不能正常安装npm和nodejs的话，采⽤源码吧）
源码安装
wget /dist/v9.8.0/node-v9.8.
xz -d node-v9.8.
tar xvf node-v9.8.0-linux-x64.tar
vim /etc/profile
export NODE_HOME=/root/node-v9.8.0-linux-x64/
export PATH=$PATH:$NODE_HOME/bin
[root@es-node1 ~]# npm -v
5.6.0
[root@es-node1 ~]# node -v
v9.8.0
[root@es-node1 ~]# cd /usr/local/
[root@es-node1 local]# git clone git://github/mobz/elasticsearch-head.git #安装head插件
[root@es-node1 local]# npm config set registry registry. #采⽤淘宝源
[root@es-node1 elasticsearch]# npm install #安装过程中，有点⼩意外，纠结了好久才到办法，确实是⼀个坑
npm ERR!
npm ERR! If you are behind a proxy, please make sure that the
npm ERR! 'proxy' config is set properly. See: 'npm help config'
npm ERR! A complete log of this run can be found in:
npm ERR! /root/.npm/_logs/2018-08-22T06_36_25_382Z-debug.log
或者
npm ERR! code ENOTFOUND
npm ERR! errno ENOTFOUND
npm ERR! network request to registry./phantomjs-prebuilt failed, reason: getaddrinfo ENOTFOUND registry. registry.:80 npm ERR! network This is a problem related to network connectivity.
npm ERR! network In most cases you are behind a proxy or have bad network settings.
npm ERR! network
npm ERR! network If you are behind a proxy, please make sure that the
npm ERR! network 'proxy' config is set properly. See: 'npm help config'
npm ERR! A complete log of this run can be found in:
npm ERR! /root/.npm/_logs/2018-08-25T09_04_38_243Z-debug.log
[root@localhost elasticsearch-head]# npm install
npm ERR! code ENOTFOUND
npm ERR! errno ENOTFOUND
npm ERR! network request to registry./grunt-contrib-jasmine failed, reason: getaddrinfo ENOTFOUND registry. registry.:80 npm ERR! network This is a problem related to network connectivity.
npm ERR! network In most cases you are behind a proxy or have bad network settings.
npm ERR! network
npm ERR! network If you are behind a proxy, please make sure that the
npm ERR! network 'proxy' config is set properly. See: 'npm help config'
npm ERR! A complete log of this run can be found in:
npm ERR! /root/.npm/_logs/2018-08-25T09_04_44_521Z-debug.log
解决⽅法：
npm config get proxy
npm config get https-proxy
npm config set registry /
npm ERR! code ELIFECYCLE
npm ERR! errno 1
npm ERR! phantomjs-prebuilt@2.1.16 install: `node install.js`
npm ERR! Exit status 1
npm ERR!
npm ERR! Failed at the phantomjs-prebuilt@2.1.16 install script.
npm ERR! This is probably not a problem with npm. There is likely additional logging output above.
npm ERR! A complete log of this run can be found in:
npm ERR! /root/.npm/_logs/2018-08-12T07_42_48_334Z-debug.log
解决办法：
# npm install phantomjs-prebuilt@2.1.14 --ignore-scripts
npm install即可
[root@es-node1 elasticsearch]# vim /usr/local/elasticsearch-head/_site/app.js
[root@es-node1 elasticsearch-head]# nohup npm run start & #后台启动head插件服务，默认端⼝为9100
web浏览器192.168.37.134:9100/验证
【其他节点配置】
【192.168.37.135】es-node2
node2节点值作为数据存储的节点⾓⾊，不参与主节点选举
#vim /usr/local/elasticsearch/l
cluster.name: es_data
node.name: es.node2
node.master: false
node.data: true
path.data: /usr/local/elasticsearch/data/
path.logs: /usr/local/elasticsearch/logs
<_lock: true
indices.fielddata.cache.size: 50mb
network.host: 0.0.0.0
http.port: 9200
vim /usr/local/elasticsearch/config/jvm.options
-Xms2g
-Xmx2g
-XX:+UseConcMarkSweepGC
-XX:CMSInitiatingOccupancyFraction=75
-XX:+UseCMSInitiatingOccupancyOnly
-XX:+DisableExplicitGC
-XX:+AlwaysPreTouch
-server
-Xss1m
-Djava.awt.headless=true
-ding=UTF-8
-sys=true
-Djdk.io.permissionsUseCanonicalPath=true
-Unsafe=true
-
-ler.maxCapacityPerThread=0
-Dlog4j.shutdownHookEnabled=false
-Dlog4j2.disable.jmx=true
-Dlog4j.skipJansi=true
-XX:+HeapDumpOnOutOfMemoryError
【192.168.37.136】es-node3节点配置（ps：这⾥的node3与node1配置都是⼀样的，都有被选举成为master主节点的可能）cluster.name: es_data
node.name: es.node3
node.master: true
node.data: true
path.data: /usr/local/elasticsearch/data/
path.logs: /usr/local/elasticsearch/logs
<_lock: true
indices.fielddata.cache.size: 50mb
network.host: 0.0.0.0
http.port: 9200
vim /usr/local/elasticsearch/config/jvm.options
-Xms2g
-Xmx2g
-XX:+UseConcMarkSweepGC
-XX:CMSInitiatingOccupancyFraction=75
-XX:+UseCMSInitiatingOccupancyOnly
-XX:+DisableExplicitGC
-XX:+AlwaysPreTouch
-server
-Xss1m
-Djava.awt.headless=true
-ding=UTF-8
-
-Djdk.io.permissionsUseCanonicalPath=true
-Unsafe=true
-KeySetOptimization=true
-ler.maxCapacityPerThread=0
-Dlog4j.shutdownHookEnabled=false
-Dlog4j2.disable.jmx=true
-Dlog4j.skipJansi=true
-XX:+HeapDumpOnOutOfMemoryError
【总结】
Elasticsearch节点⾓⾊类型
在⽣产中，⾼并发的场景容易出现脑裂问题，⽽在Elasticsearch集亦是如此，Elasticsearch集中的每个节点都可以成为主节点，存储数据，提供查询服务
⽽这些事由两个属性参数控制的也就是node,master和node.data
1：这两种组合表⽰这个节点有资格成为主节点，⼜要存储数据，如果该节点真的成为主节点的话，那么⾃⼰还要存储数据，压⼒是相当⼤的，⽣产不建议这样配置node.master: true
node.data:true
2：这种组合是数据存储节点，既不参与选举，也不会成为成为主节点，因为没有成为主节点的资格，只能是数据存储节点
node.master：false
node.data：true
3：拥有参与选举主节点的资格，不会存储节点数据，该节点我们称之为master主节点
node.master：true
node.data: false
4：该节点为客户端节点，数据存储和主节点资格两项都没有，只能处理客户端请求，对海量的请求进⾏均衡
node.master：false
node.data：false
在⼀个⽣产集中我们可以对这些节点的职责进⾏划分。
建议集中设置3台以上的节点作为master节点【node.master: true node.data: false】
这些节点只负责成为主节点，维护整个集的状态。
再根据数据量设置⼀批data节点【node.master: false node.data: true】
这些节点只负责存储数据，后期提供建⽴索引和查询索引的服务，这样的话如果⽤户请求⽐较频繁，这些节点的压⼒也会⽐较⼤
所以在集中建议再设置⼀批client节点【node.master: false node.data: true】
这些节点只负责处理⽤户请求，实现请求转发，负载均衡等功能。
版权声明：本站内容均来自互联网，仅供演示用，请勿用于商业和其他非法用途。如果侵犯了您的权益请与我们联系QQ:729038198，我们将在24小时内删除。
发表评论