优化,调整IO优先级
ionice -c 1 -n 1 -p 2728858
http://linux.51yip.com/search/ionice
https://www.cnblogs.com/lizhewei/p/13036463.html
参数优化
–heartbeat-interval ‘100’ 改为 500 –election-timeout ‘1000’ 改为 5000
测试1
etcdctl --cacert ca.crt --cert healthcheck-client.crt --key healthcheck-client.key check perf
60 / 60 Booooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooo! 100.00%1m0s
PASS: Throughput is 150 writes/s
PASS: Slowest request took 0.172822s
PASS: Stddev is 0.008926s
PASS
# etcdctl --endpoints https://10.100.0.100:2379 --cacert=/etc/etcd/pki/ca.crt --cert=/etc/etcd/pki/client.crt --key=/etc/etcd/pki/client.key check perf
60 / 60 Booooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooo! 100.00% 1m0s
PASS: Throughput is 150 writes/s
PASS: Slowest request took 0.092653s
PASS: Stddev is 0.006833s
PASS
测试2
关于 etcd 对磁盘的性能要求,以及MySQL高可用部署的节点要求 内容如下:
etcd磁盘性能要求:
etcd是磁盘IO敏感性的键值数据库,对磁盘性能的要求:99%的 fdatasync 请求延迟小于 10ms (即10000us)
相关材料: 磁盘要求:https://etcd.io/docs/v3.4.0/op-guide/hardware/#disks 测试工具:https://github.com/axboe/fio 测试方案:https://www.ibm.com/cloud/blog/using-fio-to-tell-whether-your-storage-is-fast-enough-for-etcd
获取工具包: http://mirror.centos.org/centos/7/os/x86_64/Packages/fio-3.7-2.el7.x86_64.rpm http://mirror.centos.org/altarch/7/os/armhfp/Packages/fio-3.7-2.el7.armv7hl.rpm
安装工具: yum -y install ./fio-3.7-2.el7.x86_64.rpm
测试性能: cd 磁盘挂载的数据目录 mkdir test-data fio –rw=write –ioengine=sync –fdatasync=1 –directory=test-data –size=22m –bs=2300 –name=mytest
结果分析: 测试结果中关注 fdatasync 指标中 99th ,要求小于 10000 (usec)
参数优化
--heartbeat-interval '100' 改为 500
--election-timeout '1000' 改为 5000
但是修改了这2个参数后,集群运行稳定,命令执行比较流畅,没有helm/kubectl执行失败
赵文宇: helm/kubectl 执行返回时间 0.2 秒和 2秒,在人的感知上差别不大