详解Docker⽆法正常启动的原因及解决办法
⼀、Docker启动异常表现:
1.状态反复restaring,⽤命令查看
$docker ps -a
CONTAINER ID  IMAGE COMMAND    CREATED  STATUS    PORTS  NAMES
21c09be88c11  :5000/xxx-tes/xxx_tes:1.0.6  "/usr/"  9 days ago    Restarting (1) Less than a second ago          xxx10
2.Docker⽇志有明显问题:
$docker logs [容器名/容器ID]
⼆、Docker启动异常的可能原因:
2.1.内存不够
Docker 启动⾄少需要2G内存,⾸先执⾏free -mh命令查看剩余内存是否⾜够
直接查看内存
$free -mh
total    used    free  shared buff/cache  available
Mem:    15G    14G    627M    195M    636M  726M
Swap:    0B    0B    0B
分析⽇志
有时候⼀瞬间内存过载溢出,导致部分进程被杀死,看起来内存也是够⽤的,事实上docker还是会反复重启,就需要通过docker⽇志和系统⽇志信的息来进⼀步分析:
分析docker⽇志
查看docker⽇志看到内存溢出的信息,要仔细翻阅才能到信息,并不是在最下⾯
$docker logs [容器名/容器ID]|less
Java HotSpot(TM) 64-Bit Server VM warning: INFO: os::commit_memory(0x0000000769990000, 1449590784, 0) failed; error='Cannot allocate memory' (errno=12) #
# There is insufficient memory for the Java Runtime Environment to continue.
# Native memory allocation (malloc) failed to allocate 1449590784 bytes for committing reserved memory.
# An error report file with more information is saved as:
# //hs_err_pid1.log
Java HotSpot(TM) 64-Bit Server VM warning: INFO: os::commit_memory(0x0000000769990000, 1449590784, 0) failed; error='Cannot allocate memory' (errno=12) #
# There is insufficient memory for the Java Runtime Environment to continue.
# Native memory allocation (malloc) failed to allocate 1449590784 bytes for committing reserved memory.
# An error report file with more information is saved as:
# /tmp/hs_err_pid1.log
Java HotSpot(TM) 64-Bit Server VM warning: INFO: os::commit_memory(0x0000000769990000, 1449590784, 0) failed; error='Cannot allocate memory' (errno=12) #
# There is insufficient memory for the Java Runtime Environment to continue.
# Native memory allocation (malloc) failed to allocate 1449590784 bytes for committing reserved memory.
# Can not save log file, dump to screen..
#
# There is insufficient memory for the Java Runtime Environment to continue.
# Native memory allocation (malloc) failed to allocate 1449590784 bytes for committing reserved memory.
# Possible reasons:
docker重启容器命令#  The system is out of physical RAM or swap space
#  In 32 bit mode, the process size limit was hit
# Possible solutions:
#  Reduce memory load on the system
#  Increase physical memory or swap space
#  Check if swap backing store is full
#  Use 64 bit Java on a 64 bit OS
#  Decrease Java heap size (-Xmx/-Xms)
#  Decrease number of Java threads
#  Decrease Java thread stack sizes (-Xss)
#  Set larger code cache with -XX:ReservedCodeCacheSize=
# This output file may be truncated or incomplete.
#
# Out of Memory Error (os_linux.cpp:2756), pid=1, tid=140325689620224
#
# JRE version: (7.0_79-b15) (build )
# Java VM: Java HotSpot(TM) 64-Bit Server VM (24.79-b02 mixed mode linux-amd64 compressed oops)
# Core dump written. Default location: //core or core.1
#
分析系统⽇志
查看系统⽇志,发现有⼤量由于内存溢出,进程被杀死的记录
$grep -i 'Out of Memory' /var/log/messages
Apr 7 10:04:02 centos106 kernel: Out of memory: Kill process 1192 (java) score 54 or sacrifice child
Apr 7 10:08:00 centos106 kernel: Out of memory: Kill process 2301 (java) score 54 or sacrifice child
Apr 7 10:09:59 centos106 kernel: Out of memory: Kill process 28145 (java) score 52 or sacrifice child
Apr 7 10:20:40 centos106 kernel: Out of memory: Kill process 2976 (java) score 54 or sacrifice child
Apr 7 10:21:08 centos106 kernel: Out of memory: Kill process 3577 (java) score 47 or sacrifice child
Apr 7 10:21:08 centos106 kernel: Out of memory: Kill process 3631 (java) score 47 or sacrifice child
Apr 7 10:21:08 centos106 kernel: Out of memory: Kill process 3634 (java) score 47 or sacrifice child
Apr 7 10:21:08 centos106 kernel: Out of memory: Kill process 3640 (java) score 47 or sacrifice child
Apr 7 10:21:08 centos106 kernel: Out of memory: Kill process 3654 (java) score 47 or sacrifice child
Apr 7 10:27:27 centos106 kernel: Out of memory: Kill process 6998 (java) score 51 or sacrifice child
Apr 7 10:27:28 centos106 kernel: Out of memory: Kill process 7027 (java) score 52 or sacrifice child
Apr 7 10:28:10 centos106 kernel: Out of memory: Kill process 7571 (java) score 42 or sacrifice child
Apr 7 10:28:10 centos106 kernel: Out of memory: Kill process 7586 (java) score 42 or sacrifice child
2.2.端⼝冲突
该docker监听端⼝已经被其他进程占⽤,⼀般此种问题容易出现在新部署的服务,或在原有机器上部署新的后台服务,所以在部署之前应该执⾏命令检查端⼝是否已经被占⽤,如果上线后发现占有则应改为可⽤端⼝再重启之。
检查命令:    $netstat -nltp|grep [规划的端⼝号]
三、对策
3.1.内存不够的对策:
对策1:
3.1.1 saltstack的minion在运⾏过久之后,可能占⽤⼤量内存,需要将其重启。重启命令可能有时并不起作⽤。主要检查运⾏状态,如果未成功停⽌,则重新重启;
对策2:
3.2.2 ELK⽇志收集程序或者其他java进程占⽤过⾼,⽤top和ps命令排查,谨慎确定进程的作⽤,在确保不影响业务的情况下,停⽌相关进程;
对策3:
释放被占⽤的内存(buff/cache):
$sync  #将内存数据写⼊磁盘
$echo 3 > /proc/sys/vm/drop_caches  #释放被占⽤的内存
对策4:
有时候并不是buff/cache过⾼导致内存不够⽤,确实是被很多必要的进程消耗掉了内存,那就需要从机器资源分配使⽤的层⾯去考虑和解决了。
3.2 端⼝冲突的对策
对策1:
⼀般此种问题容易出现在新部署的服务,或在原有机器上部署新的后台服务,所以在部署之前应该执⾏命令检查端⼝是否已经被占⽤,如果上线后发现占有则应改为可⽤端⼝再重启之。
检查命令:    $netstat -nltp|grep [规划的端⼝号]
以上就是本⽂的全部内容,希望对⼤家的学习有所帮助,也希望⼤家多多⽀持。

版权声明:本站内容均来自互联网,仅供演示用,请勿用于商业和其他非法用途。如果侵犯了您的权益请与我们联系QQ:729038198,我们将在24小时内删除。