Weblogic Cluster BEA-000116故障处理
部署环境:Weblogic 9.2 MP3 for linux
实验:
Weblogic Cluster: (MServer1,MServer2) 7222
Multicast IP: 239.192.0.0
AdminServer: 192.168.1.33:7001
MServer1: 192.168.1.33:7201
MServer2: 192.168.1.33:7202
Machine1: (MServer1) NodeManager(192.168.1.33:5556)
Machine2: (MServer2) NodeManager(192.168.1.33:5557)
注意:本实验环境各个Server SSL设置---->Hostname Verification: NONE
描述:
(NodeManager 成功启动)
当启动某ManagedServer时分别报以下错误:
/home/weblogic/bea/user_projects/domains/clustersys/servers/MServer_1/logs/MServer_1.out
实验:
Weblogic Cluster: (MServer1,MServer2) 7222
Multicast IP: 239.192.0.0
AdminServer: 192.168.1.33:7001
MServer1: 192.168.1.33:7201
MServer2: 192.168.1.33:7202
Machine1: (MServer1) NodeManager(192.168.1.33:5556)
Machine2: (MServer2) NodeManager(192.168.1.33:5557)
注意:本实验环境各个Server SSL设置---->Hostname Verification: NONE
描述:
(NodeManager 成功启动)
当启动某ManagedServer时分别报以下错误:
/home/weblogic/bea/user_projects/domains/clustersys/servers/MServer_1/logs/MServer_1.out
Dec 16, 2010 9:12:56 PM CST> <Info> <WebLogicServer> <BEA-000215>
<Loaded License : /home/weblogic/bea/license.bea>
<Dec 16, 2010 9:12:57 PM CST> <Notice> <WebLogicServer> <BEA-000365> <Server state changed to STARTING>
<Dec 16, 2010 9:12:57 PM CST> <Info> <WorkManager> <BEA-002900> <Initializing self-tuning thread pool>
<Dec 16, 2010 9:12:58 PM CST> <Notice> <Log Management> <BEA-170019> <The server log file
<Dec 16, 2010 9:13:15 PM CST> <Notice> <Security> <BEA-090082> <Security initializing using security realm myrealm.>
<Dec 16, 2010 9:13:31 PM CST> <Notice> <WebLogicServer> <BEA-000365> <Server state changed to STANDBY>
<Dec 16, 2010 9:13:31 PM CST> <Notice> <WebLogicServer> <BEA-000365> <Server state changed to STARTING>
<Loaded License : /home/weblogic/bea/license.bea>
<Dec 16, 2010 9:12:57 PM CST> <Notice> <WebLogicServer> <BEA-000365> <Server state changed to STARTING>
<Dec 16, 2010 9:12:57 PM CST> <Info> <WorkManager> <BEA-002900> <Initializing self-tuning thread pool>
<Dec 16, 2010 9:12:58 PM CST> <Notice> <Log Management> <BEA-170019> <The server log file
<Dec 16, 2010 9:13:15 PM CST> <Notice> <Security> <BEA-090082> <Security initializing using security realm myrealm.>
<Dec 16, 2010 9:13:31 PM CST> <Notice> <WebLogicServer> <BEA-000365> <Server state changed to STANDBY>
<Dec 16, 2010 9:13:31 PM CST> <Notice> <WebLogicServer> <BEA-000365> <Server state changed to STARTING>
<Dec 16, 2010 9:13:34 PM CST> <Error> <Cluster> <BEA-000116>
<Failed to join cluster Cluster_1 at address 239.192.0.0 due to: java.SocketException:
No such device.java.SocketException:
No such device
at java.PlainDatagramSocketImpl.join(Native Method)
at java.PlainDatagramSocketImpl.join(PlainDatagramSocketImpl.java:172)
at java.MulticastSocket.joinGroup(MulticastSocket.java:276)
at weblogic.cluster.MulticastFragmentSocket.initializeMulticastSocket(MulticastFragmentSocket.java:112)
at weblogic.cluster.MulticastFragmentSocket.start(MulticastFragmentSocket.java:134)
Truncated. see log file for complete stacktrace
>
<Dec 16, 2010 9:13:34 PM CST> <Critical> <WebLogicServer> <BEA-000362>
<Server failed. Reason: There are 1 nested errors:
<Failed to join cluster Cluster_1 at address 239.192.0.0 due to: java.SocketException:
No such device.java.SocketException:
No such device
at java.PlainDatagramSocketImpl.join(Native Method)
at java.PlainDatagramSocketImpl.join(PlainDatagramSocketImpl.java:172)
at java.MulticastSocket.joinGroup(MulticastSocket.java:276)
at weblogic.cluster.MulticastFragmentSocket.initializeMulticastSocket(MulticastFragmentSocket.java:112)
at weblogic.cluster.MulticastFragmentSocket.start(MulticastFragmentSocket.java:134)
Truncated. see log file for complete stacktrace
>
<Dec 16, 2010 9:13:34 PM CST> <Critical> <WebLogicServer> <BEA-000362>
<Server failed. Reason: There are 1 nested errors:
java.SocketException: No such device
at java.PlainDatagramSocketImpl.join(Native Method)
at java.PlainDatagramSocketImpl.join(PlainDatagramSocketImpl.java:172)
at java.MulticastSocket.joinGroup(MulticastSocket.java:276)
at weblogic.cluster.MulticastFragmentSocket.initializeMulticastSocket(MulticastFragmentSocket.java:112)
at weblogic.cluster.MulticastFragmentSocket.start(MulticastFragmentSocket.java:134)
at weblogic.cluster.FragmentSocketWrapper.start(FragmentSocketWrapper.java:87)
at weblogic.cluster.MulticastManager$1.run(MulticastManager.java:186)
at weblogic.cluster.MulticastManager.startListening(MulticastManager.java:192)
at weblogic.cluster.InboundService.startListening(InboundService.java:40)
at weblogic.cluster.InboundService.start(InboundService.java:29)
at weblogic.t3.srvr.SubsystemRequest.run(SubsystemRequest.java:64)
at weblogic.work.SelfTuningWorkManagerImpl$WorkAdapterImpl.run(SelfTuningWorkManagerImpl.java:464)
at java.PlainDatagramSocketImpl.join(Native Method)
at java.PlainDatagramSocketImpl.join(PlainDatagramSocketImpl.java:172)
at java.MulticastSocket.joinGroup(MulticastSocket.java:276)
at weblogic.cluster.MulticastFragmentSocket.initializeMulticastSocket(MulticastFragmentSocket.java:112)
at weblogic.cluster.MulticastFragmentSocket.start(MulticastFragmentSocket.java:134)
at weblogic.cluster.FragmentSocketWrapper.start(FragmentSocketWrapper.java:87)
at weblogic.cluster.MulticastManager$1.run(MulticastManager.java:186)
at weblogic.cluster.MulticastManager.startListening(MulticastManager.java:192)
at weblogic.cluster.InboundService.startListening(InboundService.java:40)
at weblogic.cluster.InboundService.start(InboundService.java:29)
at weblogic.t3.srvr.SubsystemRequest.run(SubsystemRequest.java:64)
at weblogic.work.SelfTuningWorkManagerImpl$WorkAdapterImpl.run(SelfTuningWorkManagerImpl.java:464)
at weblogic.ute(ExecuteThread.java:200)
at weblogic.work.ExecuteThread.run(ExecuteThread.java:172)
>
<Dec 16, 2010 9:13:34 PM CST> <Notice> <WebLogicServer> <BEA-000365> <Server state changed to FAILED>
<Dec 16, 2010 9:13:34 PM CST> <Error> <WebLogicServer> <BEA-000383>
<A critical service failed. The server will shut itself down>
<Dec 16, 2010 9:13:34 PM CST> <Notice> <WebLogicServer> <BEA-000365>
<Server state changed to FORCE_SHUTTING_DOWN>
<Dec 16, 2010 9:13:37 PM> <Debug> <NodeManager> <Waiting for the process to die: 26869>
<Dec 16, 2010 9:13:38 PM> <Info> <NodeManager> <Server failed during startup so will not be restarted>
<Dec 16, 2010 9:13:38 PM> <Debug> <NodeManager>
<runMonitor returned, setting finished=true and notifying waiters>
at weblogic.work.ExecuteThread.run(ExecuteThread.java:172)
>
<Dec 16, 2010 9:13:34 PM CST> <Notice> <WebLogicServer> <BEA-000365> <Server state changed to FAILED>
<Dec 16, 2010 9:13:34 PM CST> <Error> <WebLogicServer> <BEA-000383>
<A critical service failed. The server will shut itself down>
<Dec 16, 2010 9:13:34 PM CST> <Notice> <WebLogicServer> <BEA-000365>
<Server state changed to FORCE_SHUTTING_DOWN>
<Dec 16, 2010 9:13:37 PM> <Debug> <NodeManager> <Waiting for the process to die: 26869>
<Dec 16, 2010 9:13:38 PM> <Info> <NodeManager> <Server failed during startup so will not be restarted>
<Dec 16, 2010 9:13:38 PM> <Debug> <NodeManager>
<runMonitor returned, setting finished=true and notifying waiters>
AdminServer 前台报以下信息:
<Dec 16, 2010 9:13:40 PM CST> <Error> <NodeManager> <BEA-300048> <Unable to start the server MServer_1 :
Exception while starting server 'MServer_1': java.io.IOException: Server failed to start up.
See server output log for more details.>
分析:依据以下提示,可判断组播地址问题。
Dec 16, 2010 9:13:34 PM CST> <Error> <Cluster> <BEA-000116> <Failed to join cluster Cluster_1
at address 239.192.0.0 due to: java.SocketException: No such device.
查看系统路由表:
[root@localhost ~]# route -n
Kernel IP routing table
Destination Gateway Genmask Flags Metric Ref Use Iface
192.168.1.0 0.0.0.0 255.255.255.0 U 0 0 0 eth0
169.254.0.0 0.0.0.0 255.255.0.0 U 0 0 0 eth0
<Dec 16, 2010 9:13:40 PM CST> <Error> <NodeManager> <BEA-300048> <Unable to start the server MServer_1 :
Exception while starting server 'MServer_1': java.io.IOException: Server failed to start up.
See server output log for more details.>
分析:依据以下提示,可判断组播地址问题。
Dec 16, 2010 9:13:34 PM CST> <Error> <Cluster> <BEA-000116> <Failed to join cluster Cluster_1
at address 239.192.0.0 due to: java.SocketException: No such device.
查看系统路由表:
[root@localhost ~]# route -n
Kernel IP routing table
Destination Gateway Genmask Flags Metric Ref Use Iface
192.168.1.0 0.0.0.0 255.255.255.0 U 0 0 0 eth0
169.254.0.0 0.0.0.0 255.255.0.0 U 0 0 0 eth0
缺一条组播239.192.0.0路由
手动添加路由:
[root@localhost ~]# route add -host 239.192.0.0 dev eth0
[root@localhost ~]# route -n
Kernel IP routing table
Destination Gateway Genmask Flags Metric Ref Use Iface
239.192.0.0 0.0.0.0 255.255.255.255 UH 0 0 0 eth0
192.168.1.0 0.0.0.0 255.255.255.0 U 0 0 0 eth0
169.254.0.0 0.0.0.0 255.255.0.0 U 0 0 0 eth0
再启动受控服务器,OK
手动添加路由:
[root@localhost ~]# route add -host 239.192.0.0 dev eth0
[root@localhost ~]# route -n
Kernel IP routing table
Destination Gateway Genmask Flags Metric Ref Use Iface
239.192.0.0 0.0.0.0 255.255.255.255 UH 0 0 0 eth0
192.168.1.0 0.0.0.0 255.255.255.0 U 0 0 0 eth0
169.254.0.0 0.0.0.0 255.255.0.0 U 0 0 0 eth0
再启动受控服务器,OK
同样的原因也会导致以下异常抛出:
注意:1. 本实验环境各个Server SSL设置---->Hostname Verification:: BEA HOSTNAME Verifier
2. Weblogic 11gr3(10.3) 不适合此环境
基于SSL环境下 ,成功启动节点管理器后,受控服务器无法正常启动
现象:
当启动MServer_1时,服务器状态:
MServer_1 Machine_1 FAILED_NOT_RESTARTABLE FAILED
NodeManager 报错:
Dec 16, 2010 11:23:54 PM> <INFO> <clustersys> <States = {MServer_2=UNKNOWN, AdminServer=UNKNOWN, MServer_1=UNKNOWN, domain_bak=UNKNOWN}>
<Dec 16, 2010 11:23:54 PM> <INFO> <clustersys> <States = {MServer_2=UNKNOWN, AdminServer=UNKNOWN, MServer_1=UNKNOWN, domain_bak=UNKNOWN}>
<Dec 16, 2010 11:23:57 PM> <INFO> <clustersys> <States = {MServer_2=UNKNOWN, AdminServer=UNKNOWN,
<Dec 16, 2010 11:23:57 PM> <Info> <clustersys> <MServer_1>
<Dec 16, 2010 11:23:57 PM> <Info> <clustersys> <MServer_1> <Startup configuration properties saved to "/home/weblogic/bea/user_projects/domains/clustersys/servers/MSer
现象:
当启动MServer_1时,服务器状态:
MServer_1 Machine_1 FAILED_NOT_RESTARTABLE FAILED
NodeManager 报错:
Dec 16, 2010 11:23:54 PM> <INFO> <clustersys> <States = {MServer_2=UNKNOWN, AdminServer=UNKNOWN, MServer_1=UNKNOWN, domain_bak=UNKNOWN}>
<Dec 16, 2010 11:23:54 PM> <INFO> <clustersys> <States = {MServer_2=UNKNOWN, AdminServer=UNKNOWN, MServer_1=UNKNOWN, domain_bak=UNKNOWN}>
<Dec 16, 2010 11:23:57 PM> <INFO> <clustersys> <States = {MServer_2=UNKNOWN, AdminServer=UNKNOWN,
<Dec 16, 2010 11:23:57 PM> <Info> <clustersys> <MServer_1>
<Dec 16, 2010 11:23:57 PM> <Info> <clustersys> <MServer_1> <Startup configuration properties saved to "/home/weblogic/bea/user_projects/domains/clustersys/servers/MSer
ver_1/data/nodemanager/startup.properties">
<Dec 16, 2010 11:23:57 PM> <Info> <clustersys> <MServer_1> <Rotated server output log to "/home/weblogic/bea/user_projects/domains/clustersys/servers/MServer_1/logs/MServer_1.out00005">
<Dec 16, 2010 11:23:57 PM> <Info> <clustersys> <MServer_1> <Server error log also redirected to server log>
<Dec 16, 2010 11:26:44 PM> <Info> <clustersys> <MServer_1> <Server failed during startup so will not be restarted>
<Dec 16, 2010 11:26:44 PM> <Warning> <Exception while starting server 'MServer_1': java.io.IOException: Server failed to start up. See server output log for more details.>
java.io.IOException: Server failed to start up. See server output log for more details.
demanager.server.ServerManager.start(ServerManager.java:303)
<Dec 16, 2010 11:23:57 PM> <Info> <clustersys> <MServer_1> <Rotated server output log to "/home/weblogic/bea/user_projects/domains/clustersys/servers/MServer_1/logs/MServer_1.out00005">
<Dec 16, 2010 11:23:57 PM> <Info> <clustersys> <MServer_1> <Server error log also redirected to server log>
<Dec 16, 2010 11:26:44 PM> <Info> <clustersys> <MServer_1> <Server failed during startup so will not be restarted>
<Dec 16, 2010 11:26:44 PM> <Warning> <Exception while starting server 'MServer_1': java.io.IOException: Server failed to start up. See server output log for more details.>
java.io.IOException: Server failed to start up. See server output log for more details.
demanager.server.ServerManager.start(ServerManager.java:303)
demanager.server.Handler.handleStart(Handler.java:542)
demanager.server.Handler.handleCommand(Handler.java:119)
demanager.server.Handler.run(Handler.java:66)
at java.lang.Thread.run(Thread.java:595)
<Dec 16, 2010 11:28:18 PM> <INFO> <clustersys> <States = {MServer_2=UNKNOWN, AdminServer=UNKNOWN, MServer_1=FAILED_NOT_RESTARTABLE, domain_bak=UNKNOWN}>
控制台:
<Dec 16, 2010 11:26:44 PM CST> <Error> <NodeManager> <BEA-300048> <Unable to start the server MServer_1: Exception while starting server 'MServer_1': java.io.IOException: Server failed to start up. See server output log for more details.>
查看日志,分析原因:
/home/weblogic/bea/user_projects/domains/clustersys/servers/MServer_1/logs/MServer_1.out
<Dec 16, 2010 11:26:36 PM CST> <Notice> <Security> <BEA-090082> <Security initializing using security realm myrealm.>
demanager.server.Handler.handleCommand(Handler.java:119)
demanager.server.Handler.run(Handler.java:66)
at java.lang.Thread.run(Thread.java:595)
<Dec 16, 2010 11:28:18 PM> <INFO> <clustersys> <States = {MServer_2=UNKNOWN, AdminServer=UNKNOWN, MServer_1=FAILED_NOT_RESTARTABLE, domain_bak=UNKNOWN}>
控制台:
<Dec 16, 2010 11:26:44 PM CST> <Error> <NodeManager> <BEA-300048> <Unable to start the server MServer_1: Exception while starting server 'MServer_1': java.io.IOException: Server failed to start up. See server output log for more details.>
查看日志,分析原因:
/home/weblogic/bea/user_projects/domains/clustersys/servers/MServer_1/logs/MServer_1.out
<Dec 16, 2010 11:26:36 PM CST> <Notice> <Security> <BEA-090082> <Security initializing using security realm myrealm.>
<Dec 16, 2010 11:26:42 PM CST> <Notice> <WebLogicServer> <BEA-000365> <Server state changed to STANDBY>
<Dec 16, 2010 11:26:42 PM CST> <Notice> <WebLogicServer> <BEA-000365> <Server state changed to STARTING>
<Dec 16, 2010 11:26:43 PM CST> <Error> <Cluster> <BEA-000116> <Failed to join cluster Cluster_1 at address 239.192.0.0 due to :
java.SocketException: No such device
at java.PlainDatagramSocketImpl.join(Native Method)
at java.PlainDatagramSocketImpl.join(PlainDatagramSocketImpl.java:172)
at java.MulticastSocket.joinGroup(MulticastSocket.java:276)
at weblogic.cluster.MulticastFragmentSocket.initializeMulticastSocket(MulticastFragmentSocket.java:112)
at weblogic.cluster.MulticastFragmentSocket.start(MulticastFragmentSocket.java:13
<Dec 16, 2010 11:26:42 PM CST> <Notice> <WebLogicServer> <BEA-000365> <Server state changed to STARTING>
<Dec 16, 2010 11:26:43 PM CST> <Error> <Cluster> <BEA-000116> <Failed to join cluster Cluster_1 at address 239.192.0.0 due to :
java.SocketException: No such device
at java.PlainDatagramSocketImpl.join(Native Method)
at java.PlainDatagramSocketImpl.join(PlainDatagramSocketImpl.java:172)
at java.MulticastSocket.joinGroup(MulticastSocket.java:276)
at weblogic.cluster.MulticastFragmentSocket.initializeMulticastSocket(MulticastFragmentSocket.java:112)
at weblogic.cluster.MulticastFragmentSocket.start(MulticastFragmentSocket.java:13
4)
Truncated. see log file for complete stacktrace
unknown怎么处理>
<Dec 16, 2010 11:26:43 PM CST> <Critical> <WebLogicServer> <BEA-000362> <Server failed. Reason: There are 1 nested errors:
java.SocketException: No such device
at java.PlainDatagramSocketImpl.join(Native Method)
at java.PlainDatagramSocketImpl.join(PlainDatagramSocketImpl.java:172)
at java.MulticastSocket.joinGroup(MulticastSocket.java:276)
at weblogic.cluster.MulticastFragmentSocket.initializeMulticastSocket(MulticastFragmentSocket.java:112)
at weblogic.cluster.MulticastFragmentSocket.start(MulticastFragmentSocket.java:134)
at weblogic.cluster.FragmentSocketWrapper.start(FragmentSocketWrapper.java:87)
Truncated. see log file for complete stacktrace
unknown怎么处理>
<Dec 16, 2010 11:26:43 PM CST> <Critical> <WebLogicServer> <BEA-000362> <Server failed. Reason: There are 1 nested errors:
java.SocketException: No such device
at java.PlainDatagramSocketImpl.join(Native Method)
at java.PlainDatagramSocketImpl.join(PlainDatagramSocketImpl.java:172)
at java.MulticastSocket.joinGroup(MulticastSocket.java:276)
at weblogic.cluster.MulticastFragmentSocket.initializeMulticastSocket(MulticastFragmentSocket.java:112)
at weblogic.cluster.MulticastFragmentSocket.start(MulticastFragmentSocket.java:134)
at weblogic.cluster.FragmentSocketWrapper.start(FragmentSocketWrapper.java:87)
at weblogic.cluster.MulticastManager$1.run(MulticastManager.java:186)
at weblogic.cluster.MulticastManager.startListening(MulticastManager.java:192)
at weblogic.cluster.InboundService.startListening(InboundService.java:40)
at weblogic.cluster.InboundService.start(InboundService.java:29)
at weblogic.t3.srvr.SubsystemRequest.run(SubsystemRequest.java:64)
at weblogic.work.SelfTuningWorkManagerImpl$WorkAdapterImpl.run(SelfTuningWorkManagerImpl.java:464)
at weblogic.ute(ExecuteThread.java:200)
at weblogic.work.ExecuteThread.run(ExecuteThread.java:172)
>
<Dec 16, 2010 11:26:43 PM CST> <Notice> <WebLogicServer> <BEA-000365> <Server state changed to FAILED>
<Dec 16, 2010 11:26:43 PM CST> <Error> <WebLogicServer> <BEA-000383> <A critical service failed. The server will shut itself down>
<Dec 16, 2010 11:26:43 PM CST> <Notice> <WebLogicServer> <BEA-000365> <Server
at weblogic.cluster.MulticastManager.startListening(MulticastManager.java:192)
at weblogic.cluster.InboundService.startListening(InboundService.java:40)
at weblogic.cluster.InboundService.start(InboundService.java:29)
at weblogic.t3.srvr.SubsystemRequest.run(SubsystemRequest.java:64)
at weblogic.work.SelfTuningWorkManagerImpl$WorkAdapterImpl.run(SelfTuningWorkManagerImpl.java:464)
at weblogic.ute(ExecuteThread.java:200)
at weblogic.work.ExecuteThread.run(ExecuteThread.java:172)
>
<Dec 16, 2010 11:26:43 PM CST> <Notice> <WebLogicServer> <BEA-000365> <Server state changed to FAILED>
<Dec 16, 2010 11:26:43 PM CST> <Error> <WebLogicServer> <BEA-000383> <A critical service failed. The server will shut itself down>
<Dec 16, 2010 11:26:43 PM CST> <Notice> <WebLogicServer> <BEA-000365> <Server
state changed to FORCE_SHUTTING_DOWN>
<Dec 16, 2010 11:26:44 PM> <Debug> <NodeManager> <Waiting for the process to die: 4428>
<Dec 16, 2010 11:26:44 PM> <Info> <NodeManager> <Server failed during startup so will not be restarted>
<Dec 16, 2010 11:26:44 PM> <Debug> <NodeManager> <runMonitor returned, setting finished=true and notifying waiters>
解决方法同上,手动添加组播路由
<Dec 16, 2010 11:26:44 PM> <Debug> <NodeManager> <Waiting for the process to die: 4428>
<Dec 16, 2010 11:26:44 PM> <Info> <NodeManager> <Server failed during startup so will not be restarted>
<Dec 16, 2010 11:26:44 PM> <Debug> <NodeManager> <runMonitor returned, setting finished=true and notifying waiters>
解决方法同上,手动添加组播路由
版权声明:本站内容均来自互联网,仅供演示用,请勿用于商业和其他非法用途。如果侵犯了您的权益请与我们联系QQ:729038198,我们将在24小时内删除。
发表评论