OVHcloud Network Status

Current status
Legend
  • Operational
  • Degraded performance
  • Partial Outage
  • Major Outage
  • Under maintenance
rbx-g2-a9
Incident Report for Network & Infrastructure
Resolved
L'une des linecards de rbx-g2 a rebooté.

Update(s):

Date: 2011-07-26 00:46:52 UTC
La carte a été remplacée.

Date: 2011-07-26 00:34:19 UTC
Nous remplacons la carte.

Date: 2011-07-25 22:54:26 UTC
La carte va être remplacée. Le matériel devrait être livré d'ici 03:00. Nous interviendrons dans la foulée.

Date: 2011-07-25 19:04:55 UTC
On declanche le TAC

Date: 2011-07-25 19:04:46 UTC
RP/0/RSP0/CPU0:Jul 25 16:55:33 UTC: shelfmgr[314]: %PLATFORM-SHELFMGR-6-NODE_STATE_CHANGE : 0/2/CPU0 A9K-8T-L state:MBI-RUNNING
RP/0/RSP0/CPU0:Jul 25 16:56:49 UTC: invmgr[214]: %PLATFORM-INV-6-NODE_STATE_CHANGE : Node: 0/2/CPU0, state: IOS XR RUN

2ème plantage.
LC/0/2/CPU0:Jul 25 18:51:47 UTC: pfm_node_lc[227]: %PLATFORM-NP-0-HW_DOUBLE_ECC_ERROR : Set|prm_server[139342]|Network Processor Unit(0x1007002)|NP DOUBLE ECC ERROR, NP=2, memId=17, subMemId=0x1
LC/0/2/CPU0:Jul 25 18:51:47 UTC: pfm_node_lc[227]: %PLATFORM-PFM-0-CARD_RESET_REQ : pfm_dev_sm_perform_recovery_action, Card reset requested by: Process ID: 139342 (prm_server), Fault Sev: 0, Target node: 0/2/CPU0, CompId: 0x1f, Device Handle: 0x1007002, CondID: 1001, Fault Reason: NP DOUBLE ECC ERROR, NP=2, memId=17, subMemId=0x1
LC/0/2/CPU0:Jul 25 18:51:47 UTC: sysmgr[87]: %OS-SYSMGR-2-REBOOT : reboot required, process (pfm_node_lc) reason (pfm_dev_sm_perform_recovery_action, Card reset requested by: Process ID: 139342 (prm_server), Fault Sev: 0, Target node: 0/2/CPU0, CompId: 0x1f, Device Handle: 0x1007002, CondID: 1001, Fault Reason: NP DOUBLE ECC ERROR, NP=2, memId=17, subMemId=0x1)
LC/0/2/CPU0:Jul 25 18:51:47 UTC: sysmgr[87]: %OS-LIBSYSMGR-3-PARSE : parse_args: parse error: unmatched \"
LC/0/2/CPU0:Jul 25 18:51:47 UTC: sysmgr[87]: %OS-SYSMGR-3-ERROR : sysmgr_shutdown_cleanup_handler: shutdown script execution timed-out! Node will reset
LC/0/2/CPU0:1h:56:30: sysmgr[87]: %OS-SYSMGR-7-DEBUG : sysmgr_shutdown_cleanup_handler: shutdown script execution timed-out! Node will reset
LC/0/2/CPU0:1h:56:30: syslog_dev[85]: pfm_node_lc[227]: Request Graceful Reboot via Sysmgr: Reason: pfm_dev_sm_perform_recovery_action, Card reset requested by: Process ID: 139342 (prm_server), Fault Sev: 0, Target node: 0/2/CPU0, CompId: 0x1f, Device Handle: 0x1007002, CondID: 1001, Fault Reason: NP DOUBLE ECC ERROR, NP=2, memId=17, subMemId=0x1
LC/0/2/CPU0:Jul 25 18:51:47 UTC: sysmgr[87]: %OS-SYSMGR-3-ERROR : sysmgr_shutdown_cleanup_handler: shutdown triggered by (pfm_node_lc) did not complete in 45 seconds, shutting down
RP/0/RSP0/CPU0:Jul 25 18:52:09 UTC: shelfmgr[314]: %PLATFORM-SHELFMGR-3-NODE_CPU_RESET : Node 0/2/CPU0 CPU reset detected.
RP/0/RSP0/CPU0:Jul 25 18:52:09 UTC: shelfmgr[314]: %PLATFORM-SHELFMGR-6-NODE_STATE_CHANGE : 0/2/CPU0 A9K-8T-L state:BRINGDOWN
RP/0/RSP0/CPU0:Jul 25 18:52:09 UTC: invmgr[214]: %PLATFORM-INV-6-NODE_STATE_CHANGE : Node: 0/2/CPU0, state: BRINGDOWN
RP/0/RSP1/CPU0:Jul 25 18:52:10 UTC: pfm_node_rp[282]: %PLATFORM-DIAGS-3-SRSP_ACTIVE_EOBC_FAILED : Set|online_diag_rsp[229493]|SRSP active EOBC Test(0x2000002)|failure threshold is 3, slot(s) failed: 2
RP/0/RSP1/CPU0:Jul 25 18:52:15 UTC: pfm_node_rp[282]: %PLATFORM-DIAGS-3-SRSP_ACTIVE_EOBC_FAILED : Clear|online_diag_rsp[229493]|SRSP active EOBC Test(0x2000002)|failure threshold is 3, slot(s) failed: 2
RP/0/RSP0/CPU0:Jul 25 18:52:15 UTC: shelfmgr[314]: %PLATFORM-SHELFMGR-6-NODE_STATE_CHANGE : 0/2/CPU0 A9K-8T-L state:ROMMON
RP/0/RSP0/CPU0:Jul 25 18:52:37 UTC: shelfmgr[314]: %PLATFORM-SHELFMGR_HAL-6-BOOT_REQ_RECEIVED : Boot Request from 0/2/CPU0, RomMon Version: 1.3
RP/0/RSP0/CPU0:2w4d:23h:53:1: shelfmgr[314]: %PLATFORM-MBIMGR-7-IMAGE_VALIDATED : Remote location 0/2/CPU0: : MBI tftp:/disk0/asr9k-os-mbi-4.0.1/lc/mbiasr9k-lc.vm validated
RP/0/RSP0/CPU0:Jul 25 18:52:37 UTC: shelfmgr[314]: %PLATFORM-SHELFMGR-6-NODE_STATE_CHANGE : 0/2/CPU0 A9K-8T-L state:MBI-BOOTING
RP/0/RSP0/CPU0:Jul 25 18:53:34 UTC: shelfmgr[314]: %PLATFORM-SHELFMGR-6-NODE_STATE_CHANGE : 0/2/CPU0 A9K-8T-L state:MBI-RUNNING
LC/0/2/CPU0:15: init[65540]: %OS-INIT-7-MBI_STARTED : total time 8.824 seconds
RP/0/RSP0/CPU0:Jul 25 18:54:50 UTC: shelfmgr[314]: %PLATFORM-SHELFMGR-6-NODE_STATE_CHANGE : 0/2/CPU0 A9K-8T-L state:IOS XR RUN
RP/0/RSP0/CPU0:Jul 25 18:54:50 UTC: invmgr[214]: %PLATFORM-INV-6-NODE_STATE_CHANGE : Node: 0/2/CPU0, state: IOS XR RUN


Date: 2011-07-25 17:00:18 UTC
LC/0/2/CPU0:Jul 25 16:53:45 UTC: pfm_node_lc[227]: %PLATFORM-NP-0-HW_DOUBLE_ECC_ERROR : Set|prm_server[139342]|Network Processor Unit(0x1007002)|NP DOUBLE ECC ERROR, NP=2, memId=17, subMemId=0x1
LC/0/2/CPU0:Jul 25 16:53:45 UTC: pfm_node_lc[227]: %PLATFORM-PFM-0-CARD_RESET_REQ : pfm_dev_sm_perform_recovery_action, Card reset requested by: Process ID: 139342 (prm_server), Fault Sev: 0, Target node: 0/2/CPU0, CompId: 0x1f, Device Handle: 0x1007002, CondID: 1001, Fault Reason: NP DOUBLE ECC ERROR, NP=2, memId=17, subMemId=0x1
LC/0/2/CPU0:Jul 25 16:53:45 UTC: sysmgr[87]: %OS-SYSMGR-2-REBOOT : reboot required, process (pfm_node_lc) reason (pfm_dev_sm_perform_recovery_action, Card reset requested by: Process ID: 139342 (prm_server), Fault Sev: 0, Target node: 0/2/CPU0, CompId: 0x1f, Device Handle: 0x1007002, CondID: 1001, Fault Reason: NP DOUBLE ECC ERROR, NP=2, memId=17, subMemId=0x1)
Posted Jul 25, 2011 - 16:59 UTC