OVHcloud Network Status

Current status
Legend
  • Operational
  • Degraded performance
  • Partial Outage
  • Major Outage
  • Under maintenance
vss-2-6k
Incident Report for Network & Infrastructure
Resolved
La carte 4 plante
4 48 CEF720 48 port 10/100/1000mb Ethernet WS-X6748-GE-TX

Update(s):

Date: 2012-08-26 17:42:28 UTC
le routeur est stable.

en tout nous avons eu pas mal de petites coupures
durant 2h30 environ. le temps qu'on trouve les cartes
qui ne sont pas bonnes et qu'on les retire.

Date: 2012-08-26 16:33:09 UTC
donc on a progressivement perdu 3 cartes sur
le routeur. l'une après l'autre durant 2H.
c'est très rare et ça explique qu'il y a des
impacts durant 2H30

donc
la carte 4 morte
la carte 6 morte
la carte 8 morte

ça fait beaucoup.

on a coupé vss-2b-6k. on tourne sur vss-2-6k.
on regarde pour récupérer les cartes spares
en attendant d'en recevoir.

Date: 2012-08-26 15:35:08 UTC
Nous redémarrons le chassis.

Date: 2012-08-26 15:27:21 UTC
Nous constatons des problèmes de ping sur les réseaux connectés à la carte remplacée ainsi que du loss sur les autres réseaux. Nous nous préparons à redémarrer le chassis entièrement.

Date: 2012-08-26 15:11:10 UTC
Nous constatons toujours des anomalies sur les réseaux routés par vss-2-6k. Nous recherchons l'origine du problème.

Date: 2012-08-26 14:11:31 UTC
La carte est remplacée.

Date: 2012-08-26 13:51:40 UTC
Nous allons récupérer la carte 8 du vss-2b (routeur de backup) pour l'insérer à la place de la carte 4 du vss-2 qui est défectueuse.

Date: 2012-08-26 13:14:32 UTC
Aug 26 15:13:35 vss-2-6k.fr.eu 151949: Aug 26 14:13:09 GMT: %PM_SCP-SP-1-LCP_FW_ERR: System resetting module 4 to recover from error: Linecard received system exception. Errcode =
Aug 26 15:13:35 vss-2-6k.fr.eu 151950: Aug 26 14:13:09 GMT: %OIR-SP-3-PWRCYCLE: Card in module 4, is being power-cycled 'Off (Module Reset due to exception or user request)'
Aug 26 15:13:35 vss-2-6k.fr.eu 151951: .Aug 26 14:13:09 GMT: %XDR-6-XDRIPCNOTIFY: Message not sent to slot 4/0 (4) because of IPC error queue flush. Disabling linecard. (Expected during linecard OIR or system reloads)
Aug 26 15:13:37 vss-2-6k.fr.eu 151952: Aug 26 14:13:09 GMT: %C6KPWR-SP-4-DISABLED: power to module in slot 4 set Off (Module Reset due to exception or user request)
Aug 26 15:13:47 vss-2-6k.fr.eu 151953: Aug 26 14:13:20 GMT: %C6KPWR-SP-4-DISABLED: power to module in slot 4 set off (admin request)

elle est morte. on la coupe electriquement puis on cherche
un spare.


Date: 2012-08-26 13:11:28 UTC
Aug 26 14:54:25 vss-2-6k.fr.eu 151891: Aug 26 13:54:00 GMT: %C6KPWR-SP-4-DISABLED: power to module in slot 4 set Off (Module Reset due to exception or user request)

Aug 26 14:56:00 vss-2-6k.fr.eu 151893: Aug 26 12:55:35.946: %SYS-DFC4-5-RESTART: System restarted --
Aug 26 14:56:08 vss-2-6k.fr.eu 151894: Aug 26 13:55:41 GMT: %DIAG-SP-6-RUN_MINIMUM: Module 4: Running Minimal Diagnostics...
Aug 26 14:56:47 vss-2-6k.fr.eu 151895: Aug 26 13:56:22 GMT: %PM_SCP-SP-1-LCP_FW_ERR: System resetting module 4 to recover from error: Linecard received system exception. Errcode =
Aug 26 14:56:47 vss-2-6k.fr.eu 151896: Aug 26 13:56:22 GMT: %OIR-SP-3-PWRCYCLE: Card in module 4, is being power-cycled 'Off (Module Reset due to exception or user request)'
Aug 26 14:56:49 vss-2-6k.fr.eu 151897: Aug 26 13:56:22 GMT: %C6KPWR-SP-4-DISABLED: power to module in slot 4 set Off (Module Reset due to exception or user request)


Aug 26 14:58:27 vss-2-6k.fr.eu 151898: Aug 26 12:58:02.843: %SYS-DFC4-5-RESTART: System restarted --
Aug 26 14:58:33 vss-2-6k.fr.eu 151899: Aug 26 13:58:08 GMT: %DIAG-SP-6-RUN_MINIMUM: Module 4: Running Minimal Diagnostics...
Aug 26 14:59:17 vss-2-6k.fr.eu 151900: Aug 26 13:58:51 GMT: %PM_SCP-SP-1-LCP_FW_ERR: System resetting module 4 to recover from error: Linecard received system exception. Errcode =
Aug 26 14:59:17 vss-2-6k.fr.eu 151901: Aug 26 13:58:51 GMT: %OIR-SP-3-PWRCYCLE: Card in module 4, is being power-cycled 'Off (Module Reset due to exception or user request)'
Aug 26 14:59:18 vss-2-6k.fr.eu 151902: Aug 26 13:58:51 GMT: %C6KPWR-SP-4-DISABLED: power to module in slot 4 set Off (Module Reset due to exception or user request)
Aug 26 15:01:04 vss-2-6k.fr.eu 151903: Aug 26 14:00:37 GMT: %DIAG-SP-6-RUN_MINIMUM: Module 4: Running Minimal Diagnostics...
Aug 26 15:01:04 vss-2-6k.fr.eu 151904: Aug 26 13:00:30.900: %SYS-DFC4-5-RESTART: System restarted --
Aug 26 15:01:49 vss-2-6k.fr.eu 151905: Aug 26 14:01:24 GMT: %PM_SCP-SP-1-LCP_FW_ERR: System resetting module 4 to recover from error: Linecard received system exception. Errcode =
Aug 26 15:01:49 vss-2-6k.fr.eu 151906: Aug 26 14:01:24 GMT: %OIR-SP-3-PWRCYCLE: Card in module 4, is being power-cycled 'Off (Module Reset due to exception or user request)'
Aug 26 15:01:50 vss-2-6k.fr.eu 151907: Aug 26 14:01:24 GMT: %C6KPWR-SP-4-DISABLED: power to module in slot 4 set Off (Module Reset due to exception or user request)


Aug 26 15:03:28 vss-2-6k.fr.eu 151908: Aug 26 14:03:03 GMT: %OIR-SP-6-REMCARD: Card removed from slot 4, interfaces disabled
Aug 26 15:05:02 vss-2-6k.fr.eu 151916: Aug 26 13:04:38.914: %SYS-DFC4-5-RESTART: System restarted --
Aug 26 15:05:12 vss-2-6k.fr.eu 151917: Aug 26 14:04:44 GMT: %DIAG-SP-6-RUN_MINIMUM: Module 4: Running Minimal Diagnostics...
Aug 26 15:05:54 vss-2-6k.fr.eu 151920: Aug 26 14:05:28 GMT: %PM_SCP-SP-1-LCP_FW_ERR: System resetting module 4 to recover from error: Linecard received system exception. Errcode =
Aug 26 15:05:54 vss-2-6k.fr.eu 151921: Aug 26 14:05:28 GMT: %OIR-SP-3-PWRCYCLE: Card in module 4, is being power-cycled 'Off (Module Reset due to exception or user request)'
Aug 26 15:05:54 vss-2-6k.fr.eu 151922: Aug 26 14:05:28 GMT: %C6KPWR-SP-4-DISABLED: power to module in slot 4 set Off (Module Reset due to exception or user request)



Aug 26 15:07:31 vss-2-6k.fr.eu 151926: Aug 26 13:07:07.060: %SYS-DFC4-5-RESTART: System restarted --

Aug 26 15:07:40 vss-2-6k.fr.eu 151928: Aug 26 14:07:12 GMT: %DIAG-SP-6-RUN_MINIMUM: Module 4: Running Minimal Diagnostics...
Aug 26 15:08:16 vss-2-6k.fr.eu 151929: Aug 26 14:07:50 GMT: %PM_SCP-SP-1-LCP_FW_ERR: System resetting module 4 to recover from error: Linecard received system exception. Errcode =
Aug 26 15:08:16 vss-2-6k.fr.eu 151930: Aug 26 14:07:50 GMT: %OIR-SP-3-PWRCYCLE: Card in module 4, is being power-cycled 'Off (Module Reset due to exception or user request)'
Aug 26 15:08:18 vss-2-6k.fr.eu 151931: Aug 26 14:07:50 GMT: %C6KPWR-SP-4-DISABLED: power to module in slot 4 set Off (Module Reset due to exception or user request)
Posted Aug 26, 2012 - 13:08 UTC