OVHcloud Network Status
Current status
Legend
- Operational
- Degraded performance
- Partial Outage
- Major Outage
- Under maintenance
vrack 1.5
Incident
Report for Network & Infrastructure
Resolved
Nous avons un incident sur l'infra.
Update(s):
Date: 2014-05-20 16:43:02 UTC
sur l'un de switchs, nous avons fait les opérations de maintenance
habituel (ajout de nouveaux host). apparament le nouveau FEX a été
mal tiré et visiblement ca a bouclé. ceci n'explique pas pourquoi
le switch principal complet est tombé.
2014 May 20 17:24:54 rbx2b-101-n6 %LLDP-FEX105-5-SERVER_ADDED: Server with Chassis ID dca5.f430.9dcc Port ID Eth105/1/11 management address 10.1.0.2 discovered on local port Eth105/1/21 in vlan 101 with enabled capability Bridge Router
2014 May 20 17:24:54 rbx2b-101-n6 %LLDP-FEX105-5-SERVER_ADDED: Server with Chassis ID dca5.f430.9dd6 Port ID Eth105/1/21 management address 10.1.0.2 discovered on local port Eth105/1/11 in vlan 101 with enabled capability Bridge Router
2014 May 20 17:24:55 rbx2b-101-n6 %FWM-2-STM_LOOP_DETECT: Loops detected in the network for mac 000a.f394.2000 among ports Eth105/1/21 and Eth105/1/11 vlan 101 - Disabling dynamic learn notifications for 180 seconds
2014 May 20 17:27:55 rbx2b-101-n6 %FWM-2-STM_LEARNING_RE_ENABLE: Re enabling dynamic learning on all interfaces
2014 May 20 17:27:47 rbx2b-101-n6 %SYSMGR-FEX104-5-MODULE_ONLINE: System Manager has received notification of local module becoming online.
2014 May 20 17:27:47 rbx2b-101-n6 %SATCTRL-FEX104-2-SATCTRL: FEX-104 Module 1: Cold boot
2014 May 20 17:27:56 rbx2b-101-n6 %FWM-2-STM_LOOP_DETECT: Loops detected in the network for mac 90e2.ba61.13f8 among ports Eth105/1/11 and Eth105/1/21 vlan 101 - Disabling dynamic learn notifications for 180 seconds
2014 May 20 17:28:07 rbx2b-101-n6 %LLDP-FEX104-5-SERVER_ADDED: Server with Chassis ID c471.fe0e.c404 Port ID Eth104/1/3 management address 10.1.0.2 discovered on local port Eth104/1/11 in vlan 1 with enabled capability Bridge Router
2014 May 20 17:28:07 rbx2b-101-n6 %LLDP-FEX104-5-SERVER_ADDED: Server with Chassis ID c471.fe0e.c40c Port ID Eth104/1/11 management address 10.1.0.2 discovered on local port Eth104/1/3 in vlan 1 with enabled capability Bridge Router
Date: 2014-05-20 16:02:46 UTC
le lien vPC entre 2 switchs principaux Nexus 7000 s'est
mis down sans raison.
2014 May 20 17:33:24 pcc-2b-n7 %ETH_PORT_CHANNEL-5-PORT_DOWN: port-channel4096: Ethernet1/32 is down
2014 May 20 17:33:25 pcc-2b-n7 %ETH_PORT_CHANNEL-5-FOP_CHANGED: port-channel4096: first operational port changed from Ethernet1/32 to Ethernet2/32
2014 May 20 17:33:27 pcc-2b-n7 %ETHPORT-5-IF_DOWN_INITIALIZING: Interface Ethern2014 May 20 17:35:07 pcc-2b-n7 %ETH_PORT_CHANNEL-5-PORT_DOWN: port-channel4096: Ethernet2/32 is down
et1/32 is down (Initializing)
2014 May 20 17:33:42 pcc-2b-n7 %ETH_PORT_CHANNEL-5-PORT_SUSPENDED: Ethernet1/32: Ethernet1/32 is suspended
2014 May 20 17:34:58 pcc-2b-n7 %VPC-2-VPC_SUSP_ALL_VPC: Peer-link going down, suspending all vPCs on secondary
2014 May 20 17:35:07 pcc-2b-n7 %ETH_PORT_CHANNEL-5-FOP_CHANGED: port-channel4096: first operational port changed from Ethernet2/32 to none
2014 May 20 17:35:07 pcc-2b-n7 %ETHPORT-5-IF_DOWN_INITIALIZING: Interface Ethernet2/32 is down (Initializing)
2014 May 20 17:35:07 pcc-2b-n7 %ETHPORT-5-IF_DOWN_PORT_CHANNEL_MEMBERS_DOWN: Interface port-channel4096 is down (No operational members)
2014 May 20 17:35:18 pcc-2b-n7 %ETH_PORT_CHANNEL-5-PORT_SUSPENDED: Ethernet1/32: Ethernet1/32 is suspended
2014 May 20 17:35:21 pcc-2b-n7 %ETH_PORT_CHANNEL-5-PORT_SUSPENDED: Ethernet2/32: Ethernet2/32 is suspended
2014 May 20 17:35:47 pcc-2b-n7 %ETH_PORT_CHANNEL-5-PORT_SUSPENDED: Ethernet1/32: Ethernet1/32 is suspended
2014 May 20 17:35:55 pcc-2b-n7 %ETH_PORT_CHANNEL-5-PORT_SUSPENDED: Ethernet2/32: Ethernet2/32 is suspended
2014 May 20 17:36:17 pcc-2b-n7 %ETH_PORT_CHANNEL-5-PORT_SUSPENDED: Ethernet1/32: Ethernet1/32 is suspended
2014 May 20 17:36:25 pcc-2b-n7 %ETH_PORT_CHANNEL-5-PORT_SUSPENDED: Ethernet2/32: Ethernet2/32 is suspended
2014 May 20 17:36:46 pcc-2b-n7 %ETH_PORT_CHANNEL-5-PORT_SUSPENDED: Ethernet1/32: Ethernet1/32 is suspended
2014 May 20 17:36:55 pcc-2b-n7 %ETH_PORT_CHANNEL-5-PORT_SUSPENDED: Ethernet2/32: Ethernet2/32 is suspended
2014 May 20 17:37:17 pcc-2b-n7 %ETH_PORT_CHANNEL-5-PORT_SUSPENDED: Ethernet1/32: Ethernet1/32 is suspended
2014 May 20 17:37:25 pcc-2b-n7 %ETH_PORT_CHANNEL-5-PORT_SUSPENDED: Ethernet2/32: Ethernet2/32 is suspended
2014 May 20 17:37:47 pcc-2b-n7 %ETH_PORT_CHANNEL-5-PORT_SUSPENDED: Ethernet1/32: Ethernet1/32 is suspended
2014 May 20 17:37:55 pcc-2b-n7 %ETH_PORT_CHANNEL-5-PORT_SUSPENDED: Ethernet2/32: Ethernet2/32 is suspended
2014 May 20 17:38:17 pcc-2b-n7 %ETH_PORT_CHANNEL-5-PORT_SUSPENDED: Ethernet1/32: Ethernet1/32 is suspended
2014 May 20 17:38:25 pcc-2b-n7 %ETH_PORT_CHANNEL-5-PORT_SUSPENDED: Ethernet2/32: Ethernet2/32 is suspended
2014 May 20 17:38:47 pcc-2b-n7 %ETH_PORT_CHANNEL-5-PORT_SUSPENDED: Ethernet1/32: Ethernet1/32 is suspended
2014 May 20 17:38:55 pcc-2b-n7 %ETH_PORT_CHANNEL-5-PORT_SUSPENDED: Ethernet2/32: Ethernet2/32 is suspended
2014 May 20 17:39:17 pcc-2b-n7 %ETH_PORT_CHANNEL-5-PORT_SUSPENDED: Ethernet1/32: Ethernet1/32 is suspended
2014 May 20 17:39:25 pcc-2b-n7 %ETH_PORT_CHANNEL-5-PORT_SUSPENDED: Ethernet2/32: Ethernet2/32 is suspended
2014 May 20 17:39:47 pcc-2b-n7 %ETH_PORT_CHANNEL-5-PORT_SUSPENDED: Ethernet1/32: Ethernet1/32 is suspended
2014 May 20 17:39:51 pcc-2b-n7 %ETH_PORT_CHANNEL-5-PORT_UP: port-channel4096: Ethernet1/32 is up
2014 May 20 17:39:51 pcc-2b-n7 %ETH_PORT_CHANNEL-5-PORT_UP: port-channel4096: Ethernet2/32 is up
2014 May 20 17:39:51 pcc-2b-n7 %ETH_PORT_CHANNEL-5-FOP_CHANGED: port-channel4096: first operational port changed from none to Ethernet1/32
2014 May 20 17:39:52 pcc-2b-n7 %ETHPORT-5-IF_UP: Interface port-channel4096 is up in mode trunk
2014 May 20 17:39:52 pcc-2b-n7 %ETHPORT-5-IF_UP: Interface Ethernet1/32 is up in mode trunk
Date: 2014-05-20 11:00:43 UTC
Nous avons isolé un uplink défectueux entre l'infra vrack 1.5 et l'infra pcc de Roubaix. Nous ne constatons plus d'impact sur le trafic pour le moment.
Date: 2014-05-20 10:52:31 UTC
Nous suspectons un problème hardware sur une des interfaces du N7 de pcc.
Update(s):
Date: 2014-05-20 16:43:02 UTC
sur l'un de switchs, nous avons fait les opérations de maintenance
habituel (ajout de nouveaux host). apparament le nouveau FEX a été
mal tiré et visiblement ca a bouclé. ceci n'explique pas pourquoi
le switch principal complet est tombé.
2014 May 20 17:24:54 rbx2b-101-n6 %LLDP-FEX105-5-SERVER_ADDED: Server with Chassis ID dca5.f430.9dcc Port ID Eth105/1/11 management address 10.1.0.2 discovered on local port Eth105/1/21 in vlan 101 with enabled capability Bridge Router
2014 May 20 17:24:54 rbx2b-101-n6 %LLDP-FEX105-5-SERVER_ADDED: Server with Chassis ID dca5.f430.9dd6 Port ID Eth105/1/21 management address 10.1.0.2 discovered on local port Eth105/1/11 in vlan 101 with enabled capability Bridge Router
2014 May 20 17:24:55 rbx2b-101-n6 %FWM-2-STM_LOOP_DETECT: Loops detected in the network for mac 000a.f394.2000 among ports Eth105/1/21 and Eth105/1/11 vlan 101 - Disabling dynamic learn notifications for 180 seconds
2014 May 20 17:27:55 rbx2b-101-n6 %FWM-2-STM_LEARNING_RE_ENABLE: Re enabling dynamic learning on all interfaces
2014 May 20 17:27:47 rbx2b-101-n6 %SYSMGR-FEX104-5-MODULE_ONLINE: System Manager has received notification of local module becoming online.
2014 May 20 17:27:47 rbx2b-101-n6 %SATCTRL-FEX104-2-SATCTRL: FEX-104 Module 1: Cold boot
2014 May 20 17:27:56 rbx2b-101-n6 %FWM-2-STM_LOOP_DETECT: Loops detected in the network for mac 90e2.ba61.13f8 among ports Eth105/1/11 and Eth105/1/21 vlan 101 - Disabling dynamic learn notifications for 180 seconds
2014 May 20 17:28:07 rbx2b-101-n6 %LLDP-FEX104-5-SERVER_ADDED: Server with Chassis ID c471.fe0e.c404 Port ID Eth104/1/3 management address 10.1.0.2 discovered on local port Eth104/1/11 in vlan 1 with enabled capability Bridge Router
2014 May 20 17:28:07 rbx2b-101-n6 %LLDP-FEX104-5-SERVER_ADDED: Server with Chassis ID c471.fe0e.c40c Port ID Eth104/1/11 management address 10.1.0.2 discovered on local port Eth104/1/3 in vlan 1 with enabled capability Bridge Router
Date: 2014-05-20 16:02:46 UTC
le lien vPC entre 2 switchs principaux Nexus 7000 s'est
mis down sans raison.
2014 May 20 17:33:24 pcc-2b-n7 %ETH_PORT_CHANNEL-5-PORT_DOWN: port-channel4096: Ethernet1/32 is down
2014 May 20 17:33:25 pcc-2b-n7 %ETH_PORT_CHANNEL-5-FOP_CHANGED: port-channel4096: first operational port changed from Ethernet1/32 to Ethernet2/32
2014 May 20 17:33:27 pcc-2b-n7 %ETHPORT-5-IF_DOWN_INITIALIZING: Interface Ethern2014 May 20 17:35:07 pcc-2b-n7 %ETH_PORT_CHANNEL-5-PORT_DOWN: port-channel4096: Ethernet2/32 is down
et1/32 is down (Initializing)
2014 May 20 17:33:42 pcc-2b-n7 %ETH_PORT_CHANNEL-5-PORT_SUSPENDED: Ethernet1/32: Ethernet1/32 is suspended
2014 May 20 17:34:58 pcc-2b-n7 %VPC-2-VPC_SUSP_ALL_VPC: Peer-link going down, suspending all vPCs on secondary
2014 May 20 17:35:07 pcc-2b-n7 %ETH_PORT_CHANNEL-5-FOP_CHANGED: port-channel4096: first operational port changed from Ethernet2/32 to none
2014 May 20 17:35:07 pcc-2b-n7 %ETHPORT-5-IF_DOWN_INITIALIZING: Interface Ethernet2/32 is down (Initializing)
2014 May 20 17:35:07 pcc-2b-n7 %ETHPORT-5-IF_DOWN_PORT_CHANNEL_MEMBERS_DOWN: Interface port-channel4096 is down (No operational members)
2014 May 20 17:35:18 pcc-2b-n7 %ETH_PORT_CHANNEL-5-PORT_SUSPENDED: Ethernet1/32: Ethernet1/32 is suspended
2014 May 20 17:35:21 pcc-2b-n7 %ETH_PORT_CHANNEL-5-PORT_SUSPENDED: Ethernet2/32: Ethernet2/32 is suspended
2014 May 20 17:35:47 pcc-2b-n7 %ETH_PORT_CHANNEL-5-PORT_SUSPENDED: Ethernet1/32: Ethernet1/32 is suspended
2014 May 20 17:35:55 pcc-2b-n7 %ETH_PORT_CHANNEL-5-PORT_SUSPENDED: Ethernet2/32: Ethernet2/32 is suspended
2014 May 20 17:36:17 pcc-2b-n7 %ETH_PORT_CHANNEL-5-PORT_SUSPENDED: Ethernet1/32: Ethernet1/32 is suspended
2014 May 20 17:36:25 pcc-2b-n7 %ETH_PORT_CHANNEL-5-PORT_SUSPENDED: Ethernet2/32: Ethernet2/32 is suspended
2014 May 20 17:36:46 pcc-2b-n7 %ETH_PORT_CHANNEL-5-PORT_SUSPENDED: Ethernet1/32: Ethernet1/32 is suspended
2014 May 20 17:36:55 pcc-2b-n7 %ETH_PORT_CHANNEL-5-PORT_SUSPENDED: Ethernet2/32: Ethernet2/32 is suspended
2014 May 20 17:37:17 pcc-2b-n7 %ETH_PORT_CHANNEL-5-PORT_SUSPENDED: Ethernet1/32: Ethernet1/32 is suspended
2014 May 20 17:37:25 pcc-2b-n7 %ETH_PORT_CHANNEL-5-PORT_SUSPENDED: Ethernet2/32: Ethernet2/32 is suspended
2014 May 20 17:37:47 pcc-2b-n7 %ETH_PORT_CHANNEL-5-PORT_SUSPENDED: Ethernet1/32: Ethernet1/32 is suspended
2014 May 20 17:37:55 pcc-2b-n7 %ETH_PORT_CHANNEL-5-PORT_SUSPENDED: Ethernet2/32: Ethernet2/32 is suspended
2014 May 20 17:38:17 pcc-2b-n7 %ETH_PORT_CHANNEL-5-PORT_SUSPENDED: Ethernet1/32: Ethernet1/32 is suspended
2014 May 20 17:38:25 pcc-2b-n7 %ETH_PORT_CHANNEL-5-PORT_SUSPENDED: Ethernet2/32: Ethernet2/32 is suspended
2014 May 20 17:38:47 pcc-2b-n7 %ETH_PORT_CHANNEL-5-PORT_SUSPENDED: Ethernet1/32: Ethernet1/32 is suspended
2014 May 20 17:38:55 pcc-2b-n7 %ETH_PORT_CHANNEL-5-PORT_SUSPENDED: Ethernet2/32: Ethernet2/32 is suspended
2014 May 20 17:39:17 pcc-2b-n7 %ETH_PORT_CHANNEL-5-PORT_SUSPENDED: Ethernet1/32: Ethernet1/32 is suspended
2014 May 20 17:39:25 pcc-2b-n7 %ETH_PORT_CHANNEL-5-PORT_SUSPENDED: Ethernet2/32: Ethernet2/32 is suspended
2014 May 20 17:39:47 pcc-2b-n7 %ETH_PORT_CHANNEL-5-PORT_SUSPENDED: Ethernet1/32: Ethernet1/32 is suspended
2014 May 20 17:39:51 pcc-2b-n7 %ETH_PORT_CHANNEL-5-PORT_UP: port-channel4096: Ethernet1/32 is up
2014 May 20 17:39:51 pcc-2b-n7 %ETH_PORT_CHANNEL-5-PORT_UP: port-channel4096: Ethernet2/32 is up
2014 May 20 17:39:51 pcc-2b-n7 %ETH_PORT_CHANNEL-5-FOP_CHANGED: port-channel4096: first operational port changed from none to Ethernet1/32
2014 May 20 17:39:52 pcc-2b-n7 %ETHPORT-5-IF_UP: Interface port-channel4096 is up in mode trunk
2014 May 20 17:39:52 pcc-2b-n7 %ETHPORT-5-IF_UP: Interface Ethernet1/32 is up in mode trunk
Date: 2014-05-20 11:00:43 UTC
Nous avons isolé un uplink défectueux entre l'infra vrack 1.5 et l'infra pcc de Roubaix. Nous ne constatons plus d'impact sur le trafic pour le moment.
Date: 2014-05-20 10:52:31 UTC
Nous suspectons un problème hardware sur une des interfaces du N7 de pcc.