OVHcloud Network Status

Current status
Legend
  • Operational
  • Degraded performance
  • Partial Outage
  • Major Outage
  • Under maintenance
bhs2-11a-n6
Incident Report for Network & Infrastructure
Resolved
Ce nexus a rebooté:

Reason: Reset triggered due to HA policy of Reset


Les paquets sont forwardés par bhs2-11b-n6.

Update(s):

Date: 2014-12-17 23:34:53 UTC
Le couple est de nouveau stable depuis 30 minutes

Date: 2014-12-17 23:10:10 UTC
Le seul workaround possible a été de reload les 4 fexs en question.

Nous ne constatons plus de port en err-disable.

Date: 2014-12-17 22:43:36 UTC
L'ensemble des fex sont a jour. par contre nous avons toujours des port en err-disable. Nous sommes avec cisco sur le cas.

Date: 2014-12-17 21:31:07 UTC
Le fex n'est toujours pas reconnus. On reboot bhs2-11b-n6.

Date: 2014-12-17 21:26:51 UTC
installe failed.

On remplace le fex 101.

Date: 2014-12-17 20:55:01 UTC
On relance une upgrade :

Images will be upgraded according to following table:
Module Image Running-Version New-Version Upg-Required
------ ---------------- ---------------------- ---------------------- ------------
1 system 6.0(2)N2(5) 6.0(2)N2(5) no
1 kickstart 6.0(2)N2(5) 6.0(2)N2(5) no
1 bios v1.5.0(12/29/2012) v1.5.0(12/29/2012) no
1 power-seq v4.0 v4.0 no
1 fabric-power-seq v4.0 v4.0 no
2 power-seq v4.0 v4.0 no
100 fexth 6.0(2)N2(5) 6.0(2)N2(5) no
101 fexth 6.0(2)N2(3) 6.0(2)N2(5) yes
102 fexth 6.0(2)N2(3) 6.0(2)N2(5) yes
103 fexth 6.0(2)N2(3) 6.0(2)N2(5) yes
104 fexth 6.0(2)N2(3) 6.0(2)N2(5) yes
105 fexth 6.0(2)N2(3) 6.0(2)N2(5) yes
106 fexth 6.0(2)N2(3) 6.0(2)N2(5) yes
107 fexth 6.0(2)N2(3) 6.0(2)N2(5) yes
108 fexth 6.0(2)N2(3) 6.0(2)N2(5) yes
109 fexth 6.0(2)N2(3) 6.0(2)N2(5) yes
110 fexth 6.0(2)N2(3) 6.0(2)N2(5) yes
111 fexth 6.0(2)N2(3) 6.0(2)N2(5) yes
112 fexth 6.0(2)N2(3) 6.0(2)N2(5) yes
113 fexth 6.0(2)N2(3) 6.0(2)N2(5) yes
114 fexth 6.0(2)N2(3) 6.0(2)N2(5) yes
115 fexth 6.0(2)N2(3) 6.0(2)N2(5) yes
116 fexth 6.0(2)N2(3) 6.0(2)N2(5) yes
117 fexth 6.0(2)N2(3) 6.0(2)N2(5) yes
118 fexth 6.0(2)N2(3) 6.0(2)N2(5) yes
119 fexth 6.0(2)N2(3) 6.0(2)N2(5) yes
120 fexth 6.0(2)N2(3) 6.0(2)N2(5) yes
1 microcontroller v1.2.0.5 v1.2.0.5 no


Additional info for this installation:
--------------------------------------
Remove QoS & ACL config on L3 interfaces and SVIs if any

Service \"vpc\" : STP Preupgrade Check failed on VPC peer switch

Service \"lacp\" : LACP: Upgrade will be disruptive as timeout on fex lacp_issu req


Do you want to continue with the installation (y/n)? [n] y

Install is in progress, please wait.

Date: 2014-12-17 20:41:49 UTC
Le fex101 a fail pendant la mise a jour a chaud :

Module 101: Non-disruptive upgrading.
FAIL. Return code -1.

Remaining action::
\"Module(s) 102, 103, 104, 105, 106, 107, 108, 109, 110, 111, 112, 113, 114, 115, 116, 117, 118, 119, 120 still need to be upgraded\".

Install has failed. Return code 0x40930020 (Non-disruptive upgrade of a module failed).
Please identify the cause of the failure, and try 'install all' again.

Date: 2014-12-17 20:09:17 UTC
Nous avons toujours des port en err-disable. Nous mettons a jour le couple de n6 a chaud.

Date: 2014-12-17 19:00:55 UTC
Certain ports sur le fex113 (pas de chance...) sont passés en err-disable et on ne peut pas les shut/no shut, les commandes collent et ne sont pas appliqué

bhs2-11a-n6(config-if)# shut
Please check if command was successful using appropriate show commands

Seul moyens de rétablir: reload des switch, un par un.
Il n'y a pas d'impact pour les serveurs qui sont deja up (ça basculera sur le 2nd switch du couple)



Date: 2014-12-17 18:03:45 UTC
Nous allons les mettre a jour en 6.2.N2.5

Date: 2014-12-17 17:57:14 UTC
bhs2-11b-n6 a rebooté également.
Posted Dec 17, 2014 - 17:56 UTC