OVHcloud Network Status
Current status
Legend
- Operational
- Degraded performance
- Partial Outage
- Major Outage
- Under maintenance
sbg2-22a/b-n56
Incident
Report for Network & Infrastructure
Resolved
Certains machines connectées sur ce couple de switchs ne répondent plus.
Nous investiguons.
Update(s):
Date: 2016-10-14 00:04:22 UTC
22b est a jour également
kickstart: version 7.1(3)N1(4)
system: version 7.1(3)N1(4)
Tous les fexs étant des 2348, il faudrait les reloader 1 par 1 afin de mettre a jour le PHY.
Cette maj permet de fixer des bug sur le 10G baseT , or ce couple n'a que des srv en 1g.
Le reload du fex signifie du downtime, qui n'apportera pas de bugfix supplémentaire sur l'infra actuelle, je préfère reporter cette étape.
All done
Date: 2016-10-13 23:56:18 UTC
le 22b est en cours, cela va etre rapide, les fex sont deja updated
Compatibility check is done:
Module bootable Impact Install-type Reason
------ -------- -------------- ------------ ------
0 yes non-disruptive reset
2 yes non-disruptive rolling
100 yes non-disruptive none Source and Target images are of same version. Upgrade is not required
101 yes non-disruptive none Source and Target images are of same version. Upgrade is not required
102 yes non-disruptive none Source and Target images are of same version. Upgrade is not required
103 yes non-disruptive none Source and Target images are of same version. Upgrade is not required
104 yes non-disruptive none Source and Target images are of same version. Upgrade is not required
105 yes non-disruptive none Source and Target images are of same version. Upgrade is not required
106 yes non-disruptive none Source and Target images are of same version. Upgrade is not required
107 yes non-disruptive none Source and Target images are of same version. Upgrade is not required
108 yes non-disruptive none Source and Target images are of same version. Upgrade is not required
109 yes non-disruptive none Source and Target images are of same version. Upgrade is not required
110 yes non-disruptive none Source and Target images are of same version. Upgrade is not required
112 yes non-disruptive none Source and Target images are of same version. Upgrade is not required
113 yes non-disruptive none Source and Target images are of same version. Upgrade is not required
114 yes non-disruptive none Source and Target images are of same version. Upgrade is not required
115 yes non-disruptive none Source and Target images are of same version. Upgrade is not required
116 yes non-disruptive none Source and Target images are of same version. Upgrade is not required
117 yes non-disruptive none Source and Target images are of same version. Upgrade is not required
118 yes non-disruptive none Source and Target images are of same version. Upgrade is not required
119 yes non-disruptive none Source and Target images are of same version. Upgrade is not required
120 yes non-disruptive none Source and Target images are of same version. Upgrade is not required
Images will be upgraded according to following table:
Module Image Running-Version New-Version Upg-Required
------ ---------------- ---------------------- ---------------------- ------------
0 system 7.1(3)N1(1) 7.1(3)N1(4) yes
0 kickstart 7.1(3)N1(1) 7.1(3)N1(4) yes
0 bios v1.1.3(12/03/2015) v1.1.3(12/03/2015) no
0 power-seq SF-uC:39, SF-FPGA:6 SF-uC:37, SF-FPGA:5 no
0 iofpga v0.0.0.34 v0.0.0.34 no
2 power-seq SF-uC:23, SF-FPGA:7 SF-uC:21, SF-FPGA:6 no
2 iofpga v0.0.0.14 v0.0.0.14 no
100 fex4 7.1(3)N1(4) 7.1(3)N1(4) no
101 fex4 7.1(3)N1(4) 7.1(3)N1(4) no
102 fex4 7.1(3)N1(4) 7.1(3)N1(4) no
103 fex4 7.1(3)N1(4) 7.1(3)N1(4) no
104 fex4 7.1(3)N1(4) 7.1(3)N1(4) no
105 fex4 7.1(3)N1(4) 7.1(3)N1(4) no
106 fex4 7.1(3)N1(4) 7.1(3)N1(4) no
107 fex4 7.1(3)N1(4) 7.1(3)N1(4) no
108 fex4 7.1(3)N1(4) 7.1(3)N1(4) no
109 fex4 7.1(3)N1(4) 7.1(3)N1(4) no
110 fex4 7.1(3)N1(4) 7.1(3)N1(4) no
112 fex4 7.1(3)N1(4) 7.1(3)N1(4) no
113 fex4 7.1(3)N1(4) 7.1(3)N1(4) no
114 fex4 7.1(3)N1(4) 7.1(3)N1(4) no
115 fex4 7.1(3)N1(4) 7.1(3)N1(4) no
116 fex4 7.1(3)N1(4) 7.1(3)N1(4) no
117 fex4 7.1(3)N1(4) 7.1(3)N1(4) no
118 fex4 7.1(3)N1(4) 7.1(3)N1(4) no
119 fex4 7.1(3)N1(4) 7.1(3)N1(4) no
120 fex4 7.1(3)N1(4) 7.1(3)N1(4) no
Do you want to continue with the installation (y/n)? [n] y
Install is in progress, please wait.
Performing runtime checks.
[####################] 100% -- SUCCESS
Date: 2016-10-13 23:49:49 UTC
yeah 1er switch up2date
Module 120: Non-disruptive upgrading.
SUCCESS
Install has been successful.
Date: 2016-10-13 23:46:41 UTC
Module 118: Non-disruptive upgrading.
SUCCESS
Date: 2016-10-13 23:38:17 UTC
Module 109: Non-disruptive upgrading.
SUCCESS
Module 110: Non-disruptive upgrading.
SUCCESS
Date: 2016-10-13 23:23:31 UTC
Upg du premier fex en cours
Pre-loading modules.
SUCCESS
Module 100: Non-disruptive upgrading.
Date: 2016-10-13 23:17:11 UTC
le supervisor vient de reload
on est a jour
kickstart: version 7.1(3)N1(4)
system: version 7.1(3)N1(4)
BIOS compile time: 12/03/2015
kickstart image file is: bootflash:///n6000-uk9-kickstart.7.1.3.N1.4.bin
kickstart compile time: 4/25/2016 8:00:00 [04/28/2016 11:45:46]
system image file is: bootflash:///n6000-uk9.7.1.3.N1.4.bin
system compile time: 4/25/2016 8:00:00 [04/28/2016 11:46:28]
Hardware
cisco Nexus 5624Q Chassis (\"Nexus 5624Q Supervisor\")
Intel(R) Xeon(R) CPU @ 1.80 with 8230556 kB of memory.
Processor Board ID FOC1910326A
Device name: sbg2-22a-n56
bootflash: 7692288 kB
Kernel uptime is 0 day(s), 0 hour(s), 4 minute(s), 12 second(s)
Last reset at 46535 usecs after Fri Oct 14 01:12:15 2016
Reason: Reset due to upgrade
System version: 7.1(3)N1(1)
Service:
le switch est en train d'uploader les images des Fexs
sbg2-22a-n56# sh install all status
There is an on-going installation...
Enter Ctrl-C to go back to the prompt.
Continuing with installation process, please wait.
The login will be disabled until the installation is completed.
Performing supervisor state verification.
SUCCESS
Supervisor non-disruptive upgrade successful.
Pre-loading modules.
Date: 2016-10-13 23:12:13 UTC
GO !
Images will be upgraded according to following table:
Module Image Running-Version New-Version Upg-Required
------ ---------------- ---------------------- ---------------------- ------------
0 system 7.1(3)N1(1) 7.1(3)N1(4) yes
0 kickstart 7.1(3)N1(1) 7.1(3)N1(4) yes
0 bios v1.1.3(12/03/2015) v1.1.3(12/03/2015) no
0 power-seq SF-uC:39, SF-FPGA:6 SF-uC:37, SF-FPGA:5 no
0 iofpga v0.0.0.34 v0.0.0.34 no
2 power-seq SF-uC:23, SF-FPGA:7 SF-uC:21, SF-FPGA:6 no
2 iofpga v0.0.0.14 v0.0.0.14 no
100 fex4 7.1(3)N1(1) 7.1(3)N1(4) yes
101 fex4 7.1(3)N1(1) 7.1(3)N1(4) yes
102 fex4 7.1(3)N1(1) 7.1(3)N1(4) yes
103 fex4 7.1(3)N1(1) 7.1(3)N1(4) yes
104 fex4 7.1(3)N1(1) 7.1(3)N1(4) yes
105 fex4 7.1(3)N1(1) 7.1(3)N1(4) yes
106 fex4 7.1(3)N1(1) 7.1(3)N1(4) yes
107 fex4 7.1(3)N1(1) 7.1(3)N1(4) yes
108 fex4 7.1(3)N1(1) 7.1(3)N1(4) yes
109 fex4 7.1(3)N1(1) 7.1(3)N1(4) yes
110 fex4 7.1(3)N1(1) 7.1(3)N1(4) yes
112 fex4 7.1(3)N1(1) 7.1(3)N1(4) yes
113 fex4 7.1(3)N1(1) 7.1(3)N1(4) yes
114 fex4 7.1(3)N1(1) 7.1(3)N1(4) yes
115 fex4 7.1(3)N1(1) 7.1(3)N1(4) yes
116 fex4 7.1(3)N1(1) 7.1(3)N1(4) yes
117 fex4 7.1(3)N1(1) 7.1(3)N1(4) yes
118 fex4 7.1(3)N1(1) 7.1(3)N1(4) yes
119 fex4 7.1(3)N1(1) 7.1(3)N1(4) yes
120 fex4 7.1(3)N1(1) 7.1(3)N1(4) yes
Do you want to continue with the installation (y/n)? [n] y
Install is in progress, please wait.
Performing runtime checks.
[####################] 100% -- SUCCESS
Notifying services about the upgrade.
[####################] 100% -- SUCCESS
Setting boot variables.
[####################] 100% -- SUCCESS
Performing configuration copy.
[####################] 100% -- SUCCESS
Upgrade can no longer be aborted, any failure will result in a disruptive upgrade.
Requesting Line Cards to stop communication.
[####################] 100% -- SUCCESS
Requesting Sup Apps to stop communication.
[####################] 100% -- SUCCESS
Freeing memory in the file system.
[####################] 100% -- SUCCESS
Loading images into memory.
[####################] 100% -- SUCCESS
Date: 2016-10-13 22:54:57 UTC
on est pas trop mal la
[Important] [./nexus_precheck_issu.pl ] ---------------------------------------------
[Important] [./nexus_precheck_issu.pl ] - Device: sbg2-22b-n56 - Chassis: 5624Q
[Important] [./nexus_precheck_issu.pl ] ---------------------------------------------
[Important] [./nexus_precheck_issu.pl ] - SSH Check : [PASS]
[Important] [./nexus_precheck_issu.pl ] - FLASH Check : [PASS]
[Important] [./nexus_precheck_issu.pl ] - IMAGE Check : [PASS]
[Important] [./nexus_precheck_issu.pl ] - KICKSTART Check : [PASS]
[Important] [./nexus_precheck_issu.pl ] - SPACE Check : [PASS]
[Important] [./nexus_precheck_issu.pl ] - THRESHOLD Check : [PASS]
[Important] [./nexus_precheck_issu.pl ] - \\_ /var/tmp : [PASS]
[Important] [./nexus_precheck_issu.pl ] - \\_ /var/sysmgr : [PASS]
[Important] [./nexus_precheck_issu.pl ] - VLAN Check : [PASS]
[Important] [./nexus_precheck_issu.pl ] - TCAM Check : [PASS]
[Important] [./nexus_precheck_issu.pl ] - LICENCE Check : [PASS]
[Important] [./nexus_precheck_issu.pl ] - FEX Check : [PASS]
[Important] [./nexus_precheck_issu.pl ] - \\_ All Fex ONLINE : [PASS]
[Important] [./nexus_precheck_issu.pl ] - \\_ Provisionning : [PASS]
[Important] [./nexus_precheck_issu.pl ] - \\_ Attach : [PASS]
[Important] [./nexus_precheck_issu.pl ] - VPC Check : [PASS]
[Important] [./nexus_precheck_issu.pl ] - \\_ DomainID : [PASS]
[Important] [./nexus_precheck_issu.pl ] - \\_ Peer-Link : [PASS]
[Important] [./nexus_precheck_issu.pl ] - \\_ Keep-Alive : [PASS]
[Important] [./nexus_precheck_issu.pl ] - \\_ Config : [PASS]
[Important] [./nexus_precheck_issu.pl ] - \\_ Per-Vlan Cons : [PASS]
[Important] [./nexus_precheck_issu.pl ] - \\_ Type-2 Cons : [PASS]
[Important] [./nexus_precheck_issu.pl ] - INSTALL ALL IMPACT Check : [PASS]
[Important] [./nexus_precheck_issu.pl ] From prechecks you can GO! for ISSU
[Important] [./nexus_precheck_issu.pl ] ---------------------------------------------
Date: 2016-10-13 22:34:02 UTC
damned, la version du bios ne permet de passer en 7.1.4 directement. Je ferais donc l'upg vers la 7.1.3.N1.4 (contenant déjà beaucoup de bugfixs)
Date: 2016-10-13 22:24:54 UTC
Linux(debug)# bios_cmd version
card_id = 314
card_id = 314
Executing ibexpeak_bios_which_flash....
SPIBAR base = 0x0x77acb000; SPIBAR = 0x0x77acb800
spibar address = 0x77acb800
spi_cmd_32bit = 0xf9000000
spi_cmd = 0x424200; spi_cmd_32bit = 0xf9424200
SPI Flash Identification Rdid Information: 0x177120
Manufacturer Identification: 0x20
Device Information
Memory Type: 0x71
Memory ID: 0x17
Found flash device STM M25PX64, 8MB, Minimum sector size 4KB
Table: MFG Id: 0x20; DEV Id: 0x20
Table: MFG Id: 0x20; DEV Id: 0x71
Platform Type: Norcal-384
ibexpeak_set_bp_bits: setting BP2, BP1, BP0 in the status register to 0x0
magic string in flash = N5K-C5624Q
expected magic string-0 = N5K-C5624Q
expected magic string-1 = NPT6001-72
expected magic string-2 = N5K-C5672UP
expected magic string-3 = N5K-C5696Q
expected magic string-4 =
magic string found=1, i=0
BIOS version : v1.1.3(12/03/2015) <<<<<<<<<<<<<<<< avant BIOS version : v1.0.8(10/29/2014)
Linux(debug)#
Date: 2016-10-13 22:24:20 UTC
les BIOS du A et B sont a jour, on relance les precheck avant de balancer la sauce
Date: 2016-10-13 22:00:24 UTC
il faut commencer par upgrader les BIOS a la mimine
Important] [./nexus_precheck_issu.pl ] Switch: sbg2-22a-n56 - Fexes are all checked [State/Provosionning/Communication] !
[Important] [./nexus_precheck_issu.pl ] Switch: sbg2-22a-n56 Checking the ISSU Install All Impact - this can take ~5min !
[Important] [./nexus_precheck_issu.pl ] Switch: sbg2-22a-n56 | Module 0 upgrade impact : disruptive => reason: Bios needs upgrade
[Important] [./nexus_precheck_issu.pl ] Switch: sbg2-22a-n56 | Module 2 upgrade impact : disruptive => reason: Bios needs upgrade
[Important] [./nexus_precheck_issu.pl ] Switch: sbg2-22a-n56 | Module 100 upgrade impact : disruptive => reason: Bios needs upgrade
[Important] [./nexus_precheck_issu.pl ] Switch: sbg2-22a-n56 | Module 101 upgrade impact : disruptive => reason: Bios needs upgrade
[Important] [./nexus_precheck_issu.pl ] Switch: sbg2-22a-n56 | Module 102 upgrade impact : disruptive => reason: Bios needs upgrade
[Important] [./nexus_precheck_issu.pl ] Switch: sbg2-22a-n56 | Module 103 upgrade impact : disruptive => reason: Bios needs upgrade
[Important] [./nexus_precheck_issu.pl ] Switch: sbg2-22a-n56 | Module 104 upgrade impact : disruptive => reason: Bios needs upgrade
[Important] [./nexus_precheck_issu.pl ] Switch: sbg2-22a-n56 | Module 105 upgrade impact : disruptive => reason: Bios needs upgrade
[Important] [./nexus_precheck_issu.pl ] Switch: sbg2-22a-n56 | Module 106 upgrade impact : disruptive => reason: Bios needs upgrade
[Important] [./nexus_precheck_issu.pl ] Switch: sbg2-22a-n56 | Module 107 upgrade impact : disruptive => reason: Bios needs upgrade
[Important] [./nexus_precheck_issu.pl ] Switch: sbg2-22a-n56 | Module 108 upgrade impact : disruptive => reason: Bios needs upgrade
[Important] [./nexus_precheck_issu.pl ] Switch: sbg2-22a-n56 | Module 109 upgrade impact : disruptive => reason: Bios needs upgrade
[Important] [./nexus_precheck_issu.pl ] Switch: sbg2-22a-n56 | Module 110 upgrade impact : disruptive => reason: Bios needs upgrade
[Important] [./nexus_precheck_issu.pl ] Switch: sbg2-22a-n56 | Module 112 upgrade impact : disruptive => reason: Bios needs upgrade
[Important] [./nexus_precheck_issu.pl ] Switch: sbg2-22a-n56 | Module 113 upgrade impact : disruptive => reason: Bios needs upgrade
[Important] [./nexus_precheck_issu.pl ] Switch: sbg2-22a-n56 | Module 114 upgrade impact : disruptive => reason: Bios needs upgrade
[Important] [./nexus_precheck_issu.pl ] Switch: sbg2-22a-n56 | Module 115 upgrade impact : disruptive => reason: Bios needs upgrade
[Important] [./nexus_precheck_issu.pl ] Switch: sbg2-22a-n56 | Module 117 upgrade impact : disruptive => reason: Bios needs upgrade
[Important] [./nexus_precheck_issu.pl ] Switch: sbg2-22a-n56 | Module 118 upgrade impact : disruptive => reason: Bios needs upgrade
[Important] [./nexus_precheck_issu.pl ] Switch: sbg2-22a-n56 | Module 119 upgrade impact : disruptive => reason: Bios needs upgrade
[Important] [./nexus_precheck_issu.pl ] Switch: sbg2-22a-n56 | Module 120 upgrade impact : disruptive => reason: Bios needs upgrade
[Important] [./nexus_precheck_issu.pl ] Switch: sbg2-22a-n56 | Module 0 Image: system upgrade from 7.1(3)N1(1) to 7.1(4)N1(1) => upg needed = yes
Date: 2016-10-13 18:03:06 UTC
Le sw est isolés , les tech DC regardent les dernières panne
nous prenons les traces pour Cisco avant de faire le reload.
Nous allons le mettre a jour cette nuit ( 23h59 heure FR )
Date: 2016-10-13 17:20:37 UTC
Nous isolons à nouveau le sbg2-22a-n56.
Date: 2016-10-08 16:58:39 UTC
Tous les FEXs sont up. La redondance est rétablie.
Date: 2016-10-08 16:10:47 UTC
Le switch a redémarré, nous rallumons les interfaces précédemment désactivées afin de rétablir la redondance.
Date: 2016-10-08 15:07:39 UTC
Nous avons isolé sbg2-22a-n56, les alertes disparaissent.
Nous prenons des traces pour le constructeur et allons redémarrer le switch.
Nous investiguons.
Update(s):
Date: 2016-10-14 00:04:22 UTC
22b est a jour également
kickstart: version 7.1(3)N1(4)
system: version 7.1(3)N1(4)
Tous les fexs étant des 2348, il faudrait les reloader 1 par 1 afin de mettre a jour le PHY.
Cette maj permet de fixer des bug sur le 10G baseT , or ce couple n'a que des srv en 1g.
Le reload du fex signifie du downtime, qui n'apportera pas de bugfix supplémentaire sur l'infra actuelle, je préfère reporter cette étape.
All done
Date: 2016-10-13 23:56:18 UTC
le 22b est en cours, cela va etre rapide, les fex sont deja updated
Compatibility check is done:
Module bootable Impact Install-type Reason
------ -------- -------------- ------------ ------
0 yes non-disruptive reset
2 yes non-disruptive rolling
100 yes non-disruptive none Source and Target images are of same version. Upgrade is not required
101 yes non-disruptive none Source and Target images are of same version. Upgrade is not required
102 yes non-disruptive none Source and Target images are of same version. Upgrade is not required
103 yes non-disruptive none Source and Target images are of same version. Upgrade is not required
104 yes non-disruptive none Source and Target images are of same version. Upgrade is not required
105 yes non-disruptive none Source and Target images are of same version. Upgrade is not required
106 yes non-disruptive none Source and Target images are of same version. Upgrade is not required
107 yes non-disruptive none Source and Target images are of same version. Upgrade is not required
108 yes non-disruptive none Source and Target images are of same version. Upgrade is not required
109 yes non-disruptive none Source and Target images are of same version. Upgrade is not required
110 yes non-disruptive none Source and Target images are of same version. Upgrade is not required
112 yes non-disruptive none Source and Target images are of same version. Upgrade is not required
113 yes non-disruptive none Source and Target images are of same version. Upgrade is not required
114 yes non-disruptive none Source and Target images are of same version. Upgrade is not required
115 yes non-disruptive none Source and Target images are of same version. Upgrade is not required
116 yes non-disruptive none Source and Target images are of same version. Upgrade is not required
117 yes non-disruptive none Source and Target images are of same version. Upgrade is not required
118 yes non-disruptive none Source and Target images are of same version. Upgrade is not required
119 yes non-disruptive none Source and Target images are of same version. Upgrade is not required
120 yes non-disruptive none Source and Target images are of same version. Upgrade is not required
Images will be upgraded according to following table:
Module Image Running-Version New-Version Upg-Required
------ ---------------- ---------------------- ---------------------- ------------
0 system 7.1(3)N1(1) 7.1(3)N1(4) yes
0 kickstart 7.1(3)N1(1) 7.1(3)N1(4) yes
0 bios v1.1.3(12/03/2015) v1.1.3(12/03/2015) no
0 power-seq SF-uC:39, SF-FPGA:6 SF-uC:37, SF-FPGA:5 no
0 iofpga v0.0.0.34 v0.0.0.34 no
2 power-seq SF-uC:23, SF-FPGA:7 SF-uC:21, SF-FPGA:6 no
2 iofpga v0.0.0.14 v0.0.0.14 no
100 fex4 7.1(3)N1(4) 7.1(3)N1(4) no
101 fex4 7.1(3)N1(4) 7.1(3)N1(4) no
102 fex4 7.1(3)N1(4) 7.1(3)N1(4) no
103 fex4 7.1(3)N1(4) 7.1(3)N1(4) no
104 fex4 7.1(3)N1(4) 7.1(3)N1(4) no
105 fex4 7.1(3)N1(4) 7.1(3)N1(4) no
106 fex4 7.1(3)N1(4) 7.1(3)N1(4) no
107 fex4 7.1(3)N1(4) 7.1(3)N1(4) no
108 fex4 7.1(3)N1(4) 7.1(3)N1(4) no
109 fex4 7.1(3)N1(4) 7.1(3)N1(4) no
110 fex4 7.1(3)N1(4) 7.1(3)N1(4) no
112 fex4 7.1(3)N1(4) 7.1(3)N1(4) no
113 fex4 7.1(3)N1(4) 7.1(3)N1(4) no
114 fex4 7.1(3)N1(4) 7.1(3)N1(4) no
115 fex4 7.1(3)N1(4) 7.1(3)N1(4) no
116 fex4 7.1(3)N1(4) 7.1(3)N1(4) no
117 fex4 7.1(3)N1(4) 7.1(3)N1(4) no
118 fex4 7.1(3)N1(4) 7.1(3)N1(4) no
119 fex4 7.1(3)N1(4) 7.1(3)N1(4) no
120 fex4 7.1(3)N1(4) 7.1(3)N1(4) no
Do you want to continue with the installation (y/n)? [n] y
Install is in progress, please wait.
Performing runtime checks.
[####################] 100% -- SUCCESS
Date: 2016-10-13 23:49:49 UTC
yeah 1er switch up2date
Module 120: Non-disruptive upgrading.
SUCCESS
Install has been successful.
Date: 2016-10-13 23:46:41 UTC
Module 118: Non-disruptive upgrading.
SUCCESS
Date: 2016-10-13 23:38:17 UTC
Module 109: Non-disruptive upgrading.
SUCCESS
Module 110: Non-disruptive upgrading.
SUCCESS
Date: 2016-10-13 23:23:31 UTC
Upg du premier fex en cours
Pre-loading modules.
SUCCESS
Module 100: Non-disruptive upgrading.
Date: 2016-10-13 23:17:11 UTC
le supervisor vient de reload
on est a jour
kickstart: version 7.1(3)N1(4)
system: version 7.1(3)N1(4)
BIOS compile time: 12/03/2015
kickstart image file is: bootflash:///n6000-uk9-kickstart.7.1.3.N1.4.bin
kickstart compile time: 4/25/2016 8:00:00 [04/28/2016 11:45:46]
system image file is: bootflash:///n6000-uk9.7.1.3.N1.4.bin
system compile time: 4/25/2016 8:00:00 [04/28/2016 11:46:28]
Hardware
cisco Nexus 5624Q Chassis (\"Nexus 5624Q Supervisor\")
Intel(R) Xeon(R) CPU @ 1.80 with 8230556 kB of memory.
Processor Board ID FOC1910326A
Device name: sbg2-22a-n56
bootflash: 7692288 kB
Kernel uptime is 0 day(s), 0 hour(s), 4 minute(s), 12 second(s)
Last reset at 46535 usecs after Fri Oct 14 01:12:15 2016
Reason: Reset due to upgrade
System version: 7.1(3)N1(1)
Service:
le switch est en train d'uploader les images des Fexs
sbg2-22a-n56# sh install all status
There is an on-going installation...
Enter Ctrl-C to go back to the prompt.
Continuing with installation process, please wait.
The login will be disabled until the installation is completed.
Performing supervisor state verification.
SUCCESS
Supervisor non-disruptive upgrade successful.
Pre-loading modules.
Date: 2016-10-13 23:12:13 UTC
GO !
Images will be upgraded according to following table:
Module Image Running-Version New-Version Upg-Required
------ ---------------- ---------------------- ---------------------- ------------
0 system 7.1(3)N1(1) 7.1(3)N1(4) yes
0 kickstart 7.1(3)N1(1) 7.1(3)N1(4) yes
0 bios v1.1.3(12/03/2015) v1.1.3(12/03/2015) no
0 power-seq SF-uC:39, SF-FPGA:6 SF-uC:37, SF-FPGA:5 no
0 iofpga v0.0.0.34 v0.0.0.34 no
2 power-seq SF-uC:23, SF-FPGA:7 SF-uC:21, SF-FPGA:6 no
2 iofpga v0.0.0.14 v0.0.0.14 no
100 fex4 7.1(3)N1(1) 7.1(3)N1(4) yes
101 fex4 7.1(3)N1(1) 7.1(3)N1(4) yes
102 fex4 7.1(3)N1(1) 7.1(3)N1(4) yes
103 fex4 7.1(3)N1(1) 7.1(3)N1(4) yes
104 fex4 7.1(3)N1(1) 7.1(3)N1(4) yes
105 fex4 7.1(3)N1(1) 7.1(3)N1(4) yes
106 fex4 7.1(3)N1(1) 7.1(3)N1(4) yes
107 fex4 7.1(3)N1(1) 7.1(3)N1(4) yes
108 fex4 7.1(3)N1(1) 7.1(3)N1(4) yes
109 fex4 7.1(3)N1(1) 7.1(3)N1(4) yes
110 fex4 7.1(3)N1(1) 7.1(3)N1(4) yes
112 fex4 7.1(3)N1(1) 7.1(3)N1(4) yes
113 fex4 7.1(3)N1(1) 7.1(3)N1(4) yes
114 fex4 7.1(3)N1(1) 7.1(3)N1(4) yes
115 fex4 7.1(3)N1(1) 7.1(3)N1(4) yes
116 fex4 7.1(3)N1(1) 7.1(3)N1(4) yes
117 fex4 7.1(3)N1(1) 7.1(3)N1(4) yes
118 fex4 7.1(3)N1(1) 7.1(3)N1(4) yes
119 fex4 7.1(3)N1(1) 7.1(3)N1(4) yes
120 fex4 7.1(3)N1(1) 7.1(3)N1(4) yes
Do you want to continue with the installation (y/n)? [n] y
Install is in progress, please wait.
Performing runtime checks.
[####################] 100% -- SUCCESS
Notifying services about the upgrade.
[####################] 100% -- SUCCESS
Setting boot variables.
[####################] 100% -- SUCCESS
Performing configuration copy.
[####################] 100% -- SUCCESS
Upgrade can no longer be aborted, any failure will result in a disruptive upgrade.
Requesting Line Cards to stop communication.
[####################] 100% -- SUCCESS
Requesting Sup Apps to stop communication.
[####################] 100% -- SUCCESS
Freeing memory in the file system.
[####################] 100% -- SUCCESS
Loading images into memory.
[####################] 100% -- SUCCESS
Date: 2016-10-13 22:54:57 UTC
on est pas trop mal la
[Important] [./nexus_precheck_issu.pl ] ---------------------------------------------
[Important] [./nexus_precheck_issu.pl ] - Device: sbg2-22b-n56 - Chassis: 5624Q
[Important] [./nexus_precheck_issu.pl ] ---------------------------------------------
[Important] [./nexus_precheck_issu.pl ] - SSH Check : [PASS]
[Important] [./nexus_precheck_issu.pl ] - FLASH Check : [PASS]
[Important] [./nexus_precheck_issu.pl ] - IMAGE Check : [PASS]
[Important] [./nexus_precheck_issu.pl ] - KICKSTART Check : [PASS]
[Important] [./nexus_precheck_issu.pl ] - SPACE Check : [PASS]
[Important] [./nexus_precheck_issu.pl ] - THRESHOLD Check : [PASS]
[Important] [./nexus_precheck_issu.pl ] - \\_ /var/tmp : [PASS]
[Important] [./nexus_precheck_issu.pl ] - \\_ /var/sysmgr : [PASS]
[Important] [./nexus_precheck_issu.pl ] - VLAN Check : [PASS]
[Important] [./nexus_precheck_issu.pl ] - TCAM Check : [PASS]
[Important] [./nexus_precheck_issu.pl ] - LICENCE Check : [PASS]
[Important] [./nexus_precheck_issu.pl ] - FEX Check : [PASS]
[Important] [./nexus_precheck_issu.pl ] - \\_ All Fex ONLINE : [PASS]
[Important] [./nexus_precheck_issu.pl ] - \\_ Provisionning : [PASS]
[Important] [./nexus_precheck_issu.pl ] - \\_ Attach : [PASS]
[Important] [./nexus_precheck_issu.pl ] - VPC Check : [PASS]
[Important] [./nexus_precheck_issu.pl ] - \\_ DomainID : [PASS]
[Important] [./nexus_precheck_issu.pl ] - \\_ Peer-Link : [PASS]
[Important] [./nexus_precheck_issu.pl ] - \\_ Keep-Alive : [PASS]
[Important] [./nexus_precheck_issu.pl ] - \\_ Config : [PASS]
[Important] [./nexus_precheck_issu.pl ] - \\_ Per-Vlan Cons : [PASS]
[Important] [./nexus_precheck_issu.pl ] - \\_ Type-2 Cons : [PASS]
[Important] [./nexus_precheck_issu.pl ] - INSTALL ALL IMPACT Check : [PASS]
[Important] [./nexus_precheck_issu.pl ] From prechecks you can GO! for ISSU
[Important] [./nexus_precheck_issu.pl ] ---------------------------------------------
Date: 2016-10-13 22:34:02 UTC
damned, la version du bios ne permet de passer en 7.1.4 directement. Je ferais donc l'upg vers la 7.1.3.N1.4 (contenant déjà beaucoup de bugfixs)
Date: 2016-10-13 22:24:54 UTC
Linux(debug)# bios_cmd version
card_id = 314
card_id = 314
Executing ibexpeak_bios_which_flash....
SPIBAR base = 0x0x77acb000; SPIBAR = 0x0x77acb800
spibar address = 0x77acb800
spi_cmd_32bit = 0xf9000000
spi_cmd = 0x424200; spi_cmd_32bit = 0xf9424200
SPI Flash Identification Rdid Information: 0x177120
Manufacturer Identification: 0x20
Device Information
Memory Type: 0x71
Memory ID: 0x17
Found flash device STM M25PX64, 8MB, Minimum sector size 4KB
Table: MFG Id: 0x20; DEV Id: 0x20
Table: MFG Id: 0x20; DEV Id: 0x71
Platform Type: Norcal-384
ibexpeak_set_bp_bits: setting BP2, BP1, BP0 in the status register to 0x0
magic string in flash = N5K-C5624Q
expected magic string-0 = N5K-C5624Q
expected magic string-1 = NPT6001-72
expected magic string-2 = N5K-C5672UP
expected magic string-3 = N5K-C5696Q
expected magic string-4 =
magic string found=1, i=0
BIOS version : v1.1.3(12/03/2015) <<<<<<<<<<<<<<<< avant BIOS version : v1.0.8(10/29/2014)
Linux(debug)#
Date: 2016-10-13 22:24:20 UTC
les BIOS du A et B sont a jour, on relance les precheck avant de balancer la sauce
Date: 2016-10-13 22:00:24 UTC
il faut commencer par upgrader les BIOS a la mimine
Important] [./nexus_precheck_issu.pl ] Switch: sbg2-22a-n56 - Fexes are all checked [State/Provosionning/Communication] !
[Important] [./nexus_precheck_issu.pl ] Switch: sbg2-22a-n56 Checking the ISSU Install All Impact - this can take ~5min !
[Important] [./nexus_precheck_issu.pl ] Switch: sbg2-22a-n56 | Module 0 upgrade impact : disruptive => reason: Bios needs upgrade
[Important] [./nexus_precheck_issu.pl ] Switch: sbg2-22a-n56 | Module 2 upgrade impact : disruptive => reason: Bios needs upgrade
[Important] [./nexus_precheck_issu.pl ] Switch: sbg2-22a-n56 | Module 100 upgrade impact : disruptive => reason: Bios needs upgrade
[Important] [./nexus_precheck_issu.pl ] Switch: sbg2-22a-n56 | Module 101 upgrade impact : disruptive => reason: Bios needs upgrade
[Important] [./nexus_precheck_issu.pl ] Switch: sbg2-22a-n56 | Module 102 upgrade impact : disruptive => reason: Bios needs upgrade
[Important] [./nexus_precheck_issu.pl ] Switch: sbg2-22a-n56 | Module 103 upgrade impact : disruptive => reason: Bios needs upgrade
[Important] [./nexus_precheck_issu.pl ] Switch: sbg2-22a-n56 | Module 104 upgrade impact : disruptive => reason: Bios needs upgrade
[Important] [./nexus_precheck_issu.pl ] Switch: sbg2-22a-n56 | Module 105 upgrade impact : disruptive => reason: Bios needs upgrade
[Important] [./nexus_precheck_issu.pl ] Switch: sbg2-22a-n56 | Module 106 upgrade impact : disruptive => reason: Bios needs upgrade
[Important] [./nexus_precheck_issu.pl ] Switch: sbg2-22a-n56 | Module 107 upgrade impact : disruptive => reason: Bios needs upgrade
[Important] [./nexus_precheck_issu.pl ] Switch: sbg2-22a-n56 | Module 108 upgrade impact : disruptive => reason: Bios needs upgrade
[Important] [./nexus_precheck_issu.pl ] Switch: sbg2-22a-n56 | Module 109 upgrade impact : disruptive => reason: Bios needs upgrade
[Important] [./nexus_precheck_issu.pl ] Switch: sbg2-22a-n56 | Module 110 upgrade impact : disruptive => reason: Bios needs upgrade
[Important] [./nexus_precheck_issu.pl ] Switch: sbg2-22a-n56 | Module 112 upgrade impact : disruptive => reason: Bios needs upgrade
[Important] [./nexus_precheck_issu.pl ] Switch: sbg2-22a-n56 | Module 113 upgrade impact : disruptive => reason: Bios needs upgrade
[Important] [./nexus_precheck_issu.pl ] Switch: sbg2-22a-n56 | Module 114 upgrade impact : disruptive => reason: Bios needs upgrade
[Important] [./nexus_precheck_issu.pl ] Switch: sbg2-22a-n56 | Module 115 upgrade impact : disruptive => reason: Bios needs upgrade
[Important] [./nexus_precheck_issu.pl ] Switch: sbg2-22a-n56 | Module 117 upgrade impact : disruptive => reason: Bios needs upgrade
[Important] [./nexus_precheck_issu.pl ] Switch: sbg2-22a-n56 | Module 118 upgrade impact : disruptive => reason: Bios needs upgrade
[Important] [./nexus_precheck_issu.pl ] Switch: sbg2-22a-n56 | Module 119 upgrade impact : disruptive => reason: Bios needs upgrade
[Important] [./nexus_precheck_issu.pl ] Switch: sbg2-22a-n56 | Module 120 upgrade impact : disruptive => reason: Bios needs upgrade
[Important] [./nexus_precheck_issu.pl ] Switch: sbg2-22a-n56 | Module 0 Image: system upgrade from 7.1(3)N1(1) to 7.1(4)N1(1) => upg needed = yes
Date: 2016-10-13 18:03:06 UTC
Le sw est isolés , les tech DC regardent les dernières panne
nous prenons les traces pour Cisco avant de faire le reload.
Nous allons le mettre a jour cette nuit ( 23h59 heure FR )
Date: 2016-10-13 17:20:37 UTC
Nous isolons à nouveau le sbg2-22a-n56.
Date: 2016-10-08 16:58:39 UTC
Tous les FEXs sont up. La redondance est rétablie.
Date: 2016-10-08 16:10:47 UTC
Le switch a redémarré, nous rallumons les interfaces précédemment désactivées afin de rétablir la redondance.
Date: 2016-10-08 15:07:39 UTC
Nous avons isolé sbg2-22a-n56, les alertes disparaissent.
Nous prenons des traces pour le constructeur et allons redémarrer le switch.
This incident affected: Infrastructure || SBG (SBG1, SBG3, SBG4).