OVHcloud Network Status
Current status
Legend
- Operational
- Degraded performance
- Partial Outage
- Major Outage
- Under maintenance
nexus 176.31.236.0/24 et 176.31.237.0/24
Scheduled Maintenance
Report for Network & Infrastructure
Completed
Nous devons mettre à jour le couple de n5548 gérant ces 2 réseaux.
Suite a un bug en 6.0(2)N2(2) , certain port du châssis passe down et reste down.
Actuellement 3 FEX sont single homed à cause de ce bug, il n'y a pas d'impact sur la production mais il n'y a plus de redondance pour ces 3 FEXs.
Afin de préparer la mise a jour en ISSU (hitless), nous allons rebooter le module 2 du Nexus avec ses ports down.
Avec de la chance, cela suffira à re-uppe les ports, sinon il faudra reloader le châssis complètement.
Par la suite, on enchaînera par l'update en elle même vers 7.0.5.N1a
L'intervention va débuter dans qq minutes
Update(s):
Date: 2015-06-23 00:01:21 UTC
ouf !
La mise a jour est a présent terminée - 3h ...
# sh fex
FEX FEX FEX FEX Fex
Number Description State Model Serial
------------------------------------------------------------------------
100 FEX0100 Online N2K-C2248TP-1GE SSI1444081Y
101 FEX0101 Online N2K-C2248TP-1GE SSI1444081S
102 FEX0102 Online N2K-C2248TP-1GE SSI15330JJV
103 FEX0103 Online N2K-C2248TP-1GE SSI153508ET
104 FEX0104 Online N2K-C2248TP-1GE SSI15391ATG
105 FEX0105 Online N2K-C2232PP-10GE SSI1541037V
106 FEX0106 Online N2K-C2232PP-10GE SSI14160FXW
107 FEX0107 Online N2K-C2232PP-10GE SSI153409GJ
108 FEX0108 Online N2K-C2232PP-10GE SSI14160MT9
109 FEX0109 Online N2K-C2232PP-10GE SSI154103G2
110 FEX0110 Online N2K-C2232PP-10GE SSI15380LY5
111 FEX0111 Online N2K-C2248TP-1GE SSI144506KC
112 FEX0112 Online N2K-C2232PP-10GE SSI154309WX
113 FEX0113 Online N2K-C2232TM-E-10GE SSI18420A01
114 FEX0114 Online N2K-C2232TM-E-10GE SSI1845091Q
115 FEX0115 Online N2K-C2232TM-E-10GE SSI184906NR
116 FEX0116 Online N2K-C2232TM-E-10GE SSI185007M9
117 FEX0117 Online N2K-C2232TM-E-10GE SSI184906NQ
Date: 2015-06-22 23:49:31 UTC
reload power cycle done, je up les fex
# sh fex
FEX FEX FEX FEX Fex
Number Description State Model Serial
------------------------------------------------------------------------
100 FEX0100 Connected N2K-C2248TP-1GE SSI1444081Y
101 FEX0101 Online N2K-C2248TP-1GE SSI1444081S
102 FEX0102 Connected N2K-C2248TP-1GE SSI15330JJV
103 FEX0103 Connected N2K-C2248TP-1GE SSI153508ET
104 FEX0104 Connected N2K-C2248TP-1GE SSI15391ATG
105 FEX0105 Connected N2K-C2232PP-10GE SSI1541037V
106 FEX0106 Connected N2K-C2232PP-10GE SSI14160FXW
107 FEX0107 Connected N2K-C2232PP-10GE SSI153409GJ
108 FEX0108 Connected N2K-C2232PP-10GE SSI14160MT9
109 FEX0109 Connected N2K-C2232PP-10GE SSI154103G2
110 FEX0110 Connected N2K-C2232PP-10GE SSI15380LY5
111 FEX0111 Connected N2K-C2248TP-1GE SSI144506KC
112 FEX0112 Connected N2K-C2232PP-10GE SSI154309WX
113 FEX0113 Online N2K-C2232TM-E-10GE SSI18420A01
114 FEX0114 Connected N2K-C2232TM-E-10GE SSI1845091Q
115 FEX0115 Connected N2K-C2232TM-E-10GE SSI184906NR
116 FEX0116 Connected N2K-C2232TM-E-10GE SSI185007M9
117 FEX0117 Connected N2K-C2232TM-E-10GE SSI184906NQ
Date: 2015-06-22 23:27:58 UTC
all godd, j'isole a present le 1er nexus
Date: 2015-06-22 23:18:17 UTC
# sh fex
FEX FEX FEX FEX Fex
Number Description State Model Serial
------------------------------------------------------------------------
100 FEX0100 Online N2K-C2248TP-1GE SSI1444081Y
101 FEX0101 Online N2K-C2248TP-1GE SSI1444081S
102 FEX0102 Online N2K-C2248TP-1GE SSI15330JJV
103 FEX0103 Online N2K-C2248TP-1GE SSI153508ET
104 FEX0104 Online N2K-C2248TP-1GE SSI15391ATG
105 FEX0105 Connected N2K-C2232PP-10GE SSI1541037V
106 FEX0106 Connected N2K-C2232PP-10GE SSI14160FXW
107 FEX0107 Online N2K-C2232PP-10GE SSI153409GJ
108 FEX0108 Connected N2K-C2232PP-10GE SSI14160MT9
109 FEX0109 Connected N2K-C2232PP-10GE SSI154103G2
110 FEX0110 Connected N2K-C2232PP-10GE SSI15380LY5
111 FEX0111 Connected N2K-C2248TP-1GE SSI144506KC
Date: 2015-06-22 23:09:32 UTC
Reload okay, je up les fex sur ce nexus pour pouvoir reloader le 1er
Date: 2015-06-22 23:00:57 UTC
# reload power-cycle
WARNING: This command will reboot the system
Do you want to continue? (y/n) [n] y
Date: 2015-06-22 23:00:29 UTC
Performing supervisor state verification.
SUCCESS
Supervisor non-disruptive upgrade successful.
Install has been successful.
Date: 2015-06-22 22:55:47 UTC
Module 1: Refreshing compact flash and upgrading bios/loader/bootrom/power-seq.
Warning: please do not remove or power off the module at this time.
Note: Power-seq upgrade needs a power-cycle to take into effect.
Use command 'reload power-cycle'
Note: Micro-controller upgrade needs a power-cycle to take into effect.
On success of micro-controller upgrade, SWITCH OFF THE POWER to the system and then, power it up.
il faudra faire un reset hard ou reload power cycle pour maj le power sequencer
Date: 2015-06-22 22:54:51 UTC
Compatibility check is done:
Module bootable Impact Install-type Reason
------ -------- -------------- ------------ ------
1 yes non-disruptive reset
2 yes non-disruptive rolling
3 yes non-disruptive rolling
Images will be upgraded according to following table:
Module Image Running-Version New-Version Upg-Required
------ ---------------- ---------------------- ---------------------- ------------
1 system 6.0(2)N2(2) 7.0(5)N1(1a) yes
1 kickstart 6.0(2)N2(2) 7.0(5)N1(1a) yes
1 bios v3.6.0(05/09/2012) v3.6.0(05/09/2012) no
1 power-seq v1.0 v3.0 yes
1 SFP-uC v1.0.0.0 v1.0.0.0 no
2 power-seq v1.0 v1.0 no
3 power-seq v2.0 v2.0 no
1 microcontroller v1.2.0.1 v1.2.0.1 no
Do you want to continue with the installation (y/n)? [n] y
Install is in progress, please wait.
Performing runtime checks.
[####################] 100% -- SUCCESS
Notifying services about the upgrade.
[####################] 100% -- SUCCESS
Setting boot variables.
[# ] 0%
Date: 2015-06-22 22:53:27 UTC
L'install est okay sur le 1er n5
Module 117: Non-disruptive upgrading.
SUCCESS
Install has been successful.
Software
BIOS: version 3.6.0
loader: version N/A
kickstart: version 7.0(5)N1(1a)
system: version 7.0(5)N1(1a)
Je lance l'ISSU sur le 2nd (les fexs ne sont tjs pas attachés dessus, ça devrait être rapide)
# install all kickstart bootflash:n5000-uk9-kickstart.7.0.5.N1.1a.bin system bootflash:n5000-uk9.7.0.5.N1.1a.bin
Verifying image bootflash:/n5000-uk9-kickstart.7.0.5.N1.1a.bin for boot variable \"kickstart\".
[####################] 100% -- SUCCESS
Verifying image bootflash:/n5000-uk9.7.0.5.N1.1a.bin for boot variable \"system\".
[####################] 100% -- SUCCESS
Verifying image type.
[########### ] 50%
Date: 2015-06-22 22:44:18 UTC
le dernier FEX en cours
Module 115: Non-disruptive upgrading.
SUCCESS
Module 116: Non-disruptive upgrading.
SUCCESS
Module 117: Non-disruptive upgrading.
Date: 2015-06-22 22:39:05 UTC
Module 111: Non-disruptive upgrading.
SUCCESS
Module 112: Non-disruptive upgrading.
SUCCESS
Module 113: Non-disruptive upgrading.
Date: 2015-06-22 22:26:17 UTC
Pre-loading modules.
SUCCESS
Module 100: Non-disruptive upgrading.
SUCCESS
Module 101: Non-disruptive upgrading.
SUCCESS
Module 102: Non-disruptive upgrading.
SUCCESS
Module 103: Non-disruptive upgrading.
Date: 2015-06-22 22:21:47 UTC
enfin le 1er fex debute son update
Continuing with installation process, please wait.
The login will be disabled until the installation is completed.
Performing supervisor state verification.
SUCCESS
Supervisor non-disruptive upgrade successful.
Pre-loading modules.
SUCCESS
Module 100: Non-disruptive upgrading.
Date: 2015-06-22 22:07:28 UTC
yeah !
on y va
Notifying services about system upgrade.
[####################] 100% -- SUCCESS
Compatibility check is done:
Module bootable Impact Install-type Reason
------ -------- -------------- ------------ ------
1 yes non-disruptive reset
2 yes non-disruptive rolling
3 yes non-disruptive rolling
100 yes non-disruptive rolling
101 yes non-disruptive rolling
102 yes non-disruptive rolling
103 yes non-disruptive rolling
104 yes non-disruptive rolling
105 yes non-disruptive rolling
106 yes non-disruptive rolling
107 yes non-disruptive rolling
108 yes non-disruptive rolling
109 yes non-disruptive rolling
110 yes non-disruptive rolling
111 yes non-disruptive rolling
112 yes non-disruptive rolling
113 yes non-disruptive rolling
114 yes non-disruptive rolling
115 yes non-disruptive rolling
116 yes non-disruptive rolling
117 yes non-disruptive rolling
Images will be upgraded according to following table:
Module Image Running-Version New-Version Upg-Required
------ ---------------- ---------------------- ---------------------- ------------
1 system 6.0(2)N2(2) 7.0(5)N1(1a) yes
1 kickstart 6.0(2)N2(2) 7.0(5)N1(1a) yes
1 bios v3.6.0(05/09/2012) v3.6.0(05/09/2012) no
1 power-seq v1.0 v3.0 yes
1 SFP-uC v1.0.0.0 v1.0.0.0 no
2 power-seq v1.0 v1.0 no
3 power-seq v2.0 v2.0 no
100 fexth 6.0(2)N2(2) 7.0(5)N1(1a) yes
101 fexth 6.0(2)N2(2) 7.0(5)N1(1a) yes
102 fexth 6.0(2)N2(2) 7.0(5)N1(1a) yes
103 fexth 6.0(2)N2(2) 7.0(5)N1(1a) yes
104 fexth 6.0(2)N2(2) 7.0(5)N1(1a) yes
105 fexth 6.0(2)N2(2) 7.0(5)N1(1a) yes
106 fexth 6.0(2)N2(2) 7.0(5)N1(1a) yes
107 fexth 6.0(2)N2(2) 7.0(5)N1(1a) yes
108 fexth 6.0(2)N2(2) 7.0(5)N1(1a) yes
109 fexth 6.0(2)N2(2) 7.0(5)N1(1a) yes
110 fexth 6.0(2)N2(2) 7.0(5)N1(1a) yes
111 fexth 6.0(2)N2(2) 7.0(5)N1(1a) yes
112 fexth 6.0(2)N2(2) 7.0(5)N1(1a) yes
113 fexth 6.0(2)N2(2) 7.0(5)N1(1a) yes
114 fexth 6.0(2)N2(2) 7.0(5)N1(1a) yes
115 fexth 6.0(2)N2(2) 7.0(5)N1(1a) yes
116 fexth 6.0(2)N2(2) 7.0(5)N1(1a) yes
117 fexth 6.0(2)N2(2) 7.0(5)N1(1a) yes
1 microcontroller v1.2.0.1 v1.2.0.1 no
Do you want to continue with the installation (y/n)? [n] y
Install is in progress, please wait.
Performing runtime checks.
[####################] 100% -- SUCCESS
Notifying services about the upgrade.
[####################] 100% -- SUCCESS
Setting boot variables.
[# ] 0%
Date: 2015-06-22 22:05:43 UTC
encore des soucis...
# install all kickstart bootflash:n5000-uk9-kickstart.7.0.5.N1.1a.bin system bootflash:n5000-uk9.7.0.5.N1.1a.bin
Pre-upgrade check failed. Return code 0x40930062 (free space in the filesystem is below threshold).
# sh system internal flash
Mount-on 1K-blocks Used Available Use% Filesystem
/ 409600 78636 330964 20 /dev/root
/proc 0 0 0 0 proc
/post 2048 4 2044 1 none
/var 409600 78636 330964 20 none
/sys 0 0 0 0 none
/isan 1536000 732476 803524 48 none
/var/tmp 307200 3304 303896 2 none
/var/sysmgr 1024000 1024000 0 100 none <<<<<<<<<<< full 100%
/nxos/tmp 40960 36 40924 1 none
/var/sysmgr/ftp 409600 84 409516 1 none
/var/sysmgr/ftp/cores 102400 0 102400 0 none
/callhome 61440 0 61440 0 none
/dev/shm 524288 288972 235316 56 none
/volatile 153600 4 153596 1 none
/debug 20480 0 20480 0 none
/dev/mqueue 0 0 0 0 none
/debugfs 0 0 0 0 nodev
/mnt/plog 49024 1344 47680 3 /dev/mtdblock2
/mnt/cfg/0 114171 5584 102692 6 /dev/sda5
/mnt/cfg/1 113193 5584 101765 6 /dev/sda6
/var/sysmgr/startup-cfg 409600 16652 392948 5 none
/dev/pts 0 0 0 0 devpts
/mnt/pss 114203 6370 101937 6 /dev/sda4
/bootflash 1612212 620708 909604 41 /dev/sda3
/proc/bus/usb 0 0 0 0 none
/cgroup 0 0 0 0 vdccontrol
apres un coup de plugin ... on est bon
inux(debug)# df -h | grep sysmgr
none 1000M 4.0K 1000M 1% /var/sysmgr
none 400M 84K 400M 1% /var/sysmgr/ftp
none 100M 0 100M 0% /var/sysmgr/ftp/cores
none 400M 17M 384M 5% /var/sysmgr/startup-cfg
Linux(debug)# exit
exit
#
# install all kickstart bootflash:n5000-uk9-kickstart.7.0.5.N1.1a.bin system bootflash:n5000-uk9.7.0.5.N1.1a.bin
Verifying image bootflash:/n5000-uk9-kickstart.7.0.5.N1.1a.bin for boot variable \"kickstart\".
[####################] 100% -- SUCCESS
Verifying image bootflash:/n5000-uk9.7.0.5.N1.1a.bin for boot variable \"system\".
[####################] 100% -- SUCCESS
Verifying image type.
[####################] 100% -- SUCCESS
Extracting \"system\" version from image bootflash:/n5000-uk9.7.0.5.N1.1a.bin.
[####################] 100% -- SUCCESS
Extracting \"kickstart\" version from image bootflash:/n5000-uk9-kickstart.7.0.5.N1.1a.bin.
[####################] 100% -- SUCCESS
Extracting \"bios\" version from image bootflash:/n5000-uk9.7.0.5.N1.1a.bin.
[# ] 0%
. a suivre
Date: 2015-06-22 21:57:36 UTC
Au reload, ls n5 a re-reloader sur eth_port_sec ....
Kernel uptime is 0 day(s), 0 hour(s), 4 minute(s), 40 second(s)
Last reset at 256954 usecs after Mon Jun 22 22:47:15 2015
Reason: Reset triggered due to HA policy of Reset
System version: 6.0(2)N2(2)
Service: eth_port_sec hap reset
Je laisse les port-channel shut et lance l'ISSU sur celui encore UP.
Date: 2015-06-22 21:35:47 UTC
# sh fex
FEX FEX FEX FEX Fex
Number Description State Model Serial
------------------------------------------------------------------------
102 FEX0102 Offline N2K-C2248TP-1GE SSI15330JJV
103 FEX0103 Offline N2K-C2248TP-1GE SSI153508ET
104 FEX0104 Offline N2K-C2248TP-1GE SSI15391ATG
105 FEX0105 Offline N2K-C2232PP-10GE SSI1541037V
106 FEX0106 Offline N2K-C2232PP-10GE SSI14160FXW
107 FEX0107 Offline N2K-C2232PP-10GE SSI153409GJ
108 FEX0108 Offline N2K-C2232PP-10GE SSI14160MT9
109 FEX0109 Offline N2K-C2232PP-10GE SSI154103G2
110 FEX0110 Offline N2K-C2232PP-10GE SSI15380LY5
111 FEX0111 Offline N2K-C2248TP-1GE SSI144506KC
112 FEX0112 Offline N2K-C2232PP-10GE SSI154309WX
113 FEX0113 Offline N2K-C2232TM-E-10GE SSI18420A01
114 FEX0114 Offline N2K-C2232TM-E-10GE SSI1845091Q
115 FEX0115 Offline N2K-C2232TM-E-10GE SSI184906NR
116 FEX0116 Offline N2K-C2232TM-E-10GE SSI185007M9
117 FEX0117 Offline N2K-C2232TM-E-10GE SSI184906NQ
je reload
# wr
[########################################] 100%
Copy complete, now saving to disk (please wait)...
# reload
WARNING: This command will reboot the system
Do you want to continue? (y/n) [n] y
Date: 2015-06-22 21:26:45 UTC
Je coupe les port-channel des FEX 2 par 2 avant de reloader.
Date: 2015-06-22 21:19:21 UTC
une nouvelle étape ... la version du microcontroller n'est pas a jour,
Software
BIOS: version 3.6.0
loader: version N/A
kickstart: version 6.0(2)N2(2)
system: version 6.0(2)N2(2)
Power Sequencer Firmware:
Module 1: version v1.0
Module 2: version v1.0
Module 3: version v2.0
Microcontroller Firmware: version v1.2.0.1
SFP uC: Module 1: v0.0.0.0 <<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<< ça devrait être en v1.0
QSFP uC: Module not detected
BIOS compile time: 05/09/2012
kickstart image file is: bootflash:///n5000-uk9-kickstart.6.0.2.N2.2.bin
kickstart compile time: 10/4/2013 12:00:00 [10/04/2013 21:10:23]
system image file is: bootflash:///n5000-uk9.6.0.2.N2.2.bin
system compile time: 10/4/2013 12:00:00 [10/04/2013 23:29:49]
Je dois reloader le chassis .... le trafic sera forwarde par le 2nd n5
Date: 2015-06-22 21:15:56 UTC
It worked !
# sh fex
FEX FEX FEX FEX Fex
Number Description State Model Serial
------------------------------------------------------------------------
100 FEX0100 Online N2K-C2248TP-1GE SSI1444081Y
101 FEX0101 Online N2K-C2248TP-1GE SSI1444081S
102 FEX0102 Online N2K-C2248TP-1GE SSI15330JJV
103 FEX0103 Online N2K-C2248TP-1GE SSI153508ET
104 FEX0104 Online N2K-C2248TP-1GE SSI15391ATG
105 FEX0105 Online N2K-C2232PP-10GE SSI1541037V
106 FEX0106 Online N2K-C2232PP-10GE SSI14160FXW
107 FEX0107 Online N2K-C2232PP-10GE SSI153409GJ
108 FEX0108 Online N2K-C2232PP-10GE SSI14160MT9
109 FEX0109 Online N2K-C2232PP-10GE SSI154103G2
110 FEX0110 Online N2K-C2232PP-10GE SSI15380LY5
111 FEX0111 Online N2K-C2248TP-1GE SSI144506KC
112 FEX0112 Online N2K-C2232PP-10GE SSI154309WX
113 FEX0113 Online N2K-C2232TM-E-10GE SSI18420A01
114 FEX0114 Online N2K-C2232TM-E-10GE SSI1845091Q
115 FEX0115 Online N2K-C2232TM-E-10GE SSI184906NR
116 FEX0116 Online N2K-C2232TM-E-10GE SSI185007M9
117 FEX0117 Online N2K-C2232TM-E-10GE SSI184906NQ
Date: 2015-06-22 21:09:50 UTC
J'ai disable les port du le mod 2 pour forcer le trafic a passer sur le 2nd switch
# sh int status module 2
--------------------------------------------------------------------------------
Port Name Status Vlan Duplex Speed Type
--------------------------------------------------------------------------------
Eth2/1 fex 100|EG disabled 1 full 10G Fabric Exte
Eth2/2 -- sfpAbsent 1 full 10G --
Eth2/3 -- sfpAbsent 1 full 10G --
Eth2/4 -- sfpAbsent 1 full 10G --
Eth2/5 -- sfpAbsent 1 full 10G --
Eth2/6 -- sfpAbsent 1 full 10G --
Eth2/7 fex 104|EG disabled 1 full 10G Fabric Exte
Eth2/8 -- sfpAbsent 1 full 10G --
Eth2/9 -- sfpAbsent 1 full 10G --
Eth2/10 fex117|EG notconnec 1 full 10G Fabric Exte
Eth2/11 -- sfpAbsent 1 full 10G --
Eth2/12 -- sfpAbsent 1 full 10G --
Eth2/13 fex113 notconnec 1 full 10G Fabric Exte
Eth2/14 -- sfpAbsent 1 full 10G --
Eth2/15 fex110|MG notconnec 1 full 10G Fabric Exte
Eth2/16 -- sfpAbsent 1 full 10G --
#
power off du module2
# conf te
Enter configuration commands, one per line. End with CNTL/Z.
(config)# poweroff module 2
(config)#
(config)#
(config)#
(config)#
(config)# sh module
Mod Ports Module-Type Model Status
--- ----- ----------------------------------- ---------------------- ------------
1 32 O2 32X10GE/Modular Universal Platfo N5K-C5548UP-SUP active *
2 16 O2 16X10GE Ethernet Module N55-M16P present(powered-off)
3 0 O2 Non L3 Daughter Card N55-DL2 ok
Mod Sw Hw World-Wide-Name(s) (WWN)
--- -------------- ------ ----------------------------------------------------
1 6.0(2)N2(2) 1.0 --
2 6.0(2)N2(2) 1.0 --
3 6.0(2)N2(2) 1.0 --
Mod MAC-Address(es) Serial-Num
--- -------------------------------------- ----------
1 547f.ee1a.8fa8 to 547f.ee1a.8fc7 FOC153255P4
2 002a.6abc.b690 to 002a.6abc.b69f FOC18161R48
3 0000.0000.0000 to 0000.0000.000f FOC15321UJM
(config)#
up du module
(config)# no poweroff module 2
(config)#
Suite a un bug en 6.0(2)N2(2) , certain port du châssis passe down et reste down.
Actuellement 3 FEX sont single homed à cause de ce bug, il n'y a pas d'impact sur la production mais il n'y a plus de redondance pour ces 3 FEXs.
Afin de préparer la mise a jour en ISSU (hitless), nous allons rebooter le module 2 du Nexus avec ses ports down.
Avec de la chance, cela suffira à re-uppe les ports, sinon il faudra reloader le châssis complètement.
Par la suite, on enchaînera par l'update en elle même vers 7.0.5.N1a
L'intervention va débuter dans qq minutes
Update(s):
Date: 2015-06-23 00:01:21 UTC
ouf !
La mise a jour est a présent terminée - 3h ...
# sh fex
FEX FEX FEX FEX Fex
Number Description State Model Serial
------------------------------------------------------------------------
100 FEX0100 Online N2K-C2248TP-1GE SSI1444081Y
101 FEX0101 Online N2K-C2248TP-1GE SSI1444081S
102 FEX0102 Online N2K-C2248TP-1GE SSI15330JJV
103 FEX0103 Online N2K-C2248TP-1GE SSI153508ET
104 FEX0104 Online N2K-C2248TP-1GE SSI15391ATG
105 FEX0105 Online N2K-C2232PP-10GE SSI1541037V
106 FEX0106 Online N2K-C2232PP-10GE SSI14160FXW
107 FEX0107 Online N2K-C2232PP-10GE SSI153409GJ
108 FEX0108 Online N2K-C2232PP-10GE SSI14160MT9
109 FEX0109 Online N2K-C2232PP-10GE SSI154103G2
110 FEX0110 Online N2K-C2232PP-10GE SSI15380LY5
111 FEX0111 Online N2K-C2248TP-1GE SSI144506KC
112 FEX0112 Online N2K-C2232PP-10GE SSI154309WX
113 FEX0113 Online N2K-C2232TM-E-10GE SSI18420A01
114 FEX0114 Online N2K-C2232TM-E-10GE SSI1845091Q
115 FEX0115 Online N2K-C2232TM-E-10GE SSI184906NR
116 FEX0116 Online N2K-C2232TM-E-10GE SSI185007M9
117 FEX0117 Online N2K-C2232TM-E-10GE SSI184906NQ
Date: 2015-06-22 23:49:31 UTC
reload power cycle done, je up les fex
# sh fex
FEX FEX FEX FEX Fex
Number Description State Model Serial
------------------------------------------------------------------------
100 FEX0100 Connected N2K-C2248TP-1GE SSI1444081Y
101 FEX0101 Online N2K-C2248TP-1GE SSI1444081S
102 FEX0102 Connected N2K-C2248TP-1GE SSI15330JJV
103 FEX0103 Connected N2K-C2248TP-1GE SSI153508ET
104 FEX0104 Connected N2K-C2248TP-1GE SSI15391ATG
105 FEX0105 Connected N2K-C2232PP-10GE SSI1541037V
106 FEX0106 Connected N2K-C2232PP-10GE SSI14160FXW
107 FEX0107 Connected N2K-C2232PP-10GE SSI153409GJ
108 FEX0108 Connected N2K-C2232PP-10GE SSI14160MT9
109 FEX0109 Connected N2K-C2232PP-10GE SSI154103G2
110 FEX0110 Connected N2K-C2232PP-10GE SSI15380LY5
111 FEX0111 Connected N2K-C2248TP-1GE SSI144506KC
112 FEX0112 Connected N2K-C2232PP-10GE SSI154309WX
113 FEX0113 Online N2K-C2232TM-E-10GE SSI18420A01
114 FEX0114 Connected N2K-C2232TM-E-10GE SSI1845091Q
115 FEX0115 Connected N2K-C2232TM-E-10GE SSI184906NR
116 FEX0116 Connected N2K-C2232TM-E-10GE SSI185007M9
117 FEX0117 Connected N2K-C2232TM-E-10GE SSI184906NQ
Date: 2015-06-22 23:27:58 UTC
all godd, j'isole a present le 1er nexus
Date: 2015-06-22 23:18:17 UTC
# sh fex
FEX FEX FEX FEX Fex
Number Description State Model Serial
------------------------------------------------------------------------
100 FEX0100 Online N2K-C2248TP-1GE SSI1444081Y
101 FEX0101 Online N2K-C2248TP-1GE SSI1444081S
102 FEX0102 Online N2K-C2248TP-1GE SSI15330JJV
103 FEX0103 Online N2K-C2248TP-1GE SSI153508ET
104 FEX0104 Online N2K-C2248TP-1GE SSI15391ATG
105 FEX0105 Connected N2K-C2232PP-10GE SSI1541037V
106 FEX0106 Connected N2K-C2232PP-10GE SSI14160FXW
107 FEX0107 Online N2K-C2232PP-10GE SSI153409GJ
108 FEX0108 Connected N2K-C2232PP-10GE SSI14160MT9
109 FEX0109 Connected N2K-C2232PP-10GE SSI154103G2
110 FEX0110 Connected N2K-C2232PP-10GE SSI15380LY5
111 FEX0111 Connected N2K-C2248TP-1GE SSI144506KC
Date: 2015-06-22 23:09:32 UTC
Reload okay, je up les fex sur ce nexus pour pouvoir reloader le 1er
Date: 2015-06-22 23:00:57 UTC
# reload power-cycle
WARNING: This command will reboot the system
Do you want to continue? (y/n) [n] y
Date: 2015-06-22 23:00:29 UTC
Performing supervisor state verification.
SUCCESS
Supervisor non-disruptive upgrade successful.
Install has been successful.
Date: 2015-06-22 22:55:47 UTC
Module 1: Refreshing compact flash and upgrading bios/loader/bootrom/power-seq.
Warning: please do not remove or power off the module at this time.
Note: Power-seq upgrade needs a power-cycle to take into effect.
Use command 'reload power-cycle'
Note: Micro-controller upgrade needs a power-cycle to take into effect.
On success of micro-controller upgrade, SWITCH OFF THE POWER to the system and then, power it up.
il faudra faire un reset hard ou reload power cycle pour maj le power sequencer
Date: 2015-06-22 22:54:51 UTC
Compatibility check is done:
Module bootable Impact Install-type Reason
------ -------- -------------- ------------ ------
1 yes non-disruptive reset
2 yes non-disruptive rolling
3 yes non-disruptive rolling
Images will be upgraded according to following table:
Module Image Running-Version New-Version Upg-Required
------ ---------------- ---------------------- ---------------------- ------------
1 system 6.0(2)N2(2) 7.0(5)N1(1a) yes
1 kickstart 6.0(2)N2(2) 7.0(5)N1(1a) yes
1 bios v3.6.0(05/09/2012) v3.6.0(05/09/2012) no
1 power-seq v1.0 v3.0 yes
1 SFP-uC v1.0.0.0 v1.0.0.0 no
2 power-seq v1.0 v1.0 no
3 power-seq v2.0 v2.0 no
1 microcontroller v1.2.0.1 v1.2.0.1 no
Do you want to continue with the installation (y/n)? [n] y
Install is in progress, please wait.
Performing runtime checks.
[####################] 100% -- SUCCESS
Notifying services about the upgrade.
[####################] 100% -- SUCCESS
Setting boot variables.
[# ] 0%
Date: 2015-06-22 22:53:27 UTC
L'install est okay sur le 1er n5
Module 117: Non-disruptive upgrading.
SUCCESS
Install has been successful.
Software
BIOS: version 3.6.0
loader: version N/A
kickstart: version 7.0(5)N1(1a)
system: version 7.0(5)N1(1a)
Je lance l'ISSU sur le 2nd (les fexs ne sont tjs pas attachés dessus, ça devrait être rapide)
# install all kickstart bootflash:n5000-uk9-kickstart.7.0.5.N1.1a.bin system bootflash:n5000-uk9.7.0.5.N1.1a.bin
Verifying image bootflash:/n5000-uk9-kickstart.7.0.5.N1.1a.bin for boot variable \"kickstart\".
[####################] 100% -- SUCCESS
Verifying image bootflash:/n5000-uk9.7.0.5.N1.1a.bin for boot variable \"system\".
[####################] 100% -- SUCCESS
Verifying image type.
[########### ] 50%
Date: 2015-06-22 22:44:18 UTC
le dernier FEX en cours
Module 115: Non-disruptive upgrading.
SUCCESS
Module 116: Non-disruptive upgrading.
SUCCESS
Module 117: Non-disruptive upgrading.
Date: 2015-06-22 22:39:05 UTC
Module 111: Non-disruptive upgrading.
SUCCESS
Module 112: Non-disruptive upgrading.
SUCCESS
Module 113: Non-disruptive upgrading.
Date: 2015-06-22 22:26:17 UTC
Pre-loading modules.
SUCCESS
Module 100: Non-disruptive upgrading.
SUCCESS
Module 101: Non-disruptive upgrading.
SUCCESS
Module 102: Non-disruptive upgrading.
SUCCESS
Module 103: Non-disruptive upgrading.
Date: 2015-06-22 22:21:47 UTC
enfin le 1er fex debute son update
Continuing with installation process, please wait.
The login will be disabled until the installation is completed.
Performing supervisor state verification.
SUCCESS
Supervisor non-disruptive upgrade successful.
Pre-loading modules.
SUCCESS
Module 100: Non-disruptive upgrading.
Date: 2015-06-22 22:07:28 UTC
yeah !
on y va
Notifying services about system upgrade.
[####################] 100% -- SUCCESS
Compatibility check is done:
Module bootable Impact Install-type Reason
------ -------- -------------- ------------ ------
1 yes non-disruptive reset
2 yes non-disruptive rolling
3 yes non-disruptive rolling
100 yes non-disruptive rolling
101 yes non-disruptive rolling
102 yes non-disruptive rolling
103 yes non-disruptive rolling
104 yes non-disruptive rolling
105 yes non-disruptive rolling
106 yes non-disruptive rolling
107 yes non-disruptive rolling
108 yes non-disruptive rolling
109 yes non-disruptive rolling
110 yes non-disruptive rolling
111 yes non-disruptive rolling
112 yes non-disruptive rolling
113 yes non-disruptive rolling
114 yes non-disruptive rolling
115 yes non-disruptive rolling
116 yes non-disruptive rolling
117 yes non-disruptive rolling
Images will be upgraded according to following table:
Module Image Running-Version New-Version Upg-Required
------ ---------------- ---------------------- ---------------------- ------------
1 system 6.0(2)N2(2) 7.0(5)N1(1a) yes
1 kickstart 6.0(2)N2(2) 7.0(5)N1(1a) yes
1 bios v3.6.0(05/09/2012) v3.6.0(05/09/2012) no
1 power-seq v1.0 v3.0 yes
1 SFP-uC v1.0.0.0 v1.0.0.0 no
2 power-seq v1.0 v1.0 no
3 power-seq v2.0 v2.0 no
100 fexth 6.0(2)N2(2) 7.0(5)N1(1a) yes
101 fexth 6.0(2)N2(2) 7.0(5)N1(1a) yes
102 fexth 6.0(2)N2(2) 7.0(5)N1(1a) yes
103 fexth 6.0(2)N2(2) 7.0(5)N1(1a) yes
104 fexth 6.0(2)N2(2) 7.0(5)N1(1a) yes
105 fexth 6.0(2)N2(2) 7.0(5)N1(1a) yes
106 fexth 6.0(2)N2(2) 7.0(5)N1(1a) yes
107 fexth 6.0(2)N2(2) 7.0(5)N1(1a) yes
108 fexth 6.0(2)N2(2) 7.0(5)N1(1a) yes
109 fexth 6.0(2)N2(2) 7.0(5)N1(1a) yes
110 fexth 6.0(2)N2(2) 7.0(5)N1(1a) yes
111 fexth 6.0(2)N2(2) 7.0(5)N1(1a) yes
112 fexth 6.0(2)N2(2) 7.0(5)N1(1a) yes
113 fexth 6.0(2)N2(2) 7.0(5)N1(1a) yes
114 fexth 6.0(2)N2(2) 7.0(5)N1(1a) yes
115 fexth 6.0(2)N2(2) 7.0(5)N1(1a) yes
116 fexth 6.0(2)N2(2) 7.0(5)N1(1a) yes
117 fexth 6.0(2)N2(2) 7.0(5)N1(1a) yes
1 microcontroller v1.2.0.1 v1.2.0.1 no
Do you want to continue with the installation (y/n)? [n] y
Install is in progress, please wait.
Performing runtime checks.
[####################] 100% -- SUCCESS
Notifying services about the upgrade.
[####################] 100% -- SUCCESS
Setting boot variables.
[# ] 0%
Date: 2015-06-22 22:05:43 UTC
encore des soucis...
# install all kickstart bootflash:n5000-uk9-kickstart.7.0.5.N1.1a.bin system bootflash:n5000-uk9.7.0.5.N1.1a.bin
Pre-upgrade check failed. Return code 0x40930062 (free space in the filesystem is below threshold).
# sh system internal flash
Mount-on 1K-blocks Used Available Use% Filesystem
/ 409600 78636 330964 20 /dev/root
/proc 0 0 0 0 proc
/post 2048 4 2044 1 none
/var 409600 78636 330964 20 none
/sys 0 0 0 0 none
/isan 1536000 732476 803524 48 none
/var/tmp 307200 3304 303896 2 none
/var/sysmgr 1024000 1024000 0 100 none <<<<<<<<<<< full 100%
/nxos/tmp 40960 36 40924 1 none
/var/sysmgr/ftp 409600 84 409516 1 none
/var/sysmgr/ftp/cores 102400 0 102400 0 none
/callhome 61440 0 61440 0 none
/dev/shm 524288 288972 235316 56 none
/volatile 153600 4 153596 1 none
/debug 20480 0 20480 0 none
/dev/mqueue 0 0 0 0 none
/debugfs 0 0 0 0 nodev
/mnt/plog 49024 1344 47680 3 /dev/mtdblock2
/mnt/cfg/0 114171 5584 102692 6 /dev/sda5
/mnt/cfg/1 113193 5584 101765 6 /dev/sda6
/var/sysmgr/startup-cfg 409600 16652 392948 5 none
/dev/pts 0 0 0 0 devpts
/mnt/pss 114203 6370 101937 6 /dev/sda4
/bootflash 1612212 620708 909604 41 /dev/sda3
/proc/bus/usb 0 0 0 0 none
/cgroup 0 0 0 0 vdccontrol
apres un coup de plugin ... on est bon
inux(debug)# df -h | grep sysmgr
none 1000M 4.0K 1000M 1% /var/sysmgr
none 400M 84K 400M 1% /var/sysmgr/ftp
none 100M 0 100M 0% /var/sysmgr/ftp/cores
none 400M 17M 384M 5% /var/sysmgr/startup-cfg
Linux(debug)# exit
exit
#
# install all kickstart bootflash:n5000-uk9-kickstart.7.0.5.N1.1a.bin system bootflash:n5000-uk9.7.0.5.N1.1a.bin
Verifying image bootflash:/n5000-uk9-kickstart.7.0.5.N1.1a.bin for boot variable \"kickstart\".
[####################] 100% -- SUCCESS
Verifying image bootflash:/n5000-uk9.7.0.5.N1.1a.bin for boot variable \"system\".
[####################] 100% -- SUCCESS
Verifying image type.
[####################] 100% -- SUCCESS
Extracting \"system\" version from image bootflash:/n5000-uk9.7.0.5.N1.1a.bin.
[####################] 100% -- SUCCESS
Extracting \"kickstart\" version from image bootflash:/n5000-uk9-kickstart.7.0.5.N1.1a.bin.
[####################] 100% -- SUCCESS
Extracting \"bios\" version from image bootflash:/n5000-uk9.7.0.5.N1.1a.bin.
[# ] 0%
. a suivre
Date: 2015-06-22 21:57:36 UTC
Au reload, ls n5 a re-reloader sur eth_port_sec ....
Kernel uptime is 0 day(s), 0 hour(s), 4 minute(s), 40 second(s)
Last reset at 256954 usecs after Mon Jun 22 22:47:15 2015
Reason: Reset triggered due to HA policy of Reset
System version: 6.0(2)N2(2)
Service: eth_port_sec hap reset
Je laisse les port-channel shut et lance l'ISSU sur celui encore UP.
Date: 2015-06-22 21:35:47 UTC
# sh fex
FEX FEX FEX FEX Fex
Number Description State Model Serial
------------------------------------------------------------------------
102 FEX0102 Offline N2K-C2248TP-1GE SSI15330JJV
103 FEX0103 Offline N2K-C2248TP-1GE SSI153508ET
104 FEX0104 Offline N2K-C2248TP-1GE SSI15391ATG
105 FEX0105 Offline N2K-C2232PP-10GE SSI1541037V
106 FEX0106 Offline N2K-C2232PP-10GE SSI14160FXW
107 FEX0107 Offline N2K-C2232PP-10GE SSI153409GJ
108 FEX0108 Offline N2K-C2232PP-10GE SSI14160MT9
109 FEX0109 Offline N2K-C2232PP-10GE SSI154103G2
110 FEX0110 Offline N2K-C2232PP-10GE SSI15380LY5
111 FEX0111 Offline N2K-C2248TP-1GE SSI144506KC
112 FEX0112 Offline N2K-C2232PP-10GE SSI154309WX
113 FEX0113 Offline N2K-C2232TM-E-10GE SSI18420A01
114 FEX0114 Offline N2K-C2232TM-E-10GE SSI1845091Q
115 FEX0115 Offline N2K-C2232TM-E-10GE SSI184906NR
116 FEX0116 Offline N2K-C2232TM-E-10GE SSI185007M9
117 FEX0117 Offline N2K-C2232TM-E-10GE SSI184906NQ
je reload
# wr
[########################################] 100%
Copy complete, now saving to disk (please wait)...
# reload
WARNING: This command will reboot the system
Do you want to continue? (y/n) [n] y
Date: 2015-06-22 21:26:45 UTC
Je coupe les port-channel des FEX 2 par 2 avant de reloader.
Date: 2015-06-22 21:19:21 UTC
une nouvelle étape ... la version du microcontroller n'est pas a jour,
Software
BIOS: version 3.6.0
loader: version N/A
kickstart: version 6.0(2)N2(2)
system: version 6.0(2)N2(2)
Power Sequencer Firmware:
Module 1: version v1.0
Module 2: version v1.0
Module 3: version v2.0
Microcontroller Firmware: version v1.2.0.1
SFP uC: Module 1: v0.0.0.0 <<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<< ça devrait être en v1.0
QSFP uC: Module not detected
BIOS compile time: 05/09/2012
kickstart image file is: bootflash:///n5000-uk9-kickstart.6.0.2.N2.2.bin
kickstart compile time: 10/4/2013 12:00:00 [10/04/2013 21:10:23]
system image file is: bootflash:///n5000-uk9.6.0.2.N2.2.bin
system compile time: 10/4/2013 12:00:00 [10/04/2013 23:29:49]
Je dois reloader le chassis .... le trafic sera forwarde par le 2nd n5
Date: 2015-06-22 21:15:56 UTC
It worked !
# sh fex
FEX FEX FEX FEX Fex
Number Description State Model Serial
------------------------------------------------------------------------
100 FEX0100 Online N2K-C2248TP-1GE SSI1444081Y
101 FEX0101 Online N2K-C2248TP-1GE SSI1444081S
102 FEX0102 Online N2K-C2248TP-1GE SSI15330JJV
103 FEX0103 Online N2K-C2248TP-1GE SSI153508ET
104 FEX0104 Online N2K-C2248TP-1GE SSI15391ATG
105 FEX0105 Online N2K-C2232PP-10GE SSI1541037V
106 FEX0106 Online N2K-C2232PP-10GE SSI14160FXW
107 FEX0107 Online N2K-C2232PP-10GE SSI153409GJ
108 FEX0108 Online N2K-C2232PP-10GE SSI14160MT9
109 FEX0109 Online N2K-C2232PP-10GE SSI154103G2
110 FEX0110 Online N2K-C2232PP-10GE SSI15380LY5
111 FEX0111 Online N2K-C2248TP-1GE SSI144506KC
112 FEX0112 Online N2K-C2232PP-10GE SSI154309WX
113 FEX0113 Online N2K-C2232TM-E-10GE SSI18420A01
114 FEX0114 Online N2K-C2232TM-E-10GE SSI1845091Q
115 FEX0115 Online N2K-C2232TM-E-10GE SSI184906NR
116 FEX0116 Online N2K-C2232TM-E-10GE SSI185007M9
117 FEX0117 Online N2K-C2232TM-E-10GE SSI184906NQ
Date: 2015-06-22 21:09:50 UTC
J'ai disable les port du le mod 2 pour forcer le trafic a passer sur le 2nd switch
# sh int status module 2
--------------------------------------------------------------------------------
Port Name Status Vlan Duplex Speed Type
--------------------------------------------------------------------------------
Eth2/1 fex 100|EG disabled 1 full 10G Fabric Exte
Eth2/2 -- sfpAbsent 1 full 10G --
Eth2/3 -- sfpAbsent 1 full 10G --
Eth2/4 -- sfpAbsent 1 full 10G --
Eth2/5 -- sfpAbsent 1 full 10G --
Eth2/6 -- sfpAbsent 1 full 10G --
Eth2/7 fex 104|EG disabled 1 full 10G Fabric Exte
Eth2/8 -- sfpAbsent 1 full 10G --
Eth2/9 -- sfpAbsent 1 full 10G --
Eth2/10 fex117|EG notconnec 1 full 10G Fabric Exte
Eth2/11 -- sfpAbsent 1 full 10G --
Eth2/12 -- sfpAbsent 1 full 10G --
Eth2/13 fex113 notconnec 1 full 10G Fabric Exte
Eth2/14 -- sfpAbsent 1 full 10G --
Eth2/15 fex110|MG notconnec 1 full 10G Fabric Exte
Eth2/16 -- sfpAbsent 1 full 10G --
#
power off du module2
# conf te
Enter configuration commands, one per line. End with CNTL/Z.
(config)# poweroff module 2
(config)#
(config)#
(config)#
(config)#
(config)# sh module
Mod Ports Module-Type Model Status
--- ----- ----------------------------------- ---------------------- ------------
1 32 O2 32X10GE/Modular Universal Platfo N5K-C5548UP-SUP active *
2 16 O2 16X10GE Ethernet Module N55-M16P present(powered-off)
3 0 O2 Non L3 Daughter Card N55-DL2 ok
Mod Sw Hw World-Wide-Name(s) (WWN)
--- -------------- ------ ----------------------------------------------------
1 6.0(2)N2(2) 1.0 --
2 6.0(2)N2(2) 1.0 --
3 6.0(2)N2(2) 1.0 --
Mod MAC-Address(es) Serial-Num
--- -------------------------------------- ----------
1 547f.ee1a.8fa8 to 547f.ee1a.8fc7 FOC153255P4
2 002a.6abc.b690 to 002a.6abc.b69f FOC18161R48
3 0000.0000.0000 to 0000.0000.000f FOC15321UJM
(config)#
up du module
(config)# no poweroff module 2
(config)#
This scheduled maintenance affected: Infrastructure || RBX (RBX4).