Cisco Identity Services Engine Upgrade Failure 2.0 to 2.3
Experience with upgrading a two-node Cisco Identity Services Engine (ISE) deployment from version 2.0 to 2.3.
Current Cisco ISE deployment:
!!!! Hardware VMware 6.5 Node 1: Primary Admin, Secondary Monitoring, PSN 4 CPU 24G Memory Node 2: Secondary Admin, Primary Monitoring, PSN 4 CPU 24G Memory !!!! Show version Cisco Application Deployment Engine OS Release: 2.3 ADE-OS Build Version: 2.3.0.187 ADE-OS System Architecture: x86_64 Copyright (c) 2005-2014 by Cisco Systems, Inc. All rights reserved. Hostname: tnetise01 Version information of installed applications --------------------------------------------- Cisco Identity Services Engine --------------------------------------------- Version : 2.0.0.306 Build Date : Thu Oct 8 18:55:23 2015 Install Date : Mon Oct 16 13:14:57 2017 Cisco Identity Services Engine Patch --------------------------------------------- Version : 5 Install Date : Tue Oct 17 14:42:24 2017
Upgrading to version: 2.3
Running the Upgrade Readiness Tool (URT) first. URT is run on the ISE Secondary Admin Node (SAN). It performs a mock database upgrade (clones the actual and upgrades it) which validates how the actual upgrade will go when initiated. A more recent version of URT tool was available which wasn’t noticed at that time. The recommendation is to run the latest version of URT when available.
tnetise02/admin# application install ise-urtbundle-2.3.0.298-1.1.0.SPA.x86_64.tar.gz local Save the current ADE-OS running configuration? (yes/no) [yes] ? yes Generating configuration... Saved the ADE-OS running configuration to startup successfully Getting bundle to local machine... Unbundling Application Package... Verifying Application Signature... Initiating Application Install... ########################################### # Installing Upgrade Readiness Tool (URT) # ########################################### Checking ISE version compatibility - Successful Checking ISE persona - Successful Along with Administration, other services (MNT,SESSION) are enabled on this node. Installing and running URT might consume additional resources. Do you want to proceed with installing and running URT now (y/n):y Checking if URT is recent(<45 days old) - Note: URT is 244 days old and its version is 1.1.0. There might be a recent URT bundle on CCO, please verify on CCO Do you want to proceed with this version which is 244 days old (y/n):y Proceeding with this version of URT itself Installing URT bundle - Successful ######################################## # Running Upgrade Readiness Tool (URT) # ######################################## This tool will perform following tasks: 1. Pre-requisite checks 2. Clone config database 3. Copy upgrade files 4. Data upgrade on cloned database 5. Time estimate for upgrade Pre-requisite checks ==================== Disk Space sanity check - Successful NTP sanity - Successful Appliance/VM compatibility - Successful Trust Cert Validation - Successful System Cert Validation - Successful 5 out of 5 pre-requisite checks passed Clone config database ===================== [########################################] 100% Successful Copy upgrade files ================== - N/A Data upgrade on cloned database =============================== Modifying upgrade scripts to run on cloned database - Successful Running schema upgrade on cloned database - Running db sanity to check and fix if any index corruption - Auto Upgrading Schema for UPS Model - Upgrading Schema completed for UPS Model - Successful Running sanity after schema upgrade on cloned database - Successful Running data upgrade on cloned database - Data upgrade step 1/100, AuthzUpgradeService(2.0.0.308)... ..Done in 169 seconds. - Data upgrade step 2/100, NSFUpgradeService(2.1.0.102)... Done in 1 seconds. - Data upgrade step 3/100, UPSUpgradeHandler(2.1.0.105)... ..Done in 131 seconds. - Data upgrade step 4/100, UPSUpgradeHandler(2.1.0.107)... Done in 2 seconds. - Data upgrade step 5/100, NSFUpgradeService(2.1.0.109)... Done in 0 seconds. - Data upgrade step 6/100, NSFUpgradeService(2.1.0.126)... Done in 0 seconds. - Data upgrade step 7/100, NetworkAccessUpgrade(2.1.0.127)... Done in 1 seconds. - Data upgrade step 8/100, ProfilerUpgradeService(2.1.0.134)... Done in 0 seconds. - Data upgrade step 9/100, ProfilerUpgradeService(2.1.0.139)... Done in 0 seconds. - Data upgrade step 10/100, ProfilerUpgradeService(2.1.0.166)... ................- Failed Final cleanup before exiting... Collecting log files ... - Encrypting logs bundle... Please enter encryption password: Please enter encryption password again to verify: Encrypted URT logs(urt_logs.tar.gpg) are available in localdisk. Please reach out to Cisco TAC to debug % Post-install step failed. Please check the logs for more details. tnetise02/admin#
Researching online brought to our attention that this is a known bug CSCvh13873. A hotfix patch is available that can be obtained by opening a case with Cisco TAC. The patch was obtained and applied.
tnetise02/admin# application install ise-apply-CSCvh13873_2.0_2.1_2.2_2.3_common_11-SPA.tar.gz local Save the current ADE-OS running configuration? (yes/no) [yes] ? Generating configuration... Saved the ADE-OS running configuration to startup successfully Getting bundle to local machine... Unbundling Application Package... Verifying Application Signature... Initiating Application Install... Checking if CSCvh13873_2.0_2.1_2.2_2.3_common_1 is already applied - Successful Checking ISE version compatibility - Successful Applying hot patch CSCvh13873_2.0_2.1_2.2_2.3_common_1 Please take a config backup before proceeding with below steps. Do you want to proceed (y/n):y - Stopping monit and ise application server Stopping ISE Application Server... - Running hotpatch wrapper script to delete policies, checks and rules Deleting policies - 1st set Deleting policies - 2nd set Deleting checks and rules - Starting ise application server and monit Starting ISE Application Server... Hot patch applied successfully job 1 at 2018-05-15 14:35 Application successfully installed tnetise02/admin#
The URT ran successfully this time. Another good thing about this tool is it gives an estimate of the actual upgrade times on each node. This is useful in planning the actual upgrade change.
tnetise02/admin# application install ise-urtbundle-2.3.0.298-1.1.0.SPA.x86_64.tar.gz local Save the current ADE-OS running configuration? (yes/no) [yes] ? Generating configuration... Saved the ADE-OS running configuration to startup successfully Getting bundle to local machine... Unbundling Application Package... Verifying Application Signature... Initiating Application Install... ########################################### # Installing Upgrade Readiness Tool (URT) # ########################################### Checking ISE version compatibility - Successful Checking ISE persona - Successful Along with Administration, other services (MNT,SESSION) are enabled on this node. Installing and running URT might consume additional resources. Do you want to proceed with installing and running URT now (y/n):y Checking if URT is recent(<45 days old) - Note: URT is 245 days old and its version is 1.1.0. There might be a recent URT bundle on CCO, please verify on CCO Do you want to proceed with this version which is 245 days old (y/n):y Proceeding with this version of URT itself Installing URT bundle - Successful ######################################## # Running Upgrade Readiness Tool (URT) # ######################################## This tool will perform following tasks: 1. Pre-requisite checks 2. Clone config database 3. Copy upgrade files 4. Data upgrade on cloned database 5. Time estimate for upgrade Pre-requisite checks ==================== Disk Space sanity check - Successful NTP sanity - Successful Appliance/VM compatibility - Successful Trust Cert Validation - Successful System Cert Validation - Successful 5 out of 5 pre-requisite checks passed Clone config database ===================== [##--------------------------------------] 5% Validating connection to ISE data [####------------------------------------] 10% Validating available disk space [######----------------------------------] 15% Extracting base database files [##########------------------------------] 25% Cloning database [####################--------------------] 50% Exporting data from ISE database [##############################----------] 75% Importing data into cloned datab [########################################] 100% Successful Copy upgrade files ================== - N/A Data upgrade on cloned database =============================== Modifying upgrade scripts to run on cloned database - Successful Running schema upgrade on cloned database - Running db sanity to check and fix if any index corruption - Auto Upgrading Schema for UPS Model - Upgrading Schema completed for UPS Model - Successful Running sanity after schema upgrade on cloned database - Successful Running data upgrade on cloned database - Data upgrade step 1/100, AuthzUpgradeService(2.0.0.308)... Done in 48 seconds. - Data upgrade step 2/100, NSFUpgradeService(2.1.0.102)... Done in 1 seconds. - Data upgrade step 3/100, UPSUpgradeHandler(2.1.0.105)... .Done in 63 seconds. - Data upgrade step 4/100, UPSUpgradeHandler(2.1.0.107)... Done in 1 seconds. - Data upgrade step 5/100, NSFUpgradeService(2.1.0.109)... Done in 0 seconds. - Data upgrade step 6/100, NSFUpgradeService(2.1.0.126)... Done in 0 seconds. - Data upgrade step 7/100, NetworkAccessUpgrade(2.1.0.127)... Done in 1 seconds. - Data upgrade step 8/100, ProfilerUpgradeService(2.1.0.134)... Done in 0 seconds. - Data upgrade step 9/100, ProfilerUpgradeService(2.1.0.139)... Done in 0 seconds. - Data upgrade step 10/100, ProfilerUpgradeService(2.1.0.166)... ....Done in 294 seconds. - Data upgrade step 11/100, NSFUpgradeService(2.1.0.168)... Done in 0 seconds. - Data upgrade step 12/100, AlarmsUpgradeHandler(2.1.0.169)... Done in 2 seconds. - Data upgrade step 13/100, RegisterPostureTypes(2.1.0.180)... Done in 15 seconds. - Data upgrade step 14/100, RegisterPostureTypes(2.1.0.189)... Done in 0 seconds. - Data upgrade step 15/100, UPSUpgradeHandler(2.1.0.194)... Done in 0 seconds. - Data upgrade step 16/100, TrustsecWorkflowRegistration(2.1.0.203)... Done in 0 seconds. - Data upgrade step 17/100, NSFUpgradeService(2.1.0.205)... Done in 0 seconds. - Data upgrade step 18/100, NetworkAccessUpgrade(2.1.0.207)... Done in 0 seconds. - Data upgrade step 19/100, NSFUpgradeService(2.1.0.212)... Done in 0 seconds. - Data upgrade step 20/100, NetworkAccessUpgrade(2.1.0.241)... Done in 0 seconds. - Data upgrade step 21/100, NetworkAccessUpgrade(2.1.0.242)... Done in 1 seconds. - Data upgrade step 22/100, UPSUpgradeHandler(2.1.0.244)... Done in 0 seconds. - Data upgrade step 23/100, ProfilerUpgradeService(2.1.0.248)... Done in 0 seconds. - Data upgrade step 24/100, NetworkAccessUpgrade(2.1.0.254)... Done in 0 seconds. - Data upgrade step 25/100, UPSUpgradeHandler(2.1.0.255)... Done in 17 seconds. - Data upgrade step 26/100, MDMPartnerUpgradeService(2.1.0.257)... Done in 0 seconds. - Data upgrade step 27/100, NetworkAccessUpgrade(2.1.0.258)... Done in 0 seconds. - Data upgrade step 28/100, ProfilerUpgradeService(2.1.0.258)... Done in 18 seconds. - Data upgrade step 29/100, MDMPartnerUpgradeService(2.1.0.258)... Done in 0 seconds. - Data upgrade step 30/100, UPSUpgradeHandler(2.1.0.279)... Done in 1 seconds. - Data upgrade step 31/100, NSFUpgradeService(2.1.0.282)... Done in 0 seconds. - Data upgrade step 32/100, NetworkAccessUpgrade(2.1.0.288)... Done in 0 seconds. - Data upgrade step 33/100, NetworkAccessUpgrade(2.1.0.295)... Done in 1 seconds. - Data upgrade step 34/100, CertMgmtUpgradeService(2.1.0.296)... Done in 0 seconds. - Data upgrade step 35/100, NetworkAccessUpgrade(2.1.0.299)... Done in 0 seconds. - Data upgrade step 36/100, NetworkAccessUpgrade(2.1.0.322)... Done in 0 seconds. - Data upgrade step 37/100, NetworkAccessUpgrade(2.1.0.330)... Done in 0 seconds. - Data upgrade step 38/100, NSFUpgradeService(2.1.0.353)... Done in 0 seconds. - Data upgrade step 39/100, ProfilerUpgradeService(2.1.0.354)... Done in 0 seconds. - Data upgrade step 40/100, NSFUpgradeService(2.1.0.427)... Done in 0 seconds. - Data upgrade step 41/100, NSFUpgradeService(2.1.101.145)... Done in 0 seconds. - Data upgrade step 42/100, ProfilerUpgradeService(2.1.101.145)... Done in 0 seconds. - Data upgrade step 43/100, UPSUpgradeHandler(2.1.101.188)... Done in 0 seconds. - Data upgrade step 44/100, NetworkAccessUpgrade(2.2.0.007)... Done in 0 seconds. - Data upgrade step 45/100, UPSUpgradeHandler(2.2.0.118)... Done in 3 seconds. - Data upgrade step 46/100, GuestAccessUpgradeService(2.2.0.124)... Done in 36 seconds. - Data upgrade step 47/100, NSFUpgradeService(2.2.0.135)... Done in 0 seconds. - Data upgrade step 48/100, NSFUpgradeService(2.2.0.136)... Done in 0 seconds. - Data upgrade step 49/100, NetworkAccessUpgrade(2.2.0.137)... Done in 0 seconds. - Data upgrade step 50/100, NetworkAccessUpgrade(2.2.0.143)... Done in 10 seconds. - Data upgrade step 51/100, NSFUpgradeService(2.2.0.145)... Done in 1 seconds. - Data upgrade step 52/100, NSFUpgradeService(2.2.0.146)... Done in 1 seconds. - Data upgrade step 53/100, NetworkAccessUpgrade(2.2.0.155)... Done in 0 seconds. - Data upgrade step 54/100, CdaRegistration(2.2.0.156)... Done in 4 seconds. - Data upgrade step 55/100, NetworkAccessUpgrade(2.2.0.161)... Done in 0 seconds. - Data upgrade step 56/100, UPSUpgradeHandler(2.2.0.166)... Done in 0 seconds. - Data upgrade step 57/100, NetworkAccessUpgrade(2.2.0.169)... Done in 0 seconds. - Data upgrade step 58/100, UPSUpgradeHandler(2.2.0.169)... Done in 0 seconds. - Data upgrade step 59/100, NetworkAccessUpgrade(2.2.0.180)... Done in 0 seconds. - Data upgrade step 60/100, CertMgmtUpgradeService(2.2.0.200)... Done in 0 seconds. - Data upgrade step 61/100, NetworkAccessUpgrade(2.2.0.208)... Done in 0 seconds. - Data upgrade step 62/100, RegisterPostureTypes(2.2.0.218)... Done in 1 seconds. - Data upgrade step 63/100, NetworkAccessUpgrade(2.2.0.218)... Done in 2 seconds. - Data upgrade step 64/100, NetworkAccessUpgrade(2.2.0.222)... Done in 0 seconds. - Data upgrade step 65/100, NetworkAccessUpgrade(2.2.0.223)... Done in 0 seconds. - Data upgrade step 66/100, NetworkAccessUpgrade(2.2.0.224)... Done in 2 seconds. - Data upgrade step 67/100, SyslogTemplatesRegistration(2.2.0.224)... Done in 0 seconds. - Data upgrade step 68/100, ReportUpgradeHandler(2.2.0.242)... Done in 0 seconds. - Data upgrade step 69/100, IRFUpgradeService(2.2.0.242)... Done in 0 seconds. - Data upgrade step 70/100, LocalHostNADRegistrationService(2.2.0.261)... Done in 0 seconds. - Data upgrade step 71/100, NetworkAccessUpgrade(2.2.0.300)... Done in 0 seconds. - Data upgrade step 72/100, CertMgmtUpgradeService(2.2.0.300)... Done in 0 seconds. - Data upgrade step 73/100, NSFUpgradeService(2.2.0.323)... Done in 0 seconds. - Data upgrade step 74/100, NetworkAccessUpgrade(2.2.0.330)... Done in 0 seconds. - Data upgrade step 75/100, NSFUpgradeService(2.2.0.340)... Done in 0 seconds. - Data upgrade step 76/100, NetworkAccessUpgrade(2.2.0.340)... Done in 0 seconds. - Data upgrade step 77/100, NetworkAccessUpgrade(2.2.0.342)... Done in 0 seconds. - Data upgrade step 78/100, AuthzUpgradeService(2.2.0.344)... Done in 0 seconds. - Data upgrade step 79/100, RegisterPostureTypes(2.2.0.350)... Done in 0 seconds. - Data upgrade step 80/100, DictionaryUpgradeRegistration(2.2.0.374)... Done in 37 seconds. - Data upgrade step 81/100, UPSUpgradeHandler(2.2.0.403)... Done in 0 seconds. - Data upgrade step 82/100, DictionaryUpgradeRegistration(2.2.0.410)... Done in 0 seconds. - Data upgrade step 83/100, UPSUpgradeHandler(2.3.0.100)... Done in 36 seconds. - Data upgrade step 84/100, UPSUpgradeHandler(2.3.0.110)... Done in 1 seconds. - Data upgrade step 85/100, NetworkAccessUpgrade(2.3.0.145)... Done in 0 seconds. - Data upgrade step 86/100, NodeGroupUpgradeService(2.3.0.155)... Done in 1 seconds. - Data upgrade step 87/100, IRFUpgradeService(2.3.0.155)... Done in 0 seconds. - Data upgrade step 88/100, UPSUpgradeHandler(2.3.0.158)... Done in 0 seconds. - Data upgrade step 89/100, NetworkAccessUpgrade(2.3.0.178)... Done in 0 seconds. - Data upgrade step 90/100, NetworkAccessUpgrade(2.3.0.182)... Done in 0 seconds. - Data upgrade step 91/100, RegisterPostureTypes(2.3.0.189)... Done in 0 seconds. - Data upgrade step 92/100, CertMgmtUpgradeService(2.3.0.194)... Done in 2 seconds. - Data upgrade step 93/100, UPSUpgradeHandler(2.3.0.201)... Done in 0 seconds. - Data upgrade step 94/100, ProfilerUpgradeService(2.3.0.232)... Done in 0 seconds. - Data upgrade step 95/100, GuestAccessUpgradeService(2.3.0.243)... Done in 0 seconds. - Data upgrade step 96/100, NSFUpgradeService(2.3.0.249)... Done in 0 seconds. - Data upgrade step 97/100, NSFUpgradeService(2.3.0.273)... Done in 0 seconds. - Data upgrade step 98/100, NSFUpgradeService(2.3.0.298)... Done in 0 seconds. - Data upgrade step 99/100, ProfilerUpgradeService(2.3.0.298)... Done in 2 seconds. - Data upgrade step 100/100, GuestAccessUpgradeService(2.3.0.298)... Done in 14 seconds. - Successful Running data upgrade for node specific data on cloned database - Successful Time estimate for upgrade ========================= (Estimates are calculated based on size of config and mnt data only. Network latency between PAN and other nodes is not considered in calculating estimates) Estimated time for each node (in mins): MNT data is 13 GB, purging this data can reduce upgrade time tnetise02(SECONDARY PAP,MNT,PDP):179 MNT data is 13 GB, purging this data can reduce upgrade time tnetise01(PRIMARY PAP,MNT,PDP):167 Final cleanup before exiting... Application successfully installed tnetise02/admin#
Now, it is time for the actual upgrade. Following the Cisco Upgrade Guide for 2.3 available HERE, the ISE SAN node was upgraded first followed by the ISE PAN node.
tnetise02/admin# application upgrade prepare ise-upgradebundle-2.0.x-to-2.3.0.298.SPA.x86_64.tar.gz tnetbackup Getting bundle to local machine... Unbundling Application Package... Verifying Application Signature... Application upgrade preparation successful tnetise02/admin# show clock Thu May 17 13:43:18 AEST 2018 tnetise02/admin# application upgrade proceed Initiating Application Upgrade... % Warning: Do not use Ctrl-C or close this terminal window until upgrade completes. -Checking VM for minimum hardware requirements STEP 1: Stopping ISE application... STEP 2: Verifying files in bundle... -Internal hash verification passed for bundle STEP 3: Validating data before upgrade... STEP 4: De-registering node from current deployment... STEP 5: Taking backup of the configuration data... STEP 6: Running ISE configuration database schema upgrade... - Running db sanity to check and fix if any index corruption - Auto Upgrading Schema for UPS Model - Upgrading Schema completed for UPS Model ISE database schema upgrade completed. STEP 7: Running ISE configuration data upgrade... - Data upgrade step 1/100, AuthzUpgradeService(2.0.0.308)... .Done in 73 seconds. - Data upgrade step 2/100, NSFUpgradeService(2.1.0.102)... Done in 0 seconds. - Data upgrade step 3/100, UPSUpgradeHandler(2.1.0.105)... .Done in 107 seconds. - Data upgrade step 4/100, UPSUpgradeHandler(2.1.0.107)... Done in 3 seconds. - Data upgrade step 5/100, NSFUpgradeService(2.1.0.109)... Done in 0 seconds. - Data upgrade step 6/100, NSFUpgradeService(2.1.0.126)... Done in 1 seconds. - Data upgrade step 7/100, NetworkAccessUpgrade(2.1.0.127)... Done in 1 seconds. - Data upgrade step 8/100, ProfilerUpgradeService(2.1.0.134)... Done in 2 seconds. - Data upgrade step 9/100, ProfilerUpgradeService(2.1.0.139)... Done in 1 seconds. - Data upgrade step 10/100, ProfilerUpgradeService(2.1.0.166)... .......Done in 426 seconds. - Data upgrade step 11/100, NSFUpgradeService(2.1.0.168)... Done in 1 seconds. - Data upgrade step 12/100, AlarmsUpgradeHandler(2.1.0.169)... Done in 6 seconds. - Data upgrade step 13/100, RegisterPostureTypes(2.1.0.180)... Done in 33 seconds. - Data upgrade step 14/100, RegisterPostureTypes(2.1.0.189)... Done in 0 seconds. - Data upgrade step 15/100, UPSUpgradeHandler(2.1.0.194)... Done in 0 seconds. - Data upgrade step 16/100, TrustsecWorkflowRegistration(2.1.0.203)... Done in 0 seconds. - Data upgrade step 17/100, NSFUpgradeService(2.1.0.205)... Done in 0 seconds. - Data upgrade step 18/100, NetworkAccessUpgrade(2.1.0.207)... Done in 0 seconds. - Data upgrade step 19/100, NSFUpgradeService(2.1.0.212)... Done in 0 seconds. - Data upgrade step 20/100, NetworkAccessUpgrade(2.1.0.241)... Done in 1 seconds. - Data upgrade step 21/100, NetworkAccessUpgrade(2.1.0.242)... Done in 4 seconds. - Data upgrade step 22/100, UPSUpgradeHandler(2.1.0.244)... Done in 0 seconds. - Data upgrade step 23/100, ProfilerUpgradeService(2.1.0.248)... Done in 0 seconds. - Data upgrade step 24/100, NetworkAccessUpgrade(2.1.0.254)... Done in 0 seconds. - Data upgrade step 25/100, UPSUpgradeHandler(2.1.0.255)... Done in 20 seconds. - Data upgrade step 26/100, MDMPartnerUpgradeService(2.1.0.257)... Done in 0 seconds. - Data upgrade step 27/100, NetworkAccessUpgrade(2.1.0.258)... Done in 0 seconds. - Data upgrade step 28/100, ProfilerUpgradeService(2.1.0.258)... Done in 49 seconds. - Data upgrade step 29/100, MDMPartnerUpgradeService(2.1.0.258)... Done in 2 seconds. - Data upgrade step 30/100, UPSUpgradeHandler(2.1.0.279)... Done in 2 seconds. - Data upgrade step 31/100, NSFUpgradeService(2.1.0.282)... Done in 0 seconds. - Data upgrade step 32/100, NetworkAccessUpgrade(2.1.0.288)... Done in 0 seconds. - Data upgrade step 33/100, NetworkAccessUpgrade(2.1.0.295)... Done in 2 seconds. - Data upgrade step 34/100, CertMgmtUpgradeService(2.1.0.296)... Done in 0 seconds. - Data upgrade step 35/100, NetworkAccessUpgrade(2.1.0.299)... Done in 0 seconds. - Data upgrade step 36/100, NetworkAccessUpgrade(2.1.0.322)... Done in 0 seconds. - Data upgrade step 37/100, NetworkAccessUpgrade(2.1.0.330)... Done in 0 seconds. - Data upgrade step 38/100, NSFUpgradeService(2.1.0.353)... Done in 1 seconds. - Data upgrade step 39/100, ProfilerUpgradeService(2.1.0.354)... Done in 0 seconds. - Data upgrade step 40/100, NSFUpgradeService(2.1.0.427)... Done in 0 seconds. - Data upgrade step 41/100, NSFUpgradeService(2.1.101.145)... Done in 0 seconds. - Data upgrade step 42/100, ProfilerUpgradeService(2.1.101.145)... Done in 0 seconds. - Data upgrade step 43/100, UPSUpgradeHandler(2.1.101.188)... Done in 2 seconds. - Data upgrade step 44/100, NetworkAccessUpgrade(2.2.0.007)... Done in 2 seconds. - Data upgrade step 45/100, UPSUpgradeHandler(2.2.0.118)... Done in 9 seconds. - Data upgrade step 46/100, GuestAccessUpgradeService(2.2.0.124)... .Done in 77 seconds. - Data upgrade step 47/100, NSFUpgradeService(2.2.0.135)... Done in 0 seconds. - Data upgrade step 48/100, NSFUpgradeService(2.2.0.136)... Done in 0 seconds. - Data upgrade step 49/100, NetworkAccessUpgrade(2.2.0.137)... Done in 0 seconds. - Data upgrade step 50/100, NetworkAccessUpgrade(2.2.0.143)... Done in 17 seconds. - Data upgrade step 51/100, NSFUpgradeService(2.2.0.145)... Done in 1 seconds. - Data upgrade step 52/100, NSFUpgradeService(2.2.0.146)... Done in 2 seconds. - Data upgrade step 53/100, NetworkAccessUpgrade(2.2.0.155)... Done in 0 seconds. - Data upgrade step 54/100, CdaRegistration(2.2.0.156)... Done in 6 seconds. - Data upgrade step 55/100, NetworkAccessUpgrade(2.2.0.161)... Done in 0 seconds. - Data upgrade step 56/100, UPSUpgradeHandler(2.2.0.166)... Done in 0 seconds. - Data upgrade step 57/100, NetworkAccessUpgrade(2.2.0.169)... Done in 1 seconds. - Data upgrade step 58/100, UPSUpgradeHandler(2.2.0.169)... Done in 0 seconds. - Data upgrade step 59/100, NetworkAccessUpgrade(2.2.0.180)... Done in 0 seconds. - Data upgrade step 60/100, CertMgmtUpgradeService(2.2.0.200)... Done in 0 seconds. - Data upgrade step 61/100, NetworkAccessUpgrade(2.2.0.208)... Done in 0 seconds. - Data upgrade step 62/100, RegisterPostureTypes(2.2.0.218)... Done in 1 seconds. - Data upgrade step 63/100, NetworkAccessUpgrade(2.2.0.218)... Done in 4 seconds. - Data upgrade step 64/100, NetworkAccessUpgrade(2.2.0.222)... Done in 0 seconds. - Data upgrade step 65/100, NetworkAccessUpgrade(2.2.0.223)... Done in 0 seconds. - Data upgrade step 66/100, NetworkAccessUpgrade(2.2.0.224)... Done in 2 seconds. - Data upgrade step 67/100, SyslogTemplatesRegistration(2.2.0.224)... Done in 0 seconds. - Data upgrade step 68/100, ReportUpgradeHandler(2.2.0.242)... Done in 0 seconds. - Data upgrade step 69/100, IRFUpgradeService(2.2.0.242)... Done in 0 seconds. - Data upgrade step 70/100, LocalHostNADRegistrationService(2.2.0.261)... Done in 0 seconds. - Data upgrade step 71/100, NetworkAccessUpgrade(2.2.0.300)... Done in 0 seconds. - Data upgrade step 72/100, CertMgmtUpgradeService(2.2.0.300)... Done in 2 seconds. - Data upgrade step 73/100, NSFUpgradeService(2.2.0.323)... Done in 0 seconds. - Data upgrade step 74/100, NetworkAccessUpgrade(2.2.0.330)... Done in 1 seconds. - Data upgrade step 75/100, NSFUpgradeService(2.2.0.340)... Done in 0 seconds. - Data upgrade step 76/100, NetworkAccessUpgrade(2.2.0.340)... Done in 0 seconds. - Data upgrade step 77/100, NetworkAccessUpgrade(2.2.0.342)... Done in 0 seconds. - Data upgrade step 78/100, AuthzUpgradeService(2.2.0.344)... Done in 0 seconds. - Data upgrade step 79/100, RegisterPostureTypes(2.2.0.350)... Done in 0 seconds. - Data upgrade step 80/100, DictionaryUpgradeRegistration(2.2.0.374)... Done in 44 seconds. - Data upgrade step 81/100, UPSUpgradeHandler(2.2.0.403)... Done in 0 seconds. - Data upgrade step 82/100, DictionaryUpgradeRegistration(2.2.0.410)... Done in 0 seconds. - Data upgrade step 83/100, UPSUpgradeHandler(2.3.0.100)... .Done in 64 seconds. - Data upgrade step 84/100, UPSUpgradeHandler(2.3.0.110)... Done in 2 seconds. - Data upgrade step 85/100, NetworkAccessUpgrade(2.3.0.145)... Done in 0 seconds. - Data upgrade step 86/100, NodeGroupUpgradeService(2.3.0.155)... Done in 3 seconds. - Data upgrade step 87/100, IRFUpgradeService(2.3.0.155)... Done in 0 seconds. - Data upgrade step 88/100, UPSUpgradeHandler(2.3.0.158)... Done in 0 seconds. - Data upgrade step 89/100, NetworkAccessUpgrade(2.3.0.178)... Done in 0 seconds. - Data upgrade step 90/100, NetworkAccessUpgrade(2.3.0.182)... Done in 0 seconds. - Data upgrade step 91/100, RegisterPostureTypes(2.3.0.189)... Done in 0 seconds. - Data upgrade step 92/100, CertMgmtUpgradeService(2.3.0.194)... Done in 7 seconds. - Data upgrade step 93/100, UPSUpgradeHandler(2.3.0.201)... Done in 0 seconds. - Data upgrade step 94/100, ProfilerUpgradeService(2.3.0.232)... ...Done in 185 seconds. - Data upgrade step 95/100, GuestAccessUpgradeService(2.3.0.243)... Done in 0 seconds. - Data upgrade step 96/100, NSFUpgradeService(2.3.0.249)... Done in 0 seconds. - Data upgrade step 97/100, NSFUpgradeService(2.3.0.273)... Done in 0 seconds. - Data upgrade step 98/100, NSFUpgradeService(2.3.0.298)... Done in 0 seconds. - Data upgrade step 99/100, ProfilerUpgradeService(2.3.0.298)... Done in 6 seconds. - Data upgrade step 100/100, GuestAccessUpgradeService(2.3.0.298)... Done in 21 seconds. STEP 8: Running ISE configuration data upgrade for node specific data... STEP 9: Making this node PRIMARY of the new deployment. When other nodes are upgraded it will be added to this deployment. STEP 10: Running ISE M&T database upgrade... Stopping ISE Monitoring & Troubleshooting Log Processor... ISE database M&T schema upgrade completed. % Warning: Some warnings encountered during MNT sanity check % NOTICE: The appliance will reboot twice to upgrade software and ADE-OS. During this time progress of the upgrade is visible on console. It could take up to 30 minutes for this to complete. Rebooting to do Identity Service Engine upgrade... =~=~=~=~=~=~=~=~=~=~=~= PuTTY log 2018.05.17 15:30:43 =~=~=~=~=~=~=~=~=~=~=~= tnetise02/admin# show ver Cisco Application Deployment Engine OS Release: 3.0 ADE-OS Build Version: 3.0.3.030 ADE-OS System Architecture: x86_64 Copyright (c) 2005-2014 by Cisco Systems, Inc. All rights reserved. Hostname: tnetise02 Version information of installed applications --------------------------------------------- Cisco Identity Services Engine --------------------------------------------- Version : 2.3.0.298 Build Date : Wed Jul 26 03:02:37 2017 Install Date : Thu May 17 15:42:05 2018 tnetise02/admin# show application status ise ISE PROCESS NAME STATE PROCESS ID -------------------------------------------------------------------- Database Listener running 25639 Database Server running 66 PROCESSES Application Server running 30690 Profiler Database running 27040 ISE Indexing Engine running 32539 AD Connector running 21825 M&T Session Database running 26942 M&T Log Collector running 30823 M&T Log Processor running 30736 Certificate Authority Service disabled EST Service running 14094 SXP Engine Service disabled Docker Daemon running 911 TC-NAC Service disabled Wifi Setup Helper Container disabled pxGrid Infrastructure Service disabled pxGrid Publisher Subscriber Service disabled pxGrid Connection Manager disabled pxGrid Controller disabled PassiveID WMI Service disabled PassiveID Syslog Service disabled PassiveID API Service disabled PassiveID Agent Service disabled PassiveID Endpoint Service disabled PassiveID SPAN Service disabled DHCP Server (dhcpd) disabled DNS Server (named) disabled tnetise02/admin#
The old ISE PAN was upgraded in a similar manner. Which came up as a new ISE SAN node.
Finally, the PAN role was flipped to make Node 1 PAN and Node 2 SAN, as they were before the upgrade.
Share this:
Tagged with:
Identity Services Engine | Networking | Software Upgrade