LINUX OS Upgrade guidelines

Products

• CA Release Automation - Release Operations Center (Nolio)

Issue/Introduction

Hi Team,

 I want to check if there are any guidelines to be followed w.r.t. upgrading underlying OS hosting Release Automation Application.

Environment

Release : 6.6, 6.7

Component : RAROC, RACORE

Resolution

Linux OS (RHEL) upgrade guidelines

1: Is any specific RPM's of RHEL required from RA perspective? 

    No, there is no explicit dependency, RA has on any specific RHEL RPM's

2: Do any specific instruction w.r.t. to application, while doing OS upgrade?

- The most general guidelines and practice is, to stop the applications before, OS level upgrade to avoid any soft locks on resources.

- The another check point is to make sure that firewall state and permission over install directory and user running application remain same, to what it was pre-upgrade of OS

 

Application Upgrade guidelines

1. The NAC and NES should be on same version to work seamless. 
2. It is always recommended to upgrade NAC and NES to same version in same maintenance window to avoid any issue originating from version incompatibility. 
3. The NAC of lower version and NES of higher version is strictly prohibited.
4. We restrain use of, NAC 6.6/6.7 with unsupported NES version example 6.4 as unsupported/EOS version are more supported, therefore these scenario are never QA'd.
5. In any environment the number of agents are huge, compared to number of NAC and NES. Our recommendation to mitigate risk as below
   • Firstly, upgrade OS & RA version on NAC/s and NES/s servers in one shot. Please refer to prevalent practices section below for more details.
   • The agents upgrades can be delayed if time of maintenance window doesn't permit and can be upgraded in later maintenance window/s.
6. During a maintenance window when upgrade is performed the system should be down as it will not be in a position to process any request. Please avoid scenario of submitting deployments/process execution when NAC, NES/s or Agent/s are undergoing upgrade as it will result in unexpected failures.
7. The gap between, OS upgrades and running RA version on not compatible OS version should be reduced and if possible avoided, as these scenarios are not QA'd.

 

Prevalent practices/guidelines for upgrade(Below instructions are specific to customer infrastructure and process followed, please pick the appropriate judiciously)

 

1. Backup
   • Backup of DB before upgrade (this is useful for quick rollback if needed)
   • Backup of Install directory of NAC or NES (this is useful for quick restoration if needed)
   • If you are using virtual environments,  the snapshot of the VM's hosting NAC/NES can be taken and can be used as checkpoint for quick restoration.
2. Create a backup 
3. Stop Applications
4. OS Upgrade, if applicable
   • Upgrade OS
   • Do OS sanity testing
   • Check user accounts, firewall and app install directory permissions (To be same as pre OS upgrade)
   • Take snapshot or backup of NAC Install directory
5. Upgrade Application, if applicable
   • Upgrade RA to GA version
   • If any RA cumulative patch is required to be applied, please apply the same
   • Do sanity on RA Component
6. Start Application and do sanity testing