Change the IP Address of a VMware Aria Operations 8.0 - 8.12.x Deployment

Products

• VMware Aria Suite

Issue/Introduction

This article provides steps to change a node's IP address in a VMware Aria Operations (formerly known as vRealize Operations) 8.0 - 8.12.x cluster after installing and configuring VMware Aria Operations 8.0 - 8.12.x.

Environment

VMware vRealize Operations 8.x
VMware vRealize Operations 8.6.x
VMware vRealize Operations 8.1.x
VMware vRealize Operations 8.2.x
VMware vRealize Operations 8.4.x
VMware vRealize Operations 8.3.x
VMware vRealize Operations 8.0.x
VMware Aria Operations 8.12.x
VMware vRealize Operations 8.5.x
VMware vRealize Operations 8.10.x

Resolution

Quick Links:
Change IP Address
Apache Configuration
Cassandra Configuration
PostgreSQL Configuration for HA/CA
Verify

Note: The Cassandra Configuration section is only applicable to vRealize Operations 8.6.x and below.
 

Change IP Address

1. Log into the VMware Aria Operations Admin UI as the local admin user.
2. Click Take Offline under Cluster Status.

Note: Wait for Cluster Status to show as Offline.

3. Log into the vSphere UI as an administrator.
4. Perform a Guest Shutdown on all VMware Aria Operations nodes in the cluster.
5. For each node that you want to change, perform these steps:

Note: Start with the Primary node, then Primary Replica (if present), then data nodes and then RC nodes. Do not proceed to the next step before completing the sub steps on each node.

1. In vSphere UI, select the virtual machine and navigate to Configure > vApp Options > Properties.
2. For each Key below, select the Key, then click Set Value.  Enter the below values and click OK.

• IP address: vami.ip0.vRealize_Operations_Manager_Appliance
• Subnet Mask: vami.netmask0.vRealize_Operations_Manager_Appliance
• Gateway: vami.gateway.vRealize_Operations_Manager_Appliance
• DNS: vami.DNS.vRealize_Operations_Manager_Appliance

3. Power on the node.
4. Log into the Console as root by pressing ALT+F1 to log in.
5. Run the following command to update the IP address properties; choose the command applicable to your type of IP address:

Static Address

/opt/vmware/share/vami/vami_set_network eth0 STATICV4 ipv4_addr ipv4_netmask ipv4_gateway

Note: Replace ipv4_addr, ipv4_netmask, and ipv4_gateway with the correct network parameters

Example: /opt/vmware/share/vami/vami_set_network eth0 STATICV4 192.168.1.53 255.255.255.0 192.168.0.1
 

DHCP Address

/opt/vmware/share/vami/vami_set_network eth0 DHCPV4

Note: Primary and Primary replica nodes cannot use DHCP addresses.

6. Restart the node.
7. After the node has restarted, log into the Console as root by pressing ALT+F1 to log in.
8. Stop the CaSA service by running this command:

service vmware-casa stop

6. For each node that you want to change, perform these steps:

Note: Start with the Primary node, then Primary replica (if present), then data nodes, then Remote Collector nodes.  Do not proceed to the next step before completing the sub steps on each node.

1. Run the following commands on all nodes to backup the important files:

• cp /storage/db/casa/webapp/hsqldb/casa.db.script /tmp/
• cp /usr/lib/vmware-vcopssuite/utilities/sliceConfiguration/data/roleState.properties /tmp/
• cp /usr/lib/vmware-vcops/user/conf/gemfire.properties /tmp/
• cp /usr/lib/vmware-vcops/user/conf/gemfire.locator.properties /tmp/
• cp /usr/lib/vmware-vcops/user/conf/gemfire.native.properties /tmp/
• cp /usr/lib/vmware-vcops/user/conf/persistence/persistence.properties /tmp/
• cp /usr/lib/vmware-vcops/user/conf/cassandra/cassandra.yaml /tmp/
• cp /opt/vmware/etc/vami/ovfEnv.xml /tmp
• cp /usr/lib/vmware-vcops/user/conf/cis.properties /tmp

2. Open the /storage/db/casa/webapp/hsqldb/casa.db.script file using a text editor.
3. Locate any instances of the old IP address (or FQDN’s) and update them to the new IP address (or FQDN’s).
4. Save and close the casa.db.script file.
5. Open the /usr/lib/vmware-vcopssuite/utilities/sliceConfiguration/data/roleState.properties file using a text editor.
6. Locate any instances of the old IP address, and update them to the new IP address.

Example:
adminroleconnectionstring = 192.168.1.10,192.168.1.11
internaladminroleconnectionstring = 192.168.1.10,192.168.1.11

7. Save and close roleState.properties file.
8. Open the /usr/lib/vmware-vcops/user/conf/persistence/persistence.properties file using a text editor.
9. Locate the line with the old Primary nodes IP address and update it to the new Primary nodes IP address.
10. Locate the line with the old Primary Replica (if present) nodes IP address and update it to the new Primary Replica nodes IP address.
11. Save and close the persistence.properties file.
12. Run this command to start the CaSA service:

service vmware-casa start

13. Navigate to the following folder:

cd /usr/lib/vmware-vcopssuite/utilities/sliceConfiguration/bin

14. The following command needs to be run in a particular order.  Choose the command applicable to your cluster and follow each sub-step carefully.

Command for single node or non-HA multi node cluster:
$VMWARE_PYTHON_BIN ./vcopsConfigureRoles.py --adminCS=primary_ip > /dev/null 2>&1

Command for HA multi node or CA multi node cluster:
$VMWARE_PYTHON_BIN ./vcopsConfigureRoles.py --adminCS=primary_ip,replica_ip > /dev/null 2>&1

Note: Replace primary_ip and replica_ip in the above commands with the IP address of your VMware Aria Operations primary node's IP, and replica node's IP

1. First, run the command on Primary node in the cluster, and wait for the task to complete.
2. Next, run the command on Primary Replica node (if present) in the cluster, and wait for the task to complete.
3. Next, run the command on all Data nodes (if present) and the Witness node (if present), and wait for the task to complete.
4. Finally, run the command on all Remote Collector nodes (if present) in the cluster, and wait for the task to complete.

Note: This script updates the IP addresses in the below files:

• /usr/lib/vmware-vcops/user/conf/gemfire.properties
• /usr/lib/vmware-vcops/user/conf/gemfire.locator.properties
• /usr/lib/vmware-vcops/user/conf/gemfire.native.properties
• /usr/lib/vmware-vcops/user/conf/persistence/persistence.properties

After the script has been run, verify that the IP addresses have been successfully updated in the above mentioned files.

7. Run the following commands on all nodes to restart the vmware-vcops-web and vmware-vcops services

service vmware-vcops-web restart
service vmware-vcops restart

Note: The vmware-vcops-web service is only applicable for Analytics (Primary, Replica, and Data) nodes, and will fail to run on other node types.

 

Apache Configuration

Run the following steps only on the nodes where you have changed the IP address.

1. Open /etc/httpd/listen.conf in a text editor.
2. Find the line with Listen old_IP:443.

Note: old_IP will be the old IP address assigned to the node.

3. Replace old_IP with the new IP address assigned to the node.

Example: Listen 192.168.1.10:443 

4. Save and close the file.
5. Restart the Apache service by running this command:

systemctl restart httpd

 

Cassandra Configuration

Note: The Cassandra Configuration section is only applicable to vRealize Operations 8.6.x and below.  Skip this section if you are on a later release of VMware Aria Operations.

Run the following steps only on the nodes where you have changes the IP address.

1. Open /usr/lib/vmware-vcops/user/conf/cassandra/cassandra.yaml in a text editor.
2. Find the listen_address: old_IP line.
3. Replace old_IP with the new IP address assigned to the node.

Example: listen_address: 192.168.1.10

4. Find the broadcast_rpc_address: old_IP line.
5. Replace old_IP with the new IP address assigned to the node

Example: broadcast_rpc_address: 192.168.1.10
 

The following steps are only applicable if the IP address of the Primary and/or Primary Replica (if present) node has been changed.  These steps must be run on all nodes.  If the IP address of the Primary and/or Primary Replica (if present) nodes has not been changed, you may skip these steps and only complete step 8.

6. Choose the option that is applicable to your cluster:

• For single node or non-HA multi node clusters, find the seeds: "primary_old_IP" line.
• For HA multi node or CA multi node clusters, find the seeds: "primary_old_IP,replica_old_IP" line.

7. Replace primary_old_IP and replica_old_IP (if present) with the new IP addresses assigned to the Primary node and Primary Replica node (if exist).

Examples:

• seeds: "192.168.1.10"
• seeds: "192.168.1.10,192.168.1.11"

8. Save and close the file.

 

PostgreSQL Configuration for HA/CA

The following steps are only applicable is the IP address of the Primary Replica (if present) node has been changed.

1. On the Primary node, open /storage/db/vcops/vpostgres/repl/pg_hba.conf with a text editor.

Note: If a Swap file error appears, type E to Edit anyway.

2. Find the host replication vcopsrepl replica_old_IP/32 md5 line and change replica_old_IP to the new IP address of the Primary Replica node.

Example: host replication vcopsrepl 192.168.1.11/32 md5

3. Save and close the file.
4. Verify that the /storage/db/vcops/vpostgres/repl/recovery.conf file exists on the Primary Replica node and it is not empty.
5. Run the following commands to restart the vpostgres and vpostgres-repl services:

service vpostgres reload
service vpostgres-repl reload

 

Verify

To verify the configuration, complete the following.

1. Check the /opt/vmware/etc/vami/ovfEnv.xml file for vami.DNS, vami.gateway, vami.ip0, and vami.netmask0 lines to reflect the new IP addresses.

Example:
<Property oe:key="vami.DNS.vRealize_Operations_Manager_Appliance" oe:value="192.168.1.2"/>
<Property oe:key="vami.gateway.vRealize_Operations_Manager_Appliance" oe:value="192.168.0.1"/>
<Property oe:key="vami.ip0.vRealize_Operations_Manager_Appliance" oe:value="192.168.3.77"/>
<Property oe:key="vami.netmask0.vRealize_Operations_Manager_Appliance" oe:value="255.255.240.0"/>

2. In the same file, ensure hosts= and MY_LDUID= show the correct cluster name as the other nodes.

Example: If your cluster name is ops.vcloud.local, then the entries should look like the following:
hosts=opsvcloudlocal,localhost:6061
MY_LDUID=opsvcloudlocal

3. Cat the /storage/db/casa/webapp/hsqldb/casa.db.script file to ensure that FQDN’s (or IP addresses) are changed to the new values in all nodes.

Example: 
...INSERT INTO CASA_DOCS VALUES..."slices":{"<UUID_1>":{"slice_uuid":"<UUID_1>","pair_uuid":null,"is_admin_node":true,"ip_address":"host031. hostname.local","preferred_addresses":{},"slice_name":"master","membership_state":null,"region":null},"2e11a1ea-54c8-46cd-ae24-c55a182a8988":{"slice_uuid":"2e11a1ea-54c8-46cd-ae24-c55a182a8988","pair_uuid":null,"is_admin_node":false,"ip_address":"host033. hostname.local","preferred_addresses":{},"slice_name":"Data","membership_state":null,"region":null},"9042a5cd-915a-41a9-8e43-cb6c9648961a":{"slice_uuid":"9042a5cd-915a-41a9-8e43-cb6c9648961a","pair_uuid":null,"is_admin_node":false,"ip_address":"host032. hostname.local","preferred_addresses":{},"slice_name":"Replica","membership_state":null,"region":null},"01a385af-91c3-45c2-92c9-63cefb1bb50b":{"slice_uuid":"01a385af-91c3-45c2-92c9-63cefb1bb50b","pair_uuid":null,"is_admin_node":false,"ip_address":"host034. hostname.local","preferred_addresses":{},"slice_name":"RC","membership_state":null,"region":null}}}')

4. In other configuration files, verify that all instances of old IP addresses have been changed to the new values in all nodes.  Run the following command on all nodes where node_old_IP will be the old IP address assigned to the node, primary_old_IP will be the old IP address assigned to the Primary node, and replica_old_IP will be the old IP address assigned to the Primary Replica node (if present).  If  the command output is not empty, fix the address mentioned in the output file according to above instructions in Change IP Address.

grep -E 'node_old_IP|primary_old_IP|replica_old_IP' /usr/lib/vmware-vcopssuite/utilities/sliceConfiguration/data/roleState.properties /usr/lib/vmware-vcops/user/conf/gemfire.properties /usr/lib/vmware-vcops/user/conf/gemfire.locator.properties /usr/lib/vmware-vcops/user/conf/gemfire.native.properties /usr/lib/vmware-vcops/user/conf/persistence/persistence.properties /usr/lib/vmware-vcops/user/conf/cassandra/cassandra.yaml /etc/httpd/listen.conf

5. Verify the PostgreSQL configuration, only if a Primary Replica Node exists.
   1. Find the host replication vcopsrepl IP/32 md5 line in /storage/db/vcops/vpostgres/repl/pg_hba.conf in the Primary node and verify that IP address is changed.
   2. Find the line that contains primary_conninfo in /storage/db/vcops/vpostgres/repl/pg_hba.conf in the Primary Replica node and verify that the IP address mentioned in the line is changed to the Primary node's new IP address.

Additional Information

When adding a new node to the cluster, the LDUID on the newly added node should match the LDUID set in /usr/lib/vmware-vcops/user/conf/cis.properties on the other nodes.