VMware Aria Operations 8.18.x
This error occurs when you attempt to start services while the Fault Domain is marked as OFFLINE/FAILURE in casa.
To resolve this issue:
1. Log into the Aria Operations admin UI as the local admin user
2. Click Take Cluster Offline under Cluster Status
Note: Wait for Cluster Status to show as Offline
3. In vCenter, preform a Shut Down Guest OS action on the VM. Aria Operations nodes should be taken offline in the following order:
1. Witness node
2. Remote collector node(s)
3. FD2 Replica Data node(s)
4. FD2 Primary replica node
5 .FD1 Data node(s)
6. FD1 Primary node
4. In vCenter, preform a Power On action on the VM. Aria Operations nodes should be brought online in the following order:
1. FD1 Primary node
2. FD1 Data node(s)
3. FD2 Primary replica node
4. FD2 Replica Data node(s)
5. Remote collector node(s)
6. Witness node
5. Once each Aria Operations Analytics node has restarted, log back into the Aria Operations admin UI as the local admin user
6. Click Bring Cluster Online under Cluster Status
Note: Wait for the Cluster Status to show as Online
If the cluster does not respond to the "Bring Cluster Online" button, please contact VMware by Broadcom Technical Support for assistance and reference this KB
| INFO | jvm 1 | WARNING: An illegal reflective access operation has occurred
| INFO | jvm 1 | WARNING: Illegal reflective access by com.vmware.vcops.casarest.client.HttpRequesterURLConnectionImpl (file:/usr/lib/vmware-vcops/common/lib/casa-rest-client-1.0-SNAPSHOT.jar) to field java.lang.reflect.Field.modifiers
| INFO | jvm 1 | WARNING: Please consider reporting this to the maintainers of com.vmware.vcops.casarest.client.HttpRequesterURLConnectionImpl
| INFO | jvm 1 | WARNING: Use --illegal-access=warn to enable warnings of further illegal reflective access operations
| INFO | jvm 1 | WARNING: All illegal access operations will be denied in a future release
| INFO | jvm 1 | >>> AnalyticsMain.run failed with error: IllegalStateException: Attempt to start node from not accessible zone.
| INFO | jvm 1 | WrapperManager Debug: WrapperManager.stop(-1) called by thread: SystemExitThread
| INFO | jvm 1 | WrapperManager Debug: Send a packet STOP : -1
| INFO | jvm 1 | WrapperManager Debug: Pausing for 1,000ms to allow a clean shutdown...
| DEBUG | wrapperp | read a packet STOP : -1
| DEBUG | wrapper | JVM requested a shutdown. (-1)
| DEBUG | wrapper | wrapperStopProcess(-1, FALSE) called.
| DEBUG | wrapper | Sending stop signal to JVM
| DEBUG | wrapper | Enqueue Event 'jvm_stop'
| INFO | jvm 1 | WrapperManager Debug: Stopped checking for control events.
INFORMATION [Analytics Main Thread] com.vmware.vcops.analytics.cluster.ClusterCoordinatorImpl.setAccessibleZones - Available zones are: [domain2]
ERROR [Analytics Main Thread] com.integrien.analytics.AnalyticsMain.run - AnalyticsMain.run failed with error: IllegalStateException: Attempt to start node from not accessible zone.java.lang.IllegalStateException: Attempt to start node from not accessible zone.
at com.vmware.vcops.analytics.cluster.ClusterCoordinatorImpl.<init>(ClusterCoordinatorImpl.java:135) ~[vcops-analytics-1.0-SNAPSHOT.jar:?]
at com.vmware.vcops.analytics.cluster.ClusterCoordinatorImpl.getInstance(ClusterCoordinatorImpl.java:100) ~[vcops-analytics-1.0-SNAPSHOT.jar:?]
at com.integrien.analytics.AnalyticsMain.doRun(AnalyticsMain.java:475) ~[vcops-analytics-1.0-SNAPSHOT.jar:?]
at com.integrien.analytics.AnalyticsMain.run(AnalyticsMain.java:2251) ~[vcops-analytics-1.0-SNAPSHOT.jar:?]
INFO [Analytics Main Thread] com.vmware.vcops.platform.common.PlatformEnvironment.exitSystem - Terminating process with exit code -1 ...
INFORMATION [SystemExitThread] com.vmware.vcops.platform.common.PlatformEnvironment.run - exitSystem has been called by:
java.lang.Throwable: null
at com.vmware.vcops.platform.common.PlatformEnvironment.exitSystem(PlatformEnvironment.java:276) ~[alive_platform.jar:?]
at com.vmware.vcops.platform.common.PlatformEnvironment.exitSystem(PlatformEnvironment.java:303) ~[alive_platform.jar:?]
at com.integrien.analytics.AnalyticsMain.run(AnalyticsMain.java:2256) ~[vcops-analytics-1.0-SNAPSHOT.jar:?]
INFORMATION [WrapperListener_stop_runner] com.integrien.analytics.AnalyticsMain.stop - Analytics is stopping... exit code -1
WARN [WrapperListener_stop_runner] com.integrien.analytics.AnalyticsMain.stop - Analytics is stopping... exit code -1. Generating thread dump in the call stack directory ..
INFORMATION [Take Analytics Offline Thread] com.integrien.analytics.AnalyticsMain.run - Analytics is going offline..
INSERT INTO CASA_DOCS VALUES('CA_INFO','{"document_time":1716437986633,"document_name":"CA_INFO","document_version":52,"document_body":{"regionA":"FAILURE","regionA_last_down_time":1716338218598,"regionA_last_processing_time":1716338137983,"regionB":"FAILURE","regionB_last_down_time":1712270794768,"regionB_last_processing_time":1712270687798,"split_brain":null,"regionA_display_name":null,"regionB_display_name":null,"document_version":54}}')
INSERT INTO CASA_DOCS VALUES('FIPS','{"document_time":1698882002578,"document_name":"FIPS","document_version":4,"document_body":{"state":"DISABLED"}}'
)