In NSX-T Edge Nodes, the NSX CLI fails when the root password is expiring

Products

• VMware NSX

Issue/Introduction

• You are running NSX-T 3.1.x, 3.2.x or 4.1.x. 
• When BGP-related NSX CLI commands are executed, the output is shown in JSON format due to password expiry warning.
• If there is any automation being performed that utilizes NSX CLI to process the NSX Routing, the CLI will fail due to change in the output format.
• No impact to traffic flow or route exchange is observed.
• Entries similar to the below are observed in var/log/rcpm/frr-cli.log
  

  frr_cli_utils: ERROR Warning: your password will expire in 1 day
  .
  
  .
  
  frr_cli_helper: ERROR Exception: ['Traceback (most recent call last):
  ', '  File "/opt/vmware/nsx-edge/bin/frr_cli_utils.py", line 63, in load_json
      j = json.loads(out)
  ', '  File "/usr/lib/python3.8/json/__init__.py", line 357, in loads
      return _default_decoder.decode(s)
  ', '  File "/usr/lib/python3.8/json/decoder.py", line 337, in decode
      obj, end = self.raw_decode(s, idx=_w(s, 0).end())
  ', '  File "/usr/lib/python3.8/json/decoder.py", line 355, in raw_decode
      raise JSONDecodeError("Expecting value", s, err.value) from None
  ', 'json.decoder.JSONDecodeError: Expecting value: line 1 column 1 (char 0)
  ', '
  During handling of the above exception, another exception occurred:
  
  ', 'Traceback (most recent call last):
  ', '  File "/opt/vmware/nsx-edge/bin/frr_cli_helper.py", line 75, in process_cli_cmd
      _CLI_COMMANDS[cmd_list[0]](len(cmd_list[1:]), cmd_list[1:],
  ', '  File "/opt/vmware/nsx-edge/bin/frr_cli_rib.py", line 380, in get_route_vrf_all
      frr_out = utils.load_json(output)
  ', '  File "/opt/vmware/nsx-edge/bin/frr_cli_utils.py", line 68, in load_json
      sys.exit(0)
  ', 'SystemExit: 0
  ']

Environment

VMware NSX-T Data Center 3.x
VMware NSX 4.x

Cause

• By default, the admin and root passwords on Edge (release build) at deployment time (when you do not set them via the OVF properties) will expire immediately. This is by design.
• If you set them via OVF properties at deployment time then they will expire after 90 days.
• From the 84th day onwards there will be a warning indicating that the root password will expire.
• NSX routing CLI takes a JSON output from the backend and converts it into tabular output for the user but since password is expiring for root user, a warning message gets added to the JSON output, which is unexpected and python processing of this JSON output fails.

Resolution

This issue is resolved in VMware NSX 4.2.0, available at Broadcom downloads.

If you are having difficulty finding and downloading software, please review the Download Broadcom products and software KB.

 

Workaround

Changing the password for root and admin will resolve the issue.