vsfwd service fails to start after upgrading to NSX for vSphere 6.3.0
search cancel

vsfwd service fails to start after upgrading to NSX for vSphere 6.3.0

book

Article ID: 301217

calendar_today

Updated On:

Products

VMware NSX Networking VMware vSphere ESXi

Issue/Introduction

This article provides steps to workaround the issue when vsfwd service fails to start after upgrading to NSX for vSphere 6.3.0.

Symptoms:
In an vSphere 5.5 environment, after upgrading to NSX for vSphere 6.3.0, you might experience these symptoms:
  • Host prep fails
  • vsfwd service fails to start

Note: For additional symptoms and log entries, see the Additional Information section.


Environment

VMware NSX for vSphere 6.3.x

Cause

This issue occurs because of a failure to reserve required memory for successful start of vsfwd process.

Resolution

This issue is resolved in VMware NSX for vSphere 6.3.1, available at Broadcom Downloads.

To work around this issue if you do not want to upgrade, increase the memory for vsfwd process and decrease the memory reservation for the vdpi service.

  1. Log in to the ESXi host as a root user through SSH.
     
  2. Run these commands all ESXi hosts that are experiencing the issue:

    localcli --plugin-dir=/usr/lib/vmware/esxcli/int sched group add --group-name=vsfwd --parent-path=host/vim/vmvisor 2> /dev/null
    Note: This command creates a group called vsfwd under the parent process called vmvisor.

    /usr/lib/vmware/rp/bin/configRP increaseRPMemMaxSize host/vim/vmvisor/vsfwd 512
    Note: This command increases the Reservation Pool of the parent pool (vmvisor process) and allocates 512MB of memory to the vsfwd process.

    /etc/init.d/vShield-Stateful-Firewall restart
    Note: This command restarts the vShield Stateful Firewall (vsfwd) process post increasing the memory pool.
     
  3. Run this command to decrease the memory reservation for the vdpi service.
    localcli --plugin-dir=/usr/lib/vmware/esxcli/int sched group setmemconfig --group-path=host/vim/vmvisor/vdpi --min=100 --max=100 --minlimit=100 --units=mb

Notes:

  • The above commands bumps up the memory reservation for vsfwd which should help on getting the vsfwd service started.
  • The above commands are not persistent on host reboot. To make the change persistent across reboots, Contact Broadcom Support



Additional Information

You experience these additional symptoms:

  • In the /var/log/vmkernel.log file of the ESXi host, you see errors similar to:

    2017-02-05T23:43:33Z esxupdate: LiveImageInstaller: WARNING: Handling Live Vib Failure Error in running ['/etc/init.d/vShield-Stateful-Firewall', 'start', 'install']:
    Return code: 1
    Output: vShield-Stateful-Firewall is not running
    watchdog-dfwpktlogs: PID file /var/run/vmware/watchdog-dfwpktlogs.PID does not exist
    watchdog-dfwpktlogs: Unable to terminate watchdog: No running watchdog process for dfwpktlogs
    Resource pool 'host/vim/vmvisor/vsfwd' release failed. retrying..
    Unable to set memory config for vsfwd to 512MB
    Maximum memory config for vmvisor group is 1006 MB
    Resource pool 'host/vim/vmvisor/vsfwd' released.
    Resource pool creation failed. Not star
    2017-02-05T23:43:33Z esxupdate: ting vShield-Stateful-Firewall


Note: The preceding log excerpts are only examples. Date, time, and environmental variables may vary depending on your environment.