VMware Cloud Foundation WLD Domain Delete is failing for scale environments at "Update workload domain statuses in inventory" with 413 Request Entity Too Large
search cancel

VMware Cloud Foundation WLD Domain Delete is failing for scale environments at "Update workload domain statuses in inventory" with 413 Request Entity Too Large

book

Article ID: 313884

calendar_today

Updated On:

Products

VMware Cloud Foundation

Issue/Introduction

Symptoms:
  • WLD Domain Deletion for Domains with more than around 150 hosts in total.
  • WLD Domain Deletion is failing for scale environments at "Update workload domain statuses in inventory" sub-task.
  • A similar error log is present in /var/log/vmware/vcf/domainmanager/domainmanager.log on the SDDC Manager:
SDDC_MANAGER_UPDATE_DOMAIN_INVENTORY_FAILED Failed to update domain inventory for domainId 3b5e4518-df73-46ab-954c-4b76615ec458
com.vmware.evo.sddc.common.services.error.SddcManagerServicesIsException: Failed to update domain inventory for domainId 3b5e4518-df73-46ab-954c-4b76615ec458
...
Caused by: org.springframework.web.client.HttpClientErrorException: 413 Request Entity Too Large: [<html>^M
<head><title>413 Request Entity Too Large</title></head>^M
<body>^M
<center><h1>413 Request Entity Too Large</h1></center>^M
<hr><center>nginx</center>^M
</body>^M
</html>^M


Environment

VMware Cloud Foundation 3.0.x
VMware Cloud Foundation 4.3
VMware Cloud Foundation 3.10.x
VMware Cloud Foundation 4.0.x
VMware Cloud Foundation 4.3.1.0

Cause

Web server configured to restrict large file size. The default nginx client_max_body_size is 1MB

Resolution

To fix this issue, please remove clusters from the domain, until total hosts size is less than around 150.

Workaround:
  • To workaround this issue, you will need to manually bumping the client_max_body_size for the requests matching inventory/extensions/vi/domainInventory to 5m or more by following the below steps:
  1. Login to SDDC Manager; switch to root user.
  2. Make a copy of the nginx config: cp /etc/nginx/nginx.conf /home/nginx.conf.copy
  3. chmod +w /etc/nginx/nginx.conf
  4. Bump the max body size:
  • vi /etc/nginx/nginx.conf
  • add client_max_body_size 5m; in ~ ^/(activity|appliancemanager|css|credentials-service|dns|inventory|locks|task-types|networkpools|tasks|workflows)(.*) nginx location
  1. Remove file write permissions: chmod -w /etc/nginx/nginx.conf
  2. Reload nginx service: systemctl reload nginx
  3. Make sure nginx service is healthy: nginx -t


Powered by Wolken Software