How to identify connection state of VMware Tanzu Application Service (TAS) for VMs applications

Products

• VMware Tanzu Application Service for VMs

Issue/Introduction

Application connections are not getting closed out on VMware Tanzu Application Service (TAS) for VMs. An application in a CLOSE_WAIT state may indicate that the code is not properly closing out connections.

Since applications run into containers, the networking is isolated in namespace and you will need to enter namespace in order to pull container details for containerized apps.

Environment

Product Version: 2.10

Resolution

This script runs on Diego Cells and will output the app_guid, PID, and Cell IP for any application in a CLOSE_WAIT state.

source /var/vcap/jobs/cfdot/bin/setup
 
for i in $(sudo /var/vcap/packages/runc/bin/runc --root /run/containerd/runc/garden list | grep 'envoy' | awk '{print $2}')
do
 	sockets=`nsenter -t $i -n ss -ntpo | egrep CLOSE`
        if [[ "$sockets" == *"CLOSE"* ]]
	then
        	INSTANCE_GUID=`sudo /var/vcap/packages/runc/bin/runc --root /run/containerd/runc/garden list | grep $i | awk '{print $1}' | sed s/-envoy//`
        	echo -n "APPGUID="
        	cfdot actual-lrps | egrep $INSTANCE_GUID| jq -r '.process_guid' | cut -d "-" -f 1,2,3,4,5
        	echo -n "CELL_ADDRESS="
        	cfdot actual-lrps | egrep $INSTANCE_GUID| jq -r '.address'
 		nsenter -t $i -n ss -ntpo | egrep CLOSE
	fi
done

 

To run this command on all Diego Cells, you will need to scp the script to /tmp and then move it to /root with these steps:

bosh scp script diego_cell:/tmp/
bosh ssh diego_cell -c "sudo chmod 775 /tmp/script"
bosh ssh diego_cell -c "sudo mv /tmp/script /root"

You can compile a list of app GUIDs in a CLOSE_WAIT state by taking output and running this command: grep APPGUID output | awk -F '=' '{ print $2 }' > app_guids.

You can then take the list of app_guids and feed it into this script - it will output the app name, space, and org for that application.

Note: This needs to be ran from somewhere connected to same foundation with CF CLI (login with UAA admin user).

for i in $(cat app_guids)
do
 	echo "app guid..." $i
        guid=$i
 
        cf curl /v2/apps/$guid | jq .entity.name
        cf curl $(cf curl /v2/apps/$guid | jq -r .entity.space_url) | jq .entity.name
        cf curl $(cf curl $(cf curl /v2/apps/$guid | jq -r .entity.space_url) | jq -r .entity.organization_url)  | jq .entity.name
done