A VMware Blockchain client node does not have an IP address during/after a clone-based upgrade

Products

• VMware Blockchain

Issue/Introduction

Symptoms:

During or after a VMware Blockchain clone-based upgrade, one or more client node do not have an IP address

Cause

Blockchain reconfiguration operations, such as cloning, scaling, or deployment, use a token for secure communication between nodes. This token has a 45 minutes hard-coded validity. If the reconfiguration operation takes more than 45 minutes, it will fail when the token expires.

Environmental circumstances such as a slow connection to the jfrog image repository, or a slow cloning process, can trigger this scenario.

Resolution

This is expected behavior under circumstances with VMware Blockchain. There is currently no resolution.

Workaround:
You can work around this behavior by completing the following steps:

Note: These steps must be performed in the 45 minute timeframe where the token is valid. If the steps take longer than 45 minutes, you will need to start the entire process again.

1. Power off and delete all cloned nodes.
2. Execute the clone operation again. For any new node that does not have an external IP address, perform a reboot of that node. After rebooting the node, it should have an external IP address.
3. ssh to the rebooted node.
   1. Ensure sure that the /config/agent/config.json file is created with content.
   2. Execute a command similar to the following login to the jfrog image repository:

docker login -u <username> https://vmwaresaas.jfrog.io/vmwblockchain

Note: Replace <username> with your jfrog username. Enter the appropriate password when prompted. You can find the jfrog username and password in your infrastrucutre json file under the registry section.

3. Execute a command similar to the following to download the agent image:

 docker pull vmwaresaas.jfrog.io/vmwblockchain/agent:1.3.0.3.83

Note: replace 1.3.0.3.83 with the appropriate version for your VMware Blockchain installation.

4. Once the image is downloaded and listed in the ‘docker images’ command output, execute a command similar to the following to create the agent container.

grep -o ‘docker container.*’ /var/lib/cloud/instance/user-data.txt | sed ‘s/:1.3.0.3.83/1.3.0.3.83/g’ | sh

Note: replace 1.3.0.3.83 with the appropriate version for your VMware Blockchain installation.

5. Validate that agent container is created with the ‘docker ps -a’ command. Start the agent container by issuing the 'docker start agent' command.

 

Once agent container is started started, it downloads all other container images. All containers are created and not started, as expected, for the clone-based upgrade.