TLS errors during deploy-all errand for CredHub Service Broker
Article ID: 298162
Updated On:
Products
VMware Tanzu Application Service for VMs
Issue/Introduction
deploy-all for CredHub Service Broker errand is failing
===== 2024-01-19 17:05:26 UTC Running "/usr/local/bin/bosh --no-color --non-interactive --tty --environment=10.0.0.26 --deployment=credhub-service-broker-06f54d63cd4eaca90481 run-errand deploy-all" Using environment '10.31.94.26' as client 'ops_manager' Using deployment 'credhub-service-broker-06f54d63cd4eaca90481' Task 1267750 Task 1267750 | 17:05:30 | Preparing deployment: Preparing deployment Task 1267750 | 17:05:30 | Warning: Ambiguous request: the requested errand name 'deploy-all' matches both a job name and an errand instance group name. Executing errand on all relevant instances with job 'deploy-all'. Task 1267750 | 17:05:31 | Preparing package compilation: Finding packages to compile (00:00:00) Task 1267750 | 17:05:31 | Preparing deployment: Preparing deployment (00:00:01) Task 1267750 | 17:05:31 | Creating missing vms: deploy-all/3a91d9f5-f4e7-46a6-a94d-8d596e562b8b (0) (00:01:22) Task 1267750 | 17:06:53 | Updating instance deploy-all: deploy-all/3a91d9f5-f4e7-46a6-a94d-8d596e562b8b (0) (canary) Task 1267750 | 17:06:55 | L executing pre-stop: deploy-all/3a91d9f5-f4e7-46a6-a94d-8d596e562b8b (0) (canary) Task 1267750 | 17:06:55 | L executing drain: deploy-all/3a91d9f5-f4e7-46a6-a94d-8d596e562b8b (0) (canary) Task 1267750 | 17:06:55 | L stopping jobs: deploy-all/3a91d9f5-f4e7-46a6-a94d-8d596e562b8b (0) (canary) Task 1267750 | 17:06:56 | L executing post-stop: deploy-all/3a91d9f5-f4e7-46a6-a94d-8d596e562b8b (0) (canary) Task 1267750 | 17:07:05 | L installing packages: deploy-all/3a91d9f5-f4e7-46a6-a94d-8d596e562b8b (0) (canary) Task 1267750 | 17:07:06 | L configuring jobs: deploy-all/3a91d9f5-f4e7-46a6-a94d-8d596e562b8b (0) (canary) Task 1267750 | 17:07:06 | L executing pre-start: deploy-all/3a91d9f5-f4e7-46a6-a94d-8d596e562b8b (0) (canary) Task 1267750 | 17:07:07 | L starting jobs: deploy-all/3a91d9f5-f4e7-46a6-a94d-8d596e562b8b (0) (canary) Task 1267750 | 17:07:37 | L executing post-start: deploy-all/3a91d9f5-f4e7-46a6-a94d-8d596e562b8b (0) (canary) (00:00:45) Task 1267750 | 17:07:38 | Running errand: deploy-all/3a91d9f5-f4e7-46a6-a94d-8d596e562b8b (0) (00:01:39) Task 1267750 | 17:09:17 | Fetching logs for deploy-all/3a91d9f5-f4e7-46a6-a94d-8d596e562b8b (0): Finding and packing log files (00:00:01) Task 1267750 Started Fri Jan 19 17:05:30 UTC 2024 Task 1267750 Finished Fri Jan 19 17:09:18 UTC 2024 Task 1267750 Duration 00:03:48 Task 1267750 done Errand 'deploy-all' completed with error (exit code 1) Exit code 1 Instance deploy-all/3a91d9f5-f4e7-46a6-a94d-8d596e562b8b Exit Code 1 Stdout cf version 6.53.0+8e2b70a4a.2020-10-01 cf api https://api.<system-domain> cf auth system_services ******** cf target -o credhub-service-broker-org cf target -s credhub-service-broker-space cf push credhub-broker-1.5.4 -n credhub-broker -d apps.<apps-domain> -f /var/vcap/packages/credhub_broker/manifest.yml -s cflinuxfs4 --no-start cf set-env credhub-broker-1.5.4 UAA_HOST https://uaa.<system-domain> cf set-env credhub-broker-1.5.4 CC_HOST https://api.<system-domain> cf set-env credhub-broker-1.5.4 LOGIN_HOST https://login.<system-domain> cf set-env credhub-broker-1.5.4 ROOT $HOME cf set-env credhub-broker-1.5.4 SCHEME https cf set-env credhub-broker-1.5.4 VERIFY_SSL false cf set-env credhub-broker-1.5.4 CF_ORG credhub-service-broker-org cf set-env credhub-broker-1.5.4 CF_SPACE credhub-service-broker-space cf set-env credhub-broker-1.5.4 CF_TARGET https://api.<system-domain> cf set-env credhub-broker-1.5.4 CF_SKIP_SSL false cf set-env credhub-broker-1.5.4 SECURITY_USER_NAME <my-user-name> cf set-env credhub-broker-1.5.4 SECURITY_USER_PASSWORD ******** /tmp/setup /var/vcap/bosh cf push --no-route -b binary_buildpack -p /tmp/setup -u process setup -c sleep infinity /var/vcap/bosh cf start credhub-broker-1.5.4 Starting app credhub-broker-1.5.4 in org credhub-service-broker-org / space credhub-service-broker-space as system_services... Staging app and tracing logs... Downloading binary_buildpack... Downloaded binary_buildpack Cell 3a7fb6c4-68f1-4121-8feb-428ef3a229da creating container for instance 121361be-dda0-45aa-8fec-d92da1b58620 Security group rules were updated Cell 3a7fb6c4-68f1-4121-8feb-428ef3a229da successfully created container for instance 121361be-dda0-45aa-8fec-d92da1b58620 Downloading build artifacts cache... Downloading app package... Downloaded build artifacts cache (215B) Downloaded app package (5.2M) -----> Binary Buildpack version 1.1.8 Exit status 0 Uploading droplet, build artifacts cache... Uploading droplet... Uploading build artifacts cache... Uploaded build artifacts cache (218B) Uploaded droplet (5.2M) Uploading complete Cell 3a7fb6c4-68f1-4121-8feb-428ef3a229da stopping instance 121361be-dda0-45aa-8fec-d92da1b58620 Cell 3a7fb6c4-68f1-4121-8feb-428ef3a229da destroying container for instance 121361be-dda0-45aa-8fec-d92da1b58620 Cell 3a7fb6c4-68f1-4121-8feb-428ef3a229da successfully destroyed container for instance 121361be-dda0-45aa-8fec-d92da1b58620 Waiting for app to start... name: credhub-broker-1.5.4 requested state: started routes: credhub-broker.<apps-domain> last uploaded: Fri 19 Jan 17:08:06 UTC 2024 stack: cflinuxfs4 buildpacks: binary type: web instances: 1/1 memory usage: 256M start command: ./credhub-service-broker state since cpu memory disk details #0 running 2024-01-19T17:08:12Z 0.2% 17.1M of 256M 9.2M of 1G cf app credhub-broker-1.5.4 # waiting for running state cf app credhub-broker-1.5.4 # waiting for running state cf app credhub-broker-1.5.4 # waiting for running state cf app credhub-broker-1.5.4 # waiting for running state cf app credhub-broker-1.5.4 # waiting for running state cf app credhub-broker-1.5.4 # waiting for running state cf update-service-broker credhub-broker d2b0ec31be5c5b7d ******** https://credhub-broker.<apps-domain> Updating service broker credhub-broker as system_services... FAILED Server error, status code: 500, error code: 10001, message: SSL_connect returned=1 errno=0 peeraddr=10.0.0.66:443 state=error: unsafe legacy renegotiation disabled cf delete -f setup Stderr Using cflinuxfs4 stack 1 errand(s) ===== 2024-01-19 17:09:46 UTC Finished "/usr/local/bin/bosh --no-color --non-interactive --tty --environment=10.0.0.26 --deployment=credhub-service-broker-06f54d63cd4eaca90481 run-errand deploy-all"; Duration: 259s; Exit Status: 1 Exited with 1. Exited with 1.
Environment
Product Version: 4.0
Resolution
Looking at the error reported in the errand
...suggests that a change in TLS versions may be the cause of the error. The diagnostic report shows that most components are now using the Jammy stemcell; Jammy uses TLS v3.0. In another case involving a similar upgrade, the load balancers did not support renegotiation for TLS v3.
Changing the settings to TLS v3 for the NetScaler load balancers resolved the issue.
FAILED Server error, status code: 500, error code: 10001, message: SSL_connect returned=1 errno=0 peeraddr=10.30.135.66:443 state=error: unsafe legacy renegotiation disabled
...suggests that a change in TLS versions may be the cause of the error. The diagnostic report shows that most components are now using the Jammy stemcell; Jammy uses TLS v3.0. In another case involving a similar upgrade, the load balancers did not support renegotiation for TLS v3.
Changing the settings to TLS v3 for the NetScaler load balancers resolved the issue.
Feedback
Yes
No
Powered by
