Elastic Runtime CFOPS Backup Error "mysql.backup: permission denied" is seen
search cancel

Elastic Runtime CFOPS Backup Error "mysql.backup: permission denied" is seen

book

Article ID: 297815

calendar_today

Updated On:

Products

VMware Tanzu Application Service for VMs

Issue/Introduction

Symptoms:

CFOPS backups fail for elastic-runtime tile.

For example:

ubuntu@pivotal-ops-manager:/dd-bkp/17-Apr-2017$ cfops backup --opsmanagerhost <ops-mgr-URL> --adminuser admin --adminpass <password> --opsmanageruser ubuntu --opsmanagerpass <password> -d /<directory>/ --tile elastic-runtime 
2017/04/17 09:56:42 I0315 09:56:42.294729 29772 elasticruntime.go:144] Exporting %s nfs_server 
2017/04/17 09:56:42 I0315 09:56:42.295988 29772 elasticruntime.go:144] Exporting %s mysql 
2017/04/17 09:56:42 E0315 09:56:42.296628 29772 elasticruntime.go:69]Error backing up db open /<directory>/mysql.backup: permission denied
2017/04/17 09:56:42 E0315 09:56:42.296652 29772 createCliCommand.go:52] there was an error: failed to backup database running backup on elastic-runtime tile:tile

Issue

The issue exists because the VM vcap credentials are not in sync with the vcap credentials displayed in the Ops Manager Credentials tab. The password known to Ops Manager is different than that used by VM.

This can also happen when the option "Use default BOSH password" is selected under the Ops Manager tile -> Security tab.

Hence, when CFOPS is trying to SSH to the VM, it fails with permission denied.

Analysis

ubuntu@pivotal-ops-manager:/dd-bkp/15-Mar-2017$ cfops version
cfops version v3.0.5
ubuntu@pivotal-ops-manager:/dd-bkp/15-Mar-2017$ cfops list-tiles
Available Tiles:
ops-manager
elastic-runtime

Login to nfs_server using the vcap credentials of the VM.

  • Login to Ops Manager dashboard and make a note of the nfs_server VM vcap credentials from the Elastic Runtime tile -> Credentials tab
  • Make a note of the nfs_server IP Address from the Elastic Runtime tile -> Status tab
  • Using a terminal window, try to SSH into the nfs_server IP address with the vcap credentials noted above

Failing to login to nfs_server VM using above vcap credentials confirms the password sync issue.

This can be tested for any other VM under the Elastic Runtime tile.

 

Environment


Cause

There are two possible causes:

  1. During the upgrade process, the passwords were reset. However, the upgrade process failed due to an error before updating the passwords in installation YAML.
  2. The option "Use default BOSH password" is selected under Ops Manager tile -> Security tab.

 

Resolution

To fix the password sync issue, we need to recycle and regenerate the passwords by using the Bosh default password settings and setting them back to Generate passwords for Bosh.

Follow these steps to sync the vcap credentials with actual VMs.

  1. Login to Ops Manager
  2. Under, Ops Manager tile -> Security tab, select the option "Use default BOSH password"
  3. Click Apply changes on the Installation dashboard
  4. Once this is completed, go back to Ops Manager tile -> Security tab and select the option "Generate passwords" to re-generate the vcap credentials
  5. Click Apply changes to commit these changes
  6. Once done, try to log in again to nfs_server VM using vcap credentials generated on the Credentials tab of ERT
  7. Once this is working, run the CFOPS backup of Elastic Runtime tile as desired
./cfops backup --opsmanagerhost <ops-mgr-URL> --adminuser <admin-user> --adminpass <admin-password> --opsmanageruser ubuntu 
--opsmanagerpass <password> -d /<directory>/ --tile elastic-runtime
 


Powered by Wolken Software