How to make BOSH Custom Configurations persist during Ops Manager Upgrades and Apply Changes

Products

• Operations Manager

Issue/Introduction

This article describes how to provide custom or additional cloud properties to BOSH that are not available for configuration in the BOSH tile.

Sometimes you may need to introduce custom configurations to your environment. 

While there are many examples of such configurations - we will use two examples to illustrate how to go about making these changes and how to make sure they persist during an upgrade.

1. Updating our cpi-config with custom cloud properties.
2. Providing custom VM types.

Environment

Product Version: Other

Resolution

First we must understand that Ops Manager will overwrite most settings for BOSH during every Apply Changes. This is by design - Ops Manager always wants to match the desire in the BOSH tile to the actual settings. Unless Ops Manager is aware of these new configurations - they most likely will not persist on an Apply Changes or an upgrade. 

If we want to introduce custom configurations that are not available for configuration in the BOSH tile that will persist Apply Changes or upgrades, then we will need to use the Ops Manager API to introduce these changes. 

The Ops Manager API documentation can be reached various ways, here are the two most common:

1. On the Installation dashboard there is a link to the API documentation in the bottom right part of the screen.
2. The public docs (<MAJOR-MINOR> represent your Ops Manager version).

https://docs.pivotal.io/platform/<MAJOR-MINOR>/opsman-api/#the-basics

 For example Ops Manager 2.7 would be:

https://docs.pivotal.io/platform/2-7/opsman-api/

Example 1 - Updating our cpi-config with custom cloud properties

Lets say we want to add the configuration enable_human_readable_name = true. Information about what this property does can be found here. 

Note: Properties will vary depending on the CPI you're trying to use.
For example in OpenStack this property is called human_readable_vm_names where in vSphere it is called enable_human_readable_name. See the "Global Configuration" section of your CPI for more details.

There is no place in the BOSH tile to configure this, so we will utilize our Ops Manager API to add this property.

1. Obtain the TOKEN from UAA to authenticate these API calls with the instructions in the API docs. You can see can example here. 
2.  Make a call to the API to get the current configuration as documented here and pipe the output to a file. For example: 

   curl "https://example.com/api/v0/staged/director/properties" \
       -X GET \
       -H "Authorization: Bearer UAA_ACCESS_TOKEN" | jq . > config.json

3.  Open config.json and edit the desired additional_cloud_properties. For example, if you wanted to put in the enable_human_readable_name then place that in this file like this:

   "iaas_configuration": {
      ...data...(for brevity, ...data... replaces text not needed for this demonstration.)
       "additional_cloud_properties": {
         "enable_human_readable_name": true
       },
       ...data...
     }
   

4. Save config.json and then perform the PUT with the data in config.json back to that API endpoint. The successful response as stated in the documentation, should return a 200 if successful: 

   curl "https://example.com/api/v0/staged/director/properties" \
       -X PUT \
       -H "Authorization: Bearer UAA_ACCESS_TOKEN" \
       -H "Content-Type: application/json" \
       -d '@path/to/config.json'
   
   
   HTTP/1.1 200 OK {}

5. Before Kicking-off an Apply change, click "See Changes" on the bosh director tile to review pending changes and observe where in the manifest, the property is going to update (see screenshot)
6. Run an Apply Change at least on the bosh director tile to persist changes.

Note: It is worth noting that we must pass the entire contents of the config.json file here and not just the additional cloud property. This is because our api calls are replacing configurations. This is why we first obtain the configurations and then append our changes to it. When we do this, we ensure we are passing back the existing configurations plus our newly added additional custom configurations.

Example 2 - Add custom VM types so they are selectable in resource config sections of the tiles

1. Obtain the TOKEN from UAA to authenticate these API calls with the instructions in the API docs. You can see an example here.
2.  Make a call to the API to get the current VMs as documented here and pipe the output to a file. For example: 

   curl "https://example.com/api/v0/vm_types" \
       -X GET \
       -H "Authorization: Bearer UAA_ACCESS_TOKEN" | jq . > vm_types.json

3. Open the vm_types.json file and append the custom VM types in the list. For example:

   {
   	"vm_types": [{
   		"name": "nano",
   		"ram": 512,
   		"cpu": 1,
   		"ephemeral_disk": 8192,
   		"builtin": true
   	},
             ......all other VM types are not shown for brevity... 
           {
   		"name": "2xlarge.cpu",
   		"ram": 16384,
   		"cpu": 16,
   		"ephemeral_disk": 65536,
   		"builtin": true
   	},
           {
   		"name": "CUSTOM_VMTYPE_1",
   		"ram": 16384,
   		"cpu": 4,
   		"ephemeral_disk": 16384,
   		"builtin": false
   	}]
   }

4. Once custom VM types are added, save vm_types.json and perform a PUT back to the same endpoint with the contents of vm_types.json. For more information, refer to this documentation.

   curl "https://example.com/api/v0/vm_types" \
       -X PUT \
       -H "Authorization: Bearer UAA_ACCESS_TOKEN" \
       -H "Content-Type: application/json" \
       -d '@path/to/vm_types.json'

These custom VM types should now be selectable in Ops Manager (this may require a refresh of the tab).
 

These additional_cloud_properties will now persist through an Apply Changes and/or Ops Manager upgrade.