What Happens After you Click "Apply Changes" on the Operations Manager Web Console
search cancel

What Happens After you Click "Apply Changes" on the Operations Manager Web Console

book

Article ID: 293434

calendar_today

Updated On:

Products

Operations Manager

Issue/Introduction

This article explains what happens when you click "Apply Change" button on Ops Manager Web Console.

When you click Apply Changes button on Operations Manager Web Console, the following tasks are executed:

  1. Generate or update deployment manifest based on the changes in configuration
  2. Install or update BOSH Director if there are configuration changes
  3. Update cloud config based on the network, AZ, CIDR, reserved IP ranges, etc.
  4. Install or update tiles one by one on the "Pending" list
    1. Upload stemcell, skip if stemcell already exists
    2. Upload releases, skip if releases already exist
    3. Update BOSH director UUID in deployment manifest file
    4. Detect deployment changes by comparing current deployment with new deployment
    5. Deploy new tile if it is a new install or deploy change on tile if it is an update/upgrade

This article discusses in detail what happens when step 3.e above is executed.


Resolution

The BOSH Director will do the following steps when it receives a BOSH deploy (Or its related commands such as start, stop and recreate)

  1. Check if there is a deployment with the name specified by the deployment manifest
    • If no, create a deployment
    • If yes, lock the deployment so that no other operation can modify it
  2. Validate the deployment manifest syntactically and semantically
    • If invalid, return an error to the user describing the problem(s)
  3. Contact all existing VMs associated with this deployment to determine their network, configuration, and job configurations
    • If the Director cannot contact all VMs, return an error to the user. This results in a Timed out sending get_state error during the ‘Binding existing deployment’ stage. At this point, the operator is expected to use bosh cck command to determine why certain VMs are not accessible.
  4. Identify requested networking changes to the existing and new VMs
    • If network changes cannot be resolved (e.g. currently the Director does not support swapping of static IP reservations), return an error to the user
    • If network changes require more IPs than the deployment’s networks allow, return an error to the user
  5. Delete instance groups that are no longer specified by the deployment manifest
    • Issue unmount_disk Agent call for attached disks
    • Issue delete_vm CPI call for each VM
    • Orphan persistent disks
    • Update and propagate DNS records
  6. Create compilation worker VMs based on as specified by compilation section
    • Issue create_vm CPI call
  7. Determine release packages dependency graph and compile each package on compilation worker VMs
    • Issue compile_package Agent call for each package
  8. Delete all compilation worker VMs
    • Issue delete_vm CPI call
  9. Create empty VMs for new instance groups
    • Update and propagate DNS records
  10. Create empty VMs, for instance, groups that increased in instance size
    • Update and propagate DNS records
  11. Update each one of the instance groups:
    • The subset of instances (within an instance group) is selected to be updated first based on the update options for the instance group or global update options. That group of instances is called canaries.
    • Even if only one job or package changed in the instance group, stopping and starting procedure will apply to all of the jobs on the instances in that group. One of the future enhancements is to make this procedure more surgical and only affect jobs that have changed.
    1. Check if the instance previously existed
      • If no, select a VM and assign it to be this instance
      • If yes, check to see what has changed since last time it was updated
    2. Download updated jobs and packages onto the VM
      • Issue prepare Agent call
    3. Run drain and stop scripts to safely stop processes on the VM
      • Issue drain Agent call
      • Issue stop Agent call
    4. Take persistent disks snapshot associated with the job instance
      • Issue take_snapshot CPI call if Director has snapshotting enabled
    5. Check if the instance group still uses the same stemcell
      • If no, create a new VM based on a correct stemcell
        • Issue delete_vm CPI call
        • Issue create_vm CPI call
        • Update and propagate DNS records
      • If yes, do nothing
    6. Check if the instance group’s network configuration changed
      • If no, do nothing
      • If yes, reconfigure running VM to match new configuration
        • Issue delete_vm CPI call
        • Issue create_vm CPI call
        • Update and propagate DNS records
    7. Update DNS A record for this instance with new IP
    8. Check if the instance group’s persistent disk changed
      • If no, do nothing
      • If yes, create a new persistent disk with correct size and type and copy data from the old persistent disk
        • Issue create_disk CPI call for the new disk
        • Issue attach_disk CPI call on a new disk
        • Issue mount_disk Agent call on a new disk
        • Issue migrate_disk Agent call on a new disk
        • Orphan the old disk
    9. Configure VM to have new set of jobs
      • Issue apply Agent call
    10. Start processes on the VM and wait up to specified amount of time by the update_watch_time or canary_watch_time
      • Issue start Agent call
      • Issue get_state Agent call until job state is running or times out

Additional Information



Powered by Wolken Software