PSC 6.7 HA configuration for Resource vCenter Server with NSX-T Load Balancer
search cancel

PSC 6.7 HA configuration for Resource vCenter Server with NSX-T Load Balancer

book

Article ID: 321320

calendar_today

Updated On:

Products

VMware NSX VMware vCenter Server

Issue/Introduction

This Document is created based on the below vSphere and NSX-T product versions.

  • vSphere: 6.7
  • NSX-T: NSX-T Data Center



Environment

VMware vSphere 6.7
VMware NSX-T Data Center

Resolution

Following are the steps that are needed to perform and configure the Platform Service Controller High Availability for Resource vCenter Server with NSX-T 2.2 Load Balancer:

1. Deploy PSC01 and connect to MGMT VLAN port.
2. Deploy PSC02 & connect to MGMT VLAN port and join into existing domain by pointing to PSC01 (same site and domain).
3. Deploy and configure the Load Balancer for PSC’s with NSX-T bridge topology.

Prerequisites:

  • Set the MTU to 1600 for the vDS where the PSC and vCenter VMs are connected (MGMT VLAN port on DVS).
  • NSX-T Manager is up and running.
  • NSX-T Controllers are installed and configured.
  • NSX Edges are installed and configured (two nodes).
  • ESXi host (separate host) is installed and configured for Bridge cluster. (PSC & vCenter VMs should not run on this host).
  • Configured Transport zones (overlay and vlan backed), uplink profile and transport nodes (Edge’s and ESXi host).
  • Configure Edge cluster with two Edge Nodes.

Bridge Topology:

Background story to choose Bridge Topology for LB configuration:

The objective is to deploy PSC’s, vCenter and LB VIP on the same VLAN backed network and then configure the HA for both PSC’s by using NSX-T load balancer. But the NSX-T 2.2 does not support LB configuration when the Server (PSC’s), Client (vCenter) and LB VIP running on the same VLAN backed network because the LB is running on Tier-1 router and Tier-1 cannot be connected to a VLAN backed network.  

To achieve this limitation, use the below Bridge topology and configure the L2-Bridge between Overlay Logical switch and VLAN backed network.





Configure Bridge with NSX-T:

  1. In the NSX Manager UI, navigate to Fabric > Configuration > Bridges.
  2. Give the bridge cluster a name.
  3. Select a transport zone for the bridge cluster. The transport zone must be of type overlay, not VLAN.
  4. From the Available column, select transport nodes and click the right arrow to move them to the Selected column.



Create and configure Logical Switch (overlay):

  1. Navigate to Switching and click on “+ADD”.
  2. From “Add new Logical switch” wizard, provide the Logical switch name and select the “Overlay” transport zone & keep the default values and then click on “+ADD”.
  3. On the switch configuration page, select Related > Bridge Clusters.
  4. Click ATTACH, select a bridge cluster, and enter a VLAN ID.



Create and configure Tier-1 Router:

  1. In the NSX Manager UI, Select Routing from the navigation panel.
  2. Click Add and select Tier-1 Router.
  3. Assign a name for the logical router.
  4. (Optional) Select a tier-0 logical router to connect to this tier-1 logical router. If you do not yet have any tier-0 logical routers configured, you can leave this field blank for now and edit the router configuration later.
  5. Select an edge cluster to connect to this tier-1 logical router.
  6. Click Save.

Add Downlink Ports for the Tier-1 Logical Router:

a) Click the tier-1 logical router link to create ports.
b) Click the Configuration tab.
c) Click Add under the Logical Router Ports section.
d) Assign a name for the logical router port.
e) Select Type as “Downlink”.
f)  Select the Logical switch from drop down which is created in previous steps.
g) Enter the router port IP address in CIDR notation. (Since the logical switch is configured with bridge, Assign IP from the MGMT VLAN IP range) For example, the IP address can be 172.16.10.1/24.
h) Keep the remaining default settings.
i) Click Save.

Create and configure Tier-0 Router:

a) Select Routing from the navigation panel.
b) Click Add to create a tier-0 logical router.
c) Select Tier-0 Router from the drop-down menu.
d) Assign a name for the tier-0 logical router.
e) Select an existing edge cluster from the drop-down menu to back this tier-0 logical router.
f) (Optional) Select a high-availability mode.
g) Click Save.

Attach Tier-0 and Tier-1:

a) Select Routing from the navigation panel.
b) Select the tier-1 logical router.
c) From the Actions > click on “Connect to Tier-0 Router”.
d) Select the tier-0 logical router from the drop-down menu and click on “Connect”.

Configure an Uplink interface / port to Tier-0 Router:

a) Create a VLAN backed logical switch.

-> Navigate to Switching and click on “+ADD”.
-> From “Add new Logical switch” wizard, provide the Logical switch name and select the “VLAN backed” transport zone & assign VLAN ID as “0” and then click on “ADD”.

b) Connect a Tier-0 Logical Router to a VLAN Logical Switch.

a) Select Routing from the navigation panel.
b) Select the tier-0 logical router.
c) From the Configuration tab, add a new logical router port.
d) Type a name for the port, such as uplink.
e) Select the Uplink type.
f)  Select an edge transport node.
g) Select a VLAN logical switch.
h) Type an IP address in CIDR. (Assign any dummy IP here eg 172.168.16.1/24)
 
Create and Configure Load Balancer:

a) From your browser, log in to an NSX Manager.
b) Select Load Balancer > Add.
c) Enter a name and a description for the load balancer.
d) Select the load balancer virtual server size and click “OK”.

Attach the newly created load balancer to a Tier-1 logical router:

a) Select the load balancer and click Actions > Attach to a Logical Router.
b) Select an existing Tier-1 logical router from the drop-down menu. The Tier-1 router must be in the Active-Standby mode.
c) Click OK.

Create a Health Monitor:

a) Navigate to Load Balancing -> Server Pools -> Active Health Monitors.
b) Click “+ADD” to create a health monitor.
c) Monitor Properties -> Provide Name: “TCP”, select “LbTcpMonitor”.
                                   -> Health check protocol: “LbTcpMonitor”.
                                   -> Monitor Port: 443.
d) Click next and then finish.

Add a Server Pool for Load Balancing:

a) Navigate to Load Balancing > Server Pools > Server Pools > Click on “ADD”.
b) General Properties > Provide the Pool Name and select ROUND_ROBIN for Load Balancing Algorithm and click Next.
c) SNAT Translation > Translation Mode as “Auto Map” and click Next.
d) Pool Members > Select Membership Type as “Static”
e) Under Static Membership > Click “ADD” to add the pool members (PSC’s).

-> provide Member name and the IP of PSC1. Make sure the state is set to “Enabled”
Click “ADD” again to add the second member.
-> provide Member name and the IP of PSC2. Make sure the state is set to “Enabled”

e) Click Next and select “TCP” for Active Health Monitor.
f) Click Finish.

Configure Virtual Server:

a) From your browser, log in to an NSX Manager at https://NSXMGR_IP.
b) Navigate to Load Balancing > Virtual Servers > Virtual Servers and click on “ADD”
c) General Properties:
-> Provide Name for Virtual server.
-> Application Type: Select Layer 4 and then select “TCP” from drop down.
-> Application Profile: select “nsx-default-lb-fast-tcp-profile”.
Click Next
d) Virtual Server Identifiers: Provide the below details.
IP Address: provide the IP address for Virtual server.
Port: 443, 389, 636, 2012, 2014, 2020
Click Next.

e) Server Pool: Select the Server Pool which is created in previous step.
f) Load Balancing Profiles: select “nsx-default-source-ip-persistence-profile” for Source IP.

Configure Route Advertisement on a Tier-1 Logical Router:

a) From your browser, log in to an NSX Manager.
b) Select Routing and Click a tier-1 logical router.
c) Select Route Advertisement from the Routing drop-down menu.
d) Enable route advertisement by clicking Edit and making sure the Status button is Enabled.
e) Change to “Yes” for all the routes.
f) Click on “Save”.

Configure Static route on PSC VMs:

NSX-T Load balancer will use 100.64.#.# to perform the health check for Server Pool members(PSC’s). But there is no route for 100.64.#.# network on Server Pool members (PSC’s), so the LB will not be able to reach servers and it fail to perform the health check.

Use the below command to add the static route on both the PSC’s:

a) SSH to both the PSC’s and execute the below commands on both the PSC’s.

# ip route add 192.168.201.22 via 192.168.201.32 dev eth0
    Replace 192.168.201.22 with Virtual Server IP.
    Replace 192.168.201.32 with Tier-1 Down link Port IP.
# ip route add 100.64.##.##.1 via 192.168.201.32 dev eth0
    Replace 100.##.##.1 with Tier-1 “LinkedPort_Tier-0” port IP.
    Replace 192.168.201.32 with Tier-1 Down link Port IP.


Note: The above configuration is not persistent when the PSC got rebooted. Please use the below procedure to make persistent routes on both the PSC’s.

a) SSH to both the PSC’s and change the directory.

# cd /etc/systemd/network/

b) Add below highlighted new sections at the bottom of the file using vi editor:

# vi 10-eth0.network
[Match]
Name=eth0
[Network]
Gateway=192.168.201.253
Address=192.168.201.26/24
DHCP=no
[DHCP]
UseDNS=false
[Route]
Gateway=192.168.201.32
Destination=192.168.201.22
[Route]
Gateway=192.168.201.32
Destination=100.##.##.1

c) Restart the service.

# systemctl restart systemd-networkd
# route -n

After Configuring the Load Balancer by using NSX -T, please perform the below tasks to confirm that the LB is accessible from PSC’s and LB IP/FQDN is re directing to PSC home page when accessing from browser.

  1. SSH to both PSC’s (PSC01 and PSC02) and execute the below openssl command with Load Balancer IP / FQDN.

    openssl s_client -connect test-psclb.example.com:443

    This command should display the certificate (PSC01 by default)

    Output:

    # openssl s_client -connect test-psclb.example.com:443
    CONNECTED(00000003)
    depth=1 CN = CA, DC = vsphere, DC = local, C = US, ST = California, O = test-psc1.example.com, OU = VMware Engineering
    verify return:1
    depth=0 CN = test-psc1.example.com, C = US
    verify return:1
    ---
    Certificate chain
     0 s:/CN=test-psc1.example.com/C=US
       i:/CN=CA/DC=vsphere/DC=local/C=US/ST=California/O=test-psc1.example.com/OU=VMware Engineering
    ---
    Server certificate
    -----BEGIN CERTIFICATE-----
    ...
    -----END CERTIFICATE-----
    subject=/CN=test-psc1.example.com/C=US
    issuer=/CN=CA/DC=vsphere/DC=local/C=US/ST=California/O=test-psc1.example.com/OU=VMware Engineering
    ---
    No client certificate CA names sent
    Peer signing digest: SHA512
    Server Temp Key: ECDH, P-256, 256 bits
    ---
    SSL handshake has read 1515 bytes and written 433 bytes
    ---
    New, TLSv1/SSLv3, Cipher is ECDHE-RSA-AES256-GCM-SHA384
    Server public key is 2048 bit
    Secure Renegotiation IS supported
    Compression: NONE
    Expansion: NONE
    No ALPN negotiated
    SSL-Session:
        Protocol  : TLSv1.2
        Cipher    : ECDHE-RSA-AES256-GCM-SHA384
        Session-ID:
        Session-ID-ctx:
        Master-Key: #####################################################################################
        Key-Arg   : None
        PSK identity: None
        PSK identity hint: None
        SRP username: None
        Start Time: 1526024218
        Timeout   : 300 (sec)
        Verify return code: 0 (ok)
     
  2. From browser, try to access the Load Balancer IP / FQDN, URL: https://test-psclb.nfvra1.com. Should be able to access and display the PSC home page. If the above two steps are working as expected please proceed the below next tasks.

4. Preparing a certificate for PSC's.

a) Access PSC1 node from SSH and execute the below commands:

# mkdir /certs
# cd /certs/

Create a file by using the below command and copy the below content and modify the DNS entries.
        # vi psc_ha_csr_cfg.cfg
 [ req ]
distinguished_name = req_distinguished_name
encrypt_key = no
prompt = no
string_mask = nombstr
req_extensions = v3_req
[ v3_req ]
basicConstraints = CA:false
keyUsage = nonRepudiation, digitalSignature, keyEncipherment
subjectAltName = DNS:test-psc1.example.com.com, DNS:test-psc2.example.com, DNS:test-psclb.example.com

[ req_distinguished_name ]
countryName = US
stateOrProvinceName = CA
localityName = Palo Alto
0.organizationName = VMware
organizationalUnitName= NFV
commonName = test-psclb.example.com

b) Generate a Certificate Signing request and Private Key.

# openssl req -new -nodes -out /certs/psc-ha-vip.csr -newkey rsa:2048 -keyout /certs/psc-ha-vip.key -config /certs/psc_ha_csr_cfg.cfg

Generate a certificate from VMCA.

# openssl x509 -req -days 3650 -in /certs/psc-ha-vip.csr -out /certs/psc-ha-vip.crt -CA /var/lib/vmware/vmca/root.cer -CAkey /var/lib/vmware/vmca/privatekey.pem -extensions v3_req -CAcreateserial -extfile /certs/psc_ha_csr_cfg.cfg

# cp /var/lib/vmware/vmca/root.cer /certs/cachain.crt

c) Validate the certificate.

-->Verify that the Subject CN value is the correct Load Balanced FQDN.
-->Verify that all PSC FQDNs and Load Balanced FQDN are present in the DNS value.

# openssl x509 -in /certs/psc-ha-vip.crt -noout -text
    
5. Replacing the default Machine SSL Certificate.

a) Launch the Certificate-Manager and select Option 1, then select sub Option 2.

# /usr/lib/vmware-vmca/bin/certificate-manager.
Provide the paths to the psc-ha-vip.crt, psc-ha-vip.key and cachain.crt files created in the previous section.
Please provide valid custom certificate for Machine SSL.
File : /certs/psc-ha-vip.crt
Please provide valid custom key for Machine SSL.
File : /certs/psc-ha-vip.key
Please provide the signing certificate of the Machine SSL certificate
File : /certs/cachain.crt

b) Copy the /certs folder from PSC1 to PSC2.
From PSC1 # scp -r /certs/ root@PSC2 IP:/  
Repeat above step (a) for PSC2.

6. Run the PSC HA 6.5 Scripts.

updateSSOConfig.py:

Important: Ensure you perform this step on all PSCs(PSC1 & PSC2) participating in the HA instance.

# cd /usr/lib/vmware-sso/bin
# python updateSSOConfig.py --lb-fqdn=test-psclb.nfvra1.com

updateLsEndpoint.py:

Important: You only need to perform this step on a single PSC node. (peform on PSC1)

# cd /usr/lib/vmware-sso/bin

Run the updateLsEndpoint.py script, passing in your Load Balanced FQDN, Administrator User name and Administrator Password as variables.

# python UpdateLsEndpoint.py --lb-fqdn=test-psclb.example.com [email protected] --password=<password>

7. Validate PSC HA 6.7.

a) Run the following command to return the Site ID.

# python /usr/lib/vmidentity/tools/scripts/lstool.py get-site-id --url http://localhost:7080/lookupservice/sdk 2> /dev/null

b) Run the following command to list the cs.identity endpoints, passing in the site name recorded in the previous section.

# python /usr/lib/vmidentity/tools/scripts/lstool.py list --url http://localhost/lookupservice/sdk --site psc-lb --type cs.license 2> /dev/null | grep URL

A total of 8 cs.license endpoints should be updated with the PSC HA VIP (4 per PSC).

c) Verify the cs.identity endpoints. Run the following command to list the cs.identity endpoints, passing in the site name recorded earlier.

# python /usr/lib/vmidentity/tools/scripts/lstool.py list --url http://localhost/lookupservice/sdk --site psc-lb --type cs.identity 2> /dev/null | grep URL

A total of 16 cs.identity endpoints should be updated with the PSC HA VIP (8 per PSC).

8. Deploy vCenter Server 6.7 and point the vCenter server to Load Balancer FQDN instead of PSC FQDN.

 

Powered by Wolken Software