Using NSX-SM with VMware Enterprise PKS
search cancel

Using NSX-SM with VMware Enterprise PKS

book

Article ID: 314955

calendar_today

Updated On:

Products

VMware

Issue/Introduction

This article describes how to use NSX Service Mesh (NSX-SM) with VMware Enterprise PKS.

Prerequisites:
This topic assumes the following:

  • You have deployed VMware Enterprise PKS.

  • You have provisioned a Kubernetes cluster where you will install NSX-SM.

  • You enabled Pod Security Policies for the target Kubernetes cluster.



Resolution

About NSX-SM:

Authorized Kubernetes cluster users can deploy NSX Service Mesh (NSX-SM), giving you visibility, control, and security for services, data, and users at the API level. It provides a service mesh solution for Kubernetes based on the NSX platform. For more information, refer to the blog post Federated Service Mesh on VMware PKS.

NSX-SM is deployed as a Pod in a Kubernetes cluster using a YAML file. The recommendation is to deploy NSX-SM in an Enterprise PKS foundation with Pod Security Policies (PSPs) enabled. PSPs are enabled by selecting the check box in the appropriate Plan on the PKS tile.

Installing NSX-SM in the Cluster:

NSX-SM is provided as a service through VMware Cloud Services. You need to go through the service on-boarding process using the NSX-SM web site.

Register with VMware Cloud Services:

  1. Contact Sales contact to request access or send an email to Mark Schweighardt ([email protected]).

  2. Complete the registration process by following the emails you receive.

  3. Sign in to the VMware Cloud Services Console.

  4. Create a new organization.

  5. Select the NSX-SM service offering and add your account to the service.

Onboard a Kubernetes Cluster:

Complete the following steps to install NSX-SM onto a PKS-provisioned Kubernetes cluster.

  1. Once the account is sucessfully created, at the NSX-SM welcome page, select the ONBOARD CLUSTERS item on the home web page.

  2. At the Onboard Clusters screen, enter a unique name for the target cluster and click on Generate Security Token button. The cluster name is used by NSX-SM to identify a particular cluster. While the cluster name does not have to match the Enterprise PKS cluster name, it is recommended.

  3. Download the YAML file, which contains the deployment definition of the NSX-SM agent. Copy the command to apply the YAML by clicking the copy icon.

  4. Log in to your PKS-provisioned Kubernetes cluster and apply the registration YAML to the cluster by using the command you copied:
    kubectl apply -f https://prod-1.servicemesh.biz/cluster-registration/k8s/v0.8.5/k8s-registration.yaml.

    Then add the token to establish secure connection with NSX Service Mesh: 
    kubectl -n allspark create secret generic cluster-token --from-literal=token=eyJhbGciOiJSUzI1NiIsInR5cCI6IkpXVCJ9.xxxxxxxxxxxx

  5. Click on INSTALL NSX SERVICE MESH green button, to install NSX-SM on the cluster.

  6. The YAML file deploys a Pod to the target Kubernetes cluster that includes the NSX-SM agent. If the target cluster is not discovered, click Exit and Reload and try again.


Install and Configure Istio:

Once the NSX-SM agent is correctly started on a cluster, go back the NSX-SM web site and complete the on-boarding process by clicking on the Install ISTIO button in the on-boarding menu. This operation installs the ISTIO components on the target cluster. Included in these components is the ISTIO CNI plugin, which allows ISTIO to automatically inject its Envoy sidecar container whenever a new Pod is started.
Since PSPs are enabled on the cluster, deployment of services require a few additional steps, in particular you must create and use a PSP and grant use of it to a service account. While the additional steps are typical for any cluster with PSPs enabled, the content of the PSP object itself is specific to the needs of sidecar injection as requested by ISTIO. Use the psp.yaml file attached in this KB for this purpose. For instructions on how to create the PSP, see Enabling and Configuring Pod Security Policies.

Additional Information

Known Issue :
When the pks delete cluster command is issued, the system runs an errand to clean up the pods currently running in the cluster. Istio installs a few pods that have a Pod Disruption Budget that conflict with the Enterprise PKS cleanup errand. This means that the errand will run for a long time.

In Enterprise PKS v1.5 we allow the user to select a timeout for Pod Disruption Budget, therefore the errand will run up to that timeout. Prior to v1.5 the timeout was very long (approximately 24 hours) and it looked like the deleting process was hanging forever.

To avoid this problem we suggest that you first try removing the on-boarded cluster from the NSX-SM web UI. To perform this operation log on to the NSX-SM web UI and click on the name of cluster you want to remove. Near the top right corner click on: REMOVE CLUSTER. If this operation is successful the cluster can be safely deleted.

If the operation is not successful, run the following command on the cluster before attempting to delete it through Enterprise PKS:
kubectl delete namespace istio-system


Attachments

psp get_app
Powered by Wolken Software