How to configure Data Flow Shell for an air-gap Spring Cloud Data Flow instance

Products

• Support Only for Spring

Issue/Introduction

The Spring Cloud Data Flow for VMware Tanzu cf CLI plugin only works when it can access "repo1.maven.org" or "repo.spring.io" to download the plugin JAR file that is linked in the Data Flow server's "About" API response.

The version of the shell-jar should always be on the same version as the Data Flow server. This is handled transparent by the SCDF plugin for TAS.

If the SCDF instance is upgraded on an air-gap environment, make sure you have the corresponding shell-jar for the data flow server.   
 

Environment

Product Version: 1.10

Resolution

Option 1:
This option is a workaround. It involves manually downloading the shell-jar configuring the plugin

a) Identify which version of shell jar to download
    This is the same version of the Data Flow Server. On the SCDF dashboard click on the "About" link (or it's the "encircled i" on the top right hand corner). 

For example image below version shown is 2.7.2

 
b) Download the shell jar for the version indicated in Step-a manually to a device that has internet access.

example:

wget https://repo.spring.io/release/org/springframework/cloud/spring-cloud-dataflow-shell/2.7.2/spring-cloud-dataflow-shell-2.7.2.jar

For reference:
https://dataflow.spring.io/docs/2.7.x/installation/local/manual/#downloading-server-jars

c) Transfer downloaded jar to the device the will access the data flow instance. Save it to directory path
"$HOME/.cf/spring-cloud-dataflow-for-pcf/cache"

note: $HOME is the home directory

d) Retrieve the MD5 hash - "VALUE_OF_MD5_HASH"
Example:

curl -k https://repo1.maven.org/maven2/org/springframework/cloud/spring-cloud-dataflow-shell/2.7.2/spring-cloud-dataflow-shell-2.7.2.jar.md5

97e0ebc14e54bda99c7376a388b273b4

e) Edit file called  ".cachedata"  in $HOME/.cf/spring-cloud-dataflow-for-pcf/cache directory to add entry as shown.

{
  "https://repo1.maven.org/maven2/org/springframework/cloud/spring-cloud-dataflow-shell/2.7.2/spring-cloud-dataflow-shell-2.7.2.jar": "\"VALUE_OF_MD5_HASH_1\"",
  "https://repo1.maven.org/maven2/org/springframework/cloud/spring-cloud-dataflow-shell/2.8.1/spring-cloud-dataflow-shell-2.8.1.jar": "\"VALUE_OF_MD5_HASH_2\""
}

f) Execute the spring cloud data flow shell plugin 
     $ cf dataflow-shell MY_DATAFLOW

Option 2:
This option basically runs the shell-jar  

a). Manually download and transfer the shell-jar to your device

b)  Determine the data flow server URL. Data flow URL consists of SERVICE-GUID and APPS-DOMAIN - 
      https://dataflow-<SERVICE-GUID>.<APPS-DOMAIN>

example: 

Retrieve the SERVICE-GUID of the MY_DATAFLOW service in user org/space

     cf target -o MY_ORG  -s MY_SPACE
     cf service MY_DATAFLOW --guid
     adb5d92c-bca6-479a-831f-1c7df3ca1f11

And APPS_DOMAIN is
      cfapps-06.slot-59.pez.vmware.com

Data Flow URI is:
https://dataflow-adb5d92c-bca6-479a-831f-1c7df3ca1f11.cfapps-06.slot-59.pez.vmware.com


c) Execute the jar with startup parameters
https://docs.spring.io/spring-cloud-dataflow/docs/current/reference/htmlsingle/#shell

example:

java -jar spring-cloud-dataflow-shell-2.7.2.jar \
--dataflow.uri=https://dataflow-adb5d92c-bca6-479a-831f-1c7df3ca1f11.cfapps-06.slot-59.pez.vmware.com \
--dataflow.skip-ssl-validation=true \
--dataflow.credentials-provider-command="cf oauth-token"