How to Send an HTTP Request Directly to a GoRouter

Products

• VMware Tanzu Application Service for VMs

Issue/Introduction

Sometimes in order to diagnose where an issue is occurring it may be beneficial to bypass any proxies or load balancers and make HTTP requests directly to the Cloud Foundry GoRouters.

This can help to determine if the proxy/load balancer is the cause of an issue or if the issue exists inside Cloud Foundry.

 

Environment

Resolution

1. Get the FQDN of the app.

   $ cf app test-app
   Showing health and status for app test-app in org test / space test as admin...
   OK
   
   requested state: started
   instances: 1/1
   usage: 1G x 1 instances
   urls: test-app.apps.example.com
   last uploaded: Mon Nov 27 13:30:00 UTC 2017
   stack: cflinuxfs2
   buildpack: php 4.3.40
   
        state     since                    cpu    memory        disk           details
   #0   running   2017-11-27 01:31:34 PM   0.7%   24M of 1G     170.6M of 1G
   

2. Get the IP address of a GoRouter, router/0 will do:

   $ bosh vms
   | router/0 (394b1c87-7591-4def-90e5-c69fa744278e) | running | AZ01 | micro | 192.#.#.## |
   | router/1 (2857d5bc-ed37-435d-942d-8d2fb5fed1b2) | running | AZ01 | micro | 192.#.#.## |
   | router/2 (2931d750-da0d-480f-9c4c-43010daef38f) | running | AZ01 | micro | 192.#.#.## |

3. First use curl to make a request using the APP-FQDN which will follow the standard network flow through the proxy/load balancer:

   Example:
   curl -vvv test-app.apps.example.com
   
   Example output:
   *   Trying 192.#.#.###...
   * TCP_NODELAY set
   * Connected to test-app.apps.example.com (192.#.#.###) port 80 (#0)
   > GET / HTTP/1.1
   > Host: test-app.apps.example.com
   > User-Agent: curl/7.54.0
   > Accept: */*
   >
   < HTTP/1.1 403 Forbidden
   < Content-Type: text/html; charset=UTF-8
   < Date: Wed, 29 Nov 2017 16:49:28 GMT
   < Server: lighttpd
   < Vary: Accept-Encoding
   < X-Vcap-Request-Id: d9cba8be-840e-478a-73ab-9ad258fe6cef
   < Content-Length: 36
   <
   * Connection #0 to host test-app.apps.example.com left intact
   <p>Oops... something went wrong!</p>
   

4. Then use curl to construct a HTTP request targeting the application but send it directly to the GoRouter IP address:

   Syntax:
   curl -vvv -H 'Host: APP-FQDN' GOROUTER-IP
   
   Example:
   curl -vvv -H 'Host: test-app.apps.example.com' 192.#.#.##
   
   Example output:
   *   Trying 192.#.#.##...
   * TCP_NODELAY set
   * Connected to 192.#.#.##(192.#.#.##) port 80 (#0)
   > GET / HTTP/1.1
   > Host: test-app.apps.example.com
   > User-Agent: curl/7.54.0
   > Accept: */*
   >
   < HTTP/1.1 200 OK
   < Content-Type: text/html; charset=UTF-8
   < Date: Wed, 29 Nov 2017 16:45:02 GMT
   < Server: Apache
   < Vary: Accept-Encoding
   < X-Vcap-Request-Id: 94733c88-565e-4ff1-7e08-f4a9a9d9b812
   < Content-Length: 17
   <
   * Connection #0 to host 192.#.#.## left intact
   <p>Hello World</p>

5. Now if we compare the differences in the HTTP responses:
   1. Request 1 returned HTTP 403 and Request 2 returned HTTP 200.
   2. Request 1 returned "<p>Oops... something went wrong!</p>" where as the Request 2 "<p>Hello World</p>"
   3. Request 1 reported the server as lighttpd and Request 2 reported the server as Apache
   
   Lighttpd is not a standard part of Cloud Foundry so this would raise suspicions. The example here could be interpreted as a misconfigured proxy/load balancer which is forwarding the requests to a lighttpd server and the lighttpd server is expecting the requests to be authenticated.

This is just a very basic example of comparing a good and bad request. In a real situation the differences may be less obvious but the key is to look for anything which does not look normal in a Cloud Foundry environment.

 

Additional Information

; Environment:

Pivotal Elastic Runtime all versions