Can't start GPCC instance with SSL enabled - port already in use
search cancel

Can't start GPCC instance with SSL enabled - port already in use

book

Article ID: 295886

calendar_today

Updated On:

Products

VMware Tanzu Greenplum

Issue/Introduction

Symptoms:

When attempting to start a GPCC instance that was created with SSL enabled, the gpcmdr command fails with an error message similar to ERROR: Port 28080 already in use.

Note: The port can be different depending on which one was specified during the instance setup.

Error Message:

Note: This is the output we get when we attempt to start the instance. 

     [gpadmin@localhost conf]$ gpcmdr --start my_ssl_instance
Starting instance dmg_dev_ss ...
ERROR: Port 28091 already in use
==Greenplum Command Center UI for instance 'my_ssl_instance' - [STOPPED; PORT: 28080]==


 

Environment


Cause

The error message indicates the port is already in use. However, this error message can be misleading in GPCC 3.2.1. We will have to dig deeper into the instance web server logs, by default /usr/local/gpcc_install_dir>/instances/<instance_name>/webserver/logs/gpmonws.log, to find out the exact problem.

2017/06/28 06:02:25 [Init.go:241][I] Log to /greenplum/greenplum-cc-web/instances/ssl_dmg_dev/webserver/logs/gpmonws.log
2017/06/28 06:02:25 [common.go:78][I] DatabaseTimeOffset -4h0m0.057196919s
2017/06/28 06:02:25 [Init.go:136][I] StrictHostnameMode: false
2017/06/28 06:02:25 [main.go:64][I] Static directory /greenplum/greenplum-cc-web/www/webserver/static
2017/06/28 06:02:25 [asm_amd64.s:1696][I] https server Running on :28090
2017/06/28 06:02:25 [asm_amd64.s:1696][C] ListenAndServeTLS: %!(EXTRA *errors.errorString=crypto/tls: failed to parse key PEM data)
2017/06/28 06:19:39 [Init.go:241][I] Log to /greenplum/greenplum-cc-web/instances/ssl_dmg_dev/webserver/logs/gpmonws.log
2017/06/28 06:19:39 [common.go:78][I] DatabaseTimeOffset -4h0m0.64602269s
2017/06/28 06:19:39 [Init.go:136][I] StrictHostnameMode: false
2017/06/28 06:19:39 [main.go:64][I] Static directory /greenplum/greenplum-cc-web/www/webserver/static
2017/06/28 06:19:39 [asm_amd64.s:1696][I] https server Running on :28090
2017/06/28 06:19:39 [asm_amd64.s:1696][C] ListenAndServeTLS: %!(EXTRA *errors.errorString=crypto/tls: failed to parse key PEM data)

In this example, the web server log is pointing out that there is a problem parsing the key in the pem file that was provided with the SSL certificate.

 

Resolution

Follow the steps below to resolve this issue:

  1. Check if the problem is a port already in use by entering lsof -i tcp:<port> (i.e. lsof-itcp:28080)
  2. If it's not a port already in use, confirm the actual problem using the web server logs as described in the RCA section above.

Note: In the provided above where the pem file key could not be parsed, the problem was due to the file format. The pem file contained both the SSL certificate and the private key, but there was no newline character between one and another so the parser couldn't find where the private key started and the certificate ended.

-----BEGIN CERTIFICATE-----
MIIDXTCCAkWgAwIBAgIJAJC1HiIAZAiIMA0GCSqGSIb3Df
BAYTAkFVMRMwEQYDVQQIDApTb21lLVN0YXRlMSEwHwYDVx
aWRnaXRzIFB0eSBMdGQwHhcNMTExMjMxMDg1OTQ0WhcNMT
A .... MANY LINES LIKE THAT ....
JjyzfN746vaInA1KxYEeI1Rx5KXY8zIdj6a7hhphpj2E04
C3Fayua4DRHyZOLmlvQ6tIChY0ClXXuefbmVSDeUHwc8Yu
B7xxt8BVc69rLeHV15A0qyx77CLSj3tCx2IUXVqRs5mlSb
vA==
-----END CERTIFICATE----------BEGIN ENCRYPTED PRIVATE KEY-----
MIIFDjBABgkqhkiG9w0BBQ0wMzAbBgkqhkiG9w0BBQwwDg
MBQGCCqGSIb3DQMHBAgD1kGN4ZslJgSCBMi1xk9jhlPxPc
9g73NQbtqZwI+9X5OhpSg/2ALxlCCjbqvzgSu8gfFZ4yo+
A .... MANY LINES LIKE THAT ....
X0R+meOaudPTBxoSgCCM51poFgaqt4l6VlTN4FRpj+c/Wc
blK948UAda/bWVmZjXfY4Tztah0CuqlAldOQBzu8TwE7WD
H0ga/iLNvWYexG7FHLRiq5hTj0g9mUPEbeTXuPtOkTEb/0
GEs=
-----END ENCRYPTED PRIVATE KEY-----

The above can't be parsed properly and should be formatted as follows:

-----BEGIN CERTIFICATE-----
MIIDXTCCAkWgAwIBAgIJAJC1HiIAZAiIMA0GCSqGSIb3Df
BAYTAkFVMRMwEQYDVQQIDApTb21lLVN0YXRlMSEwHwYDVx
aWRnaXRzIFB0eSBMdGQwHhcNMTExMjMxMDg1OTQ0WhcNMT
A .... MANY LINES LIKE THAT ....
JjyzfN746vaInA1KxYEeI1Rx5KXY8zIdj6a7hhphpj2E04
C3Fayua4DRHyZOLmlvQ6tIChY0ClXXuefbmVSDeUHwc8Yu
B7xxt8BVc69rLeHV15A0qyx77CLSj3tCx2IUXVqRs5mlSb
vA==
-----END CERTIFICATE-----
-----BEGIN ENCRYPTED PRIVATE KEY-----
MIIFDjBABgkqhkiG9w0BBQ0wMzAbBgkqhkiG9w0BBQwwDg
MBQGCCqGSIb3DQMHBAgD1kGN4ZslJgSCBMi1xk9jhlPxPc
9g73NQbtqZwI+9X5OhpSg/2ALxlCCjbqvzgSu8gfFZ4yo+
A .... MANY LINES LIKE THAT ....
X0R+meOaudPTBxoSgCCM51poFgaqt4l6VlTN4FRpj+c/Wc
blK948UAda/bWVmZjXfY4Tztah0CuqlAldOQBzu8TwE7WD
H0ga/iLNvWYexG7FHLRiq5hTj0g9mUPEbeTXuPtOkTEb/0
GEs=
-----END ENCRYPTED PRIVATE KEY-----

 


 

 


Powered by Wolken Software