Python scripts failing on GPDB6 due to missing module from python lib (paramiko)

Products

• VMware Tanzu Greenplum

Issue/Introduction

The paramiko module has been removed from the GPDB 6 python library.

[gpadmin@mdw]-/usr/local/5.27.0/lib/python # ls -ltrh
lrwxr-xr-x 1 vmsadmin wheel 41 May 9 17:35 _posixsubprocess.so -> ../../exec/lib/python/_posixsubprocess.so
drwxr-xr-x 2 vmsadmin wheel 2.0K May 9 17:35 lockfile
lrwxr-xr-x 1 vmsadmin wheel 34 May 9 17:35 __init__.pyc -> ../../exec/lib/python/__init__.pyc
lrwxr-xr-x 1 vmsadmin wheel 33 May 9 17:35 __init__.py -> ../../exec/lib/python/__init__.py
drwxr-xr-x 11 vmsadmin wheel 4.0K May 9 17:35 gppylib
drwxr-xr-x 2 vmsadmin wheel 2.0K May 9 17:35 ecdsa
drwxr-xr-x 7 vmsadmin wheel 2.0K May 9 17:35 Crypto
drwxr-xr-x 2 vmsadmin wheel 2.0K May 9 17:35 yaml
lrwxr-xr-x 1 vmsadmin wheel 38 May 9 17:35 subprocess32.pyc -> ../../exec/lib/python/subprocess32.pyc
lrwxr-xr-x 1 vmsadmin wheel 37 May 9 17:35 subprocess32.py -> ../../exec/lib/python/subprocess32.py
lrwxr-xr-x 1 vmsadmin wheel 44 May 9 17:35 subprocess32-ChangeLog -> ../../exec/lib/python/subprocess32-ChangeLog
drwxr-xr-x 2 vmsadmin wheel 2.0K May 9 17:35 pygresql
drwxr-xr-x 3 vmsadmin wheel 2.0K May 9 17:35 psutil
drwxr-xr-x 2 vmsadmin wheel 4.0K May 9 17:35 paramiko


GP6 python modules with build( Missing paramiko module) :-

[gpadmin@mdw]-/usr/local//6.9.0/lib/python # ls -ltrh
lrwxr-xr-x 1 vmsadmin wheel 41 Jul 16 11:08 _posixsubprocess.so -> ../../exec/lib/python/_posixsubprocess.so
lrwxr-xr-x 1 vmsadmin wheel 34 Jul 16 11:08 __init__.pyc -> ../../exec/lib/python/__init__.pyc
lrwxr-xr-x 1 vmsadmin wheel 33 Jul 16 11:08 __init__.py -> ../../exec/lib/python/__init__.py
drwxr-xr-x 3 vmsadmin wheel 2.0K Jul 16 11:08 psutil
drwxr-xr-x 2 vmsadmin wheel 2.0K Jul 16 11:08 lockfile
drwxr-xr-x 10 vmsadmin wheel 4.0K Jul 16 11:08 gppylib
drwxr-xr-x 2 vmsadmin wheel 2.0K Jul 16 11:08 yaml
lrwxr-xr-x 1 vmsadmin wheel 38 Jul 16 11:08 subprocess32.pyc -> ../../exec/lib/python/subprocess32.pyc
lrwxr-xr-x 1 vmsadmin wheel 37 Jul 16 11:08 subprocess32.py -> ../../exec/lib/python/subprocess32.py
lrwxr-xr-x 1 vmsadmin wheel 44 Jul 16 11:08 subprocess32-ChangeLog -> ../../exec/lib/python/subprocess32-ChangeLog
drwxr-xr-x 2 vmsadmin wheel 2.0K Jul 16 11:08 pygresql

Because of this, any python scripts using the paramiko module will fail with the following error:

No module named paramiko

[gpadmin@mdw]-/var/gpadmin # python_script.pl
CRITICAL:root:Could not load MS python modules: No module named paramiko

Environment

Product Version: 6.9

Resolution

Confirmed solution with R&D: install pip and then paramiko

[gpadmin@gpdb6-m ~]$ curl "https://bootstrap.pypa.io/get-pip.py" -o "get-pip.py"
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100 1840k  100 1840k    0     0  2183k      0 --:--:-- --:--:-- --:--:-- 2186k
[gpadmin@gpdb6-m ~]$ python get-pip.py
DEPRECATION: Python 2.7 reached the end of its life on January 1st, 2020. Please upgrade your Python as Python 2.7 is no longer maintained. pip 21.0 will drop support for Python 2.7 in January 2021. More details about Python 2 support in pip can be found at https://pip.pypa.io/en/latest/development/release-process/#python-2-support
Defaulting to user installation because normal site-packages is not writeable
Collecting pip
  Downloading pip-20.2.2-py2.py3-none-any.whl (1.5 MB)
     |████████████████████████████████| 1.5 MB 1.3 MB/s
Collecting setuptools
  Downloading setuptools-44.1.1-py2.py3-none-any.whl (583 kB)
     |████████████████████████████████| 583 kB 4.3 MB/s
Collecting wheel
  Downloading wheel-0.35.1-py2.py3-none-any.whl (33 kB)
Installing collected packages: pip, setuptools, wheel
Successfully installed pip-20.2.2 setuptools-44.1.1 wheel-0.35.1
[gpadmin@gpdb6-m ~]$ pip install paramiko
DEPRECATION: Python 2.7 reached the end of its life on January 1st, 2020. Please upgrade your Python as Python 2.7 is no longer maintained. pip 21.0 will drop support for Python 2.7 in January 2021. More details about Python 2 support in pip can be found at https://pip.pypa.io/en/latest/development/release-process/#python-2-support
Defaulting to user installation because normal site-packages is not writeable
Collecting paramiko
  Downloading paramiko-2.7.1-py2.py3-none-any.whl (206 kB)
     |████████████████████████████████| 206 kB 1.5 MB/s
Collecting bcrypt>=3.1.3
  Downloading bcrypt-3.1.7-cp27-cp27mu-manylinux1_x86_64.whl (59 kB)
     |████████████████████████████████| 59 kB 4.2 MB/s
Collecting pynacl>=1.0.1
  Downloading PyNaCl-1.4.0-cp27-cp27mu-manylinux1_x86_64.whl (964 kB)
     |████████████████████████████████| 964 kB 5.4 MB/s
Collecting cryptography>=2.5
  Downloading cryptography-3.1-cp27-cp27mu-manylinux2010_x86_64.whl (2.6 MB)
     |████████████████████████████████| 2.6 MB 6.2 MB/s
Collecting six>=1.4.1
  Downloading six-1.15.0-py2.py3-none-any.whl (10 kB)
Collecting cffi>=1.1
  Downloading cffi-1.14.2-cp27-cp27mu-manylinux1_x86_64.whl (388 kB)
     |████████████████████████████████| 388 kB 4.8 MB/s
Collecting ipaddress; python_version < "3"
  Downloading ipaddress-1.0.23-py2.py3-none-any.whl (18 kB)
Collecting enum34; python_version < "3"
  Downloading enum34-1.1.10-py2-none-any.whl (11 kB)
Collecting pycparser
  Downloading pycparser-2.20-py2.py3-none-any.whl (112 kB)
     |████████████████████████████████| 112 kB 4.3 MB/s
Installing collected packages: six, pycparser, cffi, bcrypt, pynacl, ipaddress, enum34, cryptography, paramiko
Successfully installed bcrypt-3.1.7 cffi-1.14.2 cryptography-3.1 enum34-1.1.10 ipaddress-1.0.23 paramiko-2.7.1 pycparser-2.20 pynacl-1.4.0 six-1.15.0
[gpadmin@gpdb6-m ~]$ python -m site
sys.path = [
    '/home/gpadmin',
    '/usr/local/greenplum-db/lib/python',
    '/usr/local/greenplum-db/ext/python/lib/python27.zip',
    '/usr/local/greenplum-db/ext/python/lib/python2.7',
    '/usr/local/greenplum-db/ext/python/lib/python2.7/plat-linux2',
    '/usr/local/greenplum-db/ext/python/lib/python2.7/lib-tk',
    '/usr/local/greenplum-db/ext/python/lib/python2.7/lib-old',
    '/usr/local/greenplum-db/ext/python/lib/python2.7/lib-dynload',
    '/home/gpadmin/.local/lib/python2.7/site-packages',
    '/usr/local/greenplum-db/ext/python/lib/python2.7/site-packages',
]
USER_BASE: '/home/gpadmin/.local' (exists)
USER_SITE: '/home/gpadmin/.local/lib/python2.7/site-packages' (exists)
ENABLE_USER_SITE: True

[gpadmin@gpdb6-m ~]$ pip show paramiko
DEPRECATION: Python 2.7 reached the end of its life on January 1st, 2020. Please upgrade your Python as Python 2.7 is no longer maintained. pip 21.0 will drop support for Python 2.7 in January 2021. More details about Python 2 support in pip can be found at https://pip.pypa.io/en/latest/development/release-process/#python-2-support
Name: paramiko
Version: 2.7.1
Summary: SSH2 protocol library
Home-page: https://github.com/paramiko/paramiko/
Author: Jeff Forcier
Author-email: [email protected]
License: LGPL
Location: /home/gpadmin/.local/lib/python2.7/site-packages
Requires: bcrypt, pynacl, cryptography
Required-by:

Now you can import a paramiko module.

[gpadmin@gpdb6-m ~]$ python
Python 2.7.12 (default, Jun  2 2020, 23:33:09)
[GCC 4.8.5 20150623 (Red Hat 4.8.5-39)] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> import os
>>> import paramiko
/home/gpadmin/.local/lib/python2.7/site-packages/paramiko/transport.py:33: CryptographyDeprecationWarning: Python 2 is no longer supported by the Python core team. Support for it is now deprecated in cryptography, and will be removed in a future release.
  from cryptography.hazmat.backends import default_backend
>>>