Core dump file of vim.tiny generated.

Products

• VMware NSX

Issue/Introduction

• You see an alarm similar to the following:
  
  An application on an edge node crashed, resulting in a core dump file of vim.tiny being generated.
  
  
• It is confirmed the core dump is generated when opening "/etc/default/kdump-tools" file using vim.tiny tool.
  
  Core was generated by `vim.tiny /etc/default/kdump-tools'.
Program terminated with signal SIGSEGV, Segmentation fault.

• The syslog indicates that the core dump is generated in the ssh session from the specific client.
  
  2024-08-24T10:23:31.610Z <HOSTNAME> kernel - - - [78725.005151] get_sigframe: 13 callbacks suppressed
2024-08-24T10:23:31.616Z <HOSTNAME> kernel - - - [78725.005153] signal: vim.tiny[57860] overflowed sigaltstack
2024-08-24T10:23:31.617Z <HOSTNAME> kernel - - - [78725.005156] signal: vim.tiny[57860] overflowed sigaltstack
 
2024-08-24T10:23:31.618Z <HOSTNAME> kernel - - - [78725.005164] grsec: From <CLIENT_IP_ADDRESS>: Segmentation fault occurred at 0000000000000000 in /usr/bin/vim.tiny[vim.tiny:57860] uid/euid:0/0 gid/egid:0/0, parent /[bash:57551] uid/euid:0/0 gid/egid:0/0
2024-08-24T10:23:31.948Z <HOSTNAME> NSX 691637 - [nsx@6876 comp="nsx-edge" subcomp="node-mgmt" username="root" level="WARNING"] Core file generated: /var/log/core/core.vim.tiny.1724495011.57860.0.11.gz
  
  
• The audit log also shows that the creation/close time of the ssh session.
  
  2024-08-23T13:37:47.769Z <HOSTNAME> sshd 57540 - -  Accepted password for root from <CLIENT_IP_ADDRESS> port 56526 ssh2
2024-08-23T13:37:47.547Z <HOSTNAME> sshd 57540 - -  pam_unix(sshd:session): session opened for user root by (uid=0)
2024-08-23T13:37:47.303Z <HOSTNAME> systemd-logind 3039 - -  New session <SESSION_ID> of user root.
...
2024-08-24T10:23:32.116Z <HOSTNAME> sshd 57540 - -  pam_unix(sshd:session): session closed for user root
2024-08-24T10:23:32.188Z <HOSTNAME> sshd 57540 - -  pam_systemd(sshd:session): Failed to release session: Interrupted system call
2024-08-24T10:23:32.149Z <HOSTNAME> systemd-logind 3039 - -  Removed session <SESSION_ID>

Environment

VMware NSX-T Data Center 3.2.3.0.1

Resolution

It is not recommended to do any operations using root user on NSX appliances, unless instructed by Broadcom support.

In the above Issue section, we can see the segmentation fault occurred during a remote session with: <CLIENT_IP_ADDRESS>: Segmentation fault occurred when the VIM application was used.

Then in the auth.log, we can see that it was an ssh session and the connection from the client IP address: Accepted password for root from <CLIENT_IP_ADDRESS> port 56526 ssh2

And the segmentation fault occurred nearly a day later, when the ssh session was interrupted and closed, at that time the VIM application was still open and then the ssh session interrupt lead to the core dump.