vCenter Server /storage/core filling up with core.in:imfile.#### or core.in:imuxsock.#### when configuring vCenter syslog server
search cancel

vCenter Server /storage/core filling up with core.in:imfile.#### or core.in:imuxsock.#### when configuring vCenter syslog server

book

Article ID: 318159

calendar_today

Updated On:

Products

VMware vCenter Server

Issue/Introduction

  • After configuring a remote syslog server, the following error messages appear in the /var/log/vmware/rsyslogd/rsyslogd-syslog.log

err rsyslogd imfile error trying to access state file for '/var/log/vmware/sso/tomcat/localhost_access.<YYYY-MM-DD>.log': Too many open files [v8.2001.0 try https://www.rsyslog.com/e/2027 ]
err rsyslogd file '/var/log/vmware/sso/tomcat/localhost_access.<YYYY-MM-DD>.log': open error: Too many open files [v8.2001.0 try https://www.rsyslog.com/e/2433 ]
err rsyslogd imfile: error accessing file '/var/log/vmware/sso/tomcat/localhost_access.<YYYY-MM-DD>.log': Too many open files [v8.2001.0]
err rsyslogd imfile: error accessing file '/var/log/vmware/sso/tomcat/localhost_access.<YYYY-MM-DD>.log': Too many open files [v8.2001.0]
err rsyslogd imfile: error accessing file '/var/log/vmware/vapi/endpoint/vmware-vapi-endpoint. stdout-20250726031001': Too many open files v8.2306.0]
err rsyslogd imfile: error accessing file '/var/log/vmware/vapi/endpoint/vmware-vapi-endpoint. stdout-20250726031001': Too many open files v8.2306.0]

  • Additionally, multiple core files such as core.in:imfile.#### or core.in:imuxsock.#### are found under vCenter /storage/core/.

total 50738008
-r-x-  1 root   root  218468352 MMM DD HH:MM core.in: imuxsock.10087
-r-x-  1 root   root  218468232 MMM DD HH:MM core.in: imuxsock.1017
-r-x-  1 root   root  218468452 MMM DD HH:MM core.in: imuxsock.10195
-r-x-  1 root   root  218468345 MMM DD HH:MM core.in: imuxsock.10475
-r-x-  1 root   root  218468367 MMM DD HH:MM core.in: imuxsock.10833
-r-x-  1 root   root  218468334 MMM DD HH:MM core.in: imuxsock.11547
-r-x-  1 root   root  218468378 MMM DD HH:MM core.in: imuxsock.11632
-r-x-  1 root   root  218468345 MMM DD HH:MM core.in: imuxsock.11648

Environment

  • VMware vCenter Server 7.0.x
  • VMware vCenter Server 8.0.x

Cause

The issue occurs due to a file descriptor (FD) limit being exceeded by the rsyslog service, especially when monitoring a large number of files.

Resolution

This is a known issue in vCenter Server 7.0. To permanently fix the problem, upgrade to vCenter Server 7.0 U3l or newer.

To download the latest version of vCenter Server Appliance (vCSA), refer to the My VMware Downloads portal My Downloads.

Note: Ensure there is valid backup/offline snapshot of the VCSA prior to implementing the workaround. Refer to VMware vCenter in Enhanced Linked Mode pre-changes snapshot (online or offline) best practice.

Workarounds

Workaround 1: Reconfigure the postgres-archiver rsyslog input

  1. Backup the existing configuration

    cp /etc/vmware-syslog/vmware-services-vmware-postgres-archiver.conf /etc/vmware-syslog/vmware-services-vmware-postgres-archiver.conf.orig

  2. Edit the configuration file

    vi /etc/vmware-syslog/vmware-services-vmware-postgres-archiver.conf

  3. Replace the contents with the following:

    #vmware-postgres-archiver logs
    input(type="imfile"
    File="/var/log/vmware/vpostgres/pg_archiver.log.stdout"
    Tag="postgres-archiver"
    Severity="info"
    Facility="local0")
    input(type="imfile"
    File="/var/log/vmware/vpostgres/pg_archiver.log.stderr"
    Tag="postgres-archiver"
    Severity="info"
    Facility="local0")

  4. Restart rsyslog

    systemctl restart rsyslog

  5. Verify the service status:

    systemctl status rsyslog

    syslog.service - System Logging Service
    Loaded: loaded (/lib/systemd/system/rsyslog.service; enabled; vendor preset: enabled)
    Active: active (running) since Fri YYYY-MM-DD HH:MM:SS UTC; 32s ago
    Docs: man:rsyslogd(8)
    http://www.rsyslog.com/doc/
    Main PID: 38846 (rsyslogd)
    Tasks: 13 (limit: 9830)
    Memory: 5.3M
    CGroup: /system.slice/rsyslog.service
    └─38846 /usr/sbin/rsyslogd -n

Workaround 2: Remove core files and schedule cleanup

  1. Stop rsyslog

    systemctl stop rsyslog

  2. Delete existing core files

    rm -f /storage/core/core.in:imfile*

    rm -f /storage/core/core.in:imuxsock*

  3. Clean up old access logs:

    find /var/log/vmware/ -mtime +1 -type f -name "localhost*access*log*" | while read file; do rm "$file"; done

  4. (Optional) Automate cleanup with a cron job
    1. Create a script:

      vi /root/cleanupAccessLog.sh


    2. Add the following:

      !/bin/bash
      find /var/log/vmware/ -mtime +1 -type f -name "localhost*access*log*" | while read file; do rm "$file"; done
      find /storage/core/ -mtime +1 -type f -name "core.in:imfile*" | while read file; do rm "$file"; done
      find /storage/core/ -mtime +1 -type f -name "core.in:imuxsock*" | while read file; do rm "$file"; done

    3. Make the script executable

      chmod 755 /root/cleanupAccessLog.sh


    4. Schedule with cron

      vi /etc/cron.d/cleanAccessLog.cron

    5. Add:

      0 0 * * 0 root /root/cleanupAccessLog.sh 2>&1


    6. Restart rsyslog

      systemctl start rsyslog


If the issue still persists, after following the above steps. Follow the resolution steps from KB core.in:imuxsock.xxxxx core dump files filling up /dev/mapper/core_vg-core mounted on /storage/core in vCenter.