Access log intermittently disconnects and reconnects to Reporters
Article ID: 370757
Updated On:
Products
Issue/Introduction
Access log get uploaded properly from time to time but you also see that it fails intermittently. This happens on only SCP Upload and only to Reporter.
2024-04-11 14:40:12-05:00CDT "Access Log (main): Unable to connect to remote server for log uploading" 0 E0008:1 alog_facility_impl.cpp:2792
2024-04-11 14:41:25-05:00CDT "Access Log (main): Upload completed successfully. Maximum bandwidth used was 9516.00 KB/sec." 0 E0009:96 alog_manager.cpp:1212
2024-04-11 14:41:25-05:00CDT "Access Log (main): Last remote filename: ThuApr11_1939.log.gz size: 19254342 bytes" 0 E0009:96 alog_manager.cpp:1218
2024-04-11 14:41:35-05:00CDT "Access Log (main): Upload completed successfully. Maximum bandwidth used was 1751.00 KB/sec." 0 E0009:96 alog_manager.cpp:1212
2024-04-11 14:41:35-05:00CDT "Access Log (main): Last remote filename: ThuApr11_1941.log.gz size: 2255751 bytes" 0 E0009:96 alog_manager.cpp:1218
2024-04-11 14:51:51-05:00CDT "Access Log (main): Upload completed successfully. Maximum bandwidth used was 10864.00 KB/sec." 0 E0009:96 alog_manager.cpp:1212
2024-04-11 14:51:51-05:00CDT "Access Log (main): Last remote filename: ThuApr11_1951.log.gz size: 100139858 bytes" 0 E0009:96 alog_manager.cpp:1218
2024-04-11 15:01:50-05:00CDT "Access Log (main): Upload completed successfully. Maximum bandwidth used was 10789.00 KB/sec." 0 E0009:96 alog_manager.cpp:1212
2024-04-11 15:01:50-05:00CDT "Access Log (main): Last remote filename: ThuApr11_2001.log.gz size: 93901807 bytes" 0 E0009:96 alog_manager.cpp:1218
2024-04-11 15:11:33-05:00CDT "Access Log (main): Unable to connect to remote server for log uploading" 0 E0008:1 Mailed alog_facility_impl.cpp:2792
2024-04-11 15:12:34-05:00CDT "Access Log (main): Unable to connect to remote server for log uploading" 0 E0008:1 alog_facility_impl.cpp:2792
2024-04-11 15:13:34-05:00CDT "Access Log (main): Unable to connect to remote server for log uploading" 0 E0008:1 alog_facility_impl.cpp:2792
2024-04-11 15:14:55-05:00CDT "Access Log (main): Upload completed successfully. Maximum bandwidth used was 10731.00 KB/sec." 0 E0009:96 alog_manager.cpp:1212
2024-04-11 15:14:55-05:00CDT "Access Log (main): Last remote filename: ThuApr11_2011.log.gz size: 99775870 bytes" 0 E0009:96 alog_manager.cpp:1218
2024-04-11 15:14:56-05:00CDT "Access Log (main): Unable to connect to remote server for log uploading" 0 E0008:1 Mailed alog_facility_impl.cpp:2792
2024-04-11 15:15:57-05:00CDT "Access Log (main): Unable to connect to remote server for log uploading" 0 E0008:1 alog_facility_impl.cpp:2792
2024-04-11 15:17:09-05:00CDT "Access Log (main): Upload completed successfully. Maximum bandwidth used was 9791.00 KB/sec." 0 E0009:96 alog_manager.cpp:1212
Cause
Running a packet capture on SCP port (default is 2024) finds the error below.
SSH-2.0-OpenSSH_7.2
Exceeded MaxStartups
The MaxStartup is a default configuration parameter in sshd_config for the SSH daemon in general. It tracks the number of unauthenticated connections globally to SSHD. The default value is 10 which is also configured in Reporter and it's unchangeable.
This issue happens when there are too many connections hitting Reporter saturating the 10 unauthenticated connection causing the failure intermittently.
Resolution
1. One way to resolved this is to find out (possibly by PCAP) any unnecessary traffic on the SCP port and stop it.
2. Manually decrease the access log upload interval following the steps below
a. Login to EdgeSWG console - Administration - Access logging
b. Edit the log (e.g main) and scroll to the part where you see "Wait Between Connect Attempts:"
c. Change the existing value larger (e.g from 15 to 25) and revisit accordingly
Note: This would delay the logs by value you set in c. above.
Feedback
