DUAS: corrupted data files (wrong object names) after system reboot
Article ID: 203665
Updated On:
Products
Issue/Introduction
Dollar Universe node looks corrupted after system patching/reboot. UPROC names got changed.
For example, UPROC names are displayed as U000000304 instead of the real object names.
| 2020-11-18 15:19:36 |ERROR|X|IO |pid=91337.139839638861568| xn_get_extended_name | O_TYPE_UPROC name not found for ID <U000000303>
| 2020-11-18 15:19:36 |ERROR|X|IO |pid=91337.139839638861568| o_io_cache_data_provider_ | Error getting Name for <U000000303>: 600
| 2020-11-18 15:19:36 |ERROR|X|IO |pid=91337.139839638861568| owls_dua_rawdata_populate | cannot find UPROC name for [U000000303]
| 2020-11-18 15:19:36 |ERROR|X|IO |pid=91337.139839996184320| xn_get_extended_name | O_TYPE_UPROC name not found for ID <U000000250>
| 2020-11-18 15:19:36 |ERROR|X|IO |pid=91337.139839996184320| o_io_cache_data_provider_ | Error getting Name for <U000000250>: 600
| 2020-11-18 15:19:36 |ERROR|X|IO |pid=91337.139839996184320| owls_dua_rawdata_populate | cannot find UPROC name for [U000000250]
| 2020-11-18 15:19:36 |ERROR|X|IO |pid=91337.139839638861568| xn_get_extended_name | O_TYPE_UPROC name not found for ID <U000000250>
This occurred after a system reboot that had not stopped the Dollar Universe node properly.
In universe.log, we observe that a reorganization was launched twice upon system startup, one by root and one by univadm:
| 2020-11-18 12:17:48 |INFO |X|IO |pid=3080.139726513194816| unireorg | Offline reorg for general area
| 2020-11-18 12:17:48 |INFO |X|IO |pid=3119.139646998218560| unireorg | Offline reorg for general area
| 2020-11-18 12:17:49 |ERROR|X|UTL|pid=3254.139852411848512| u_ouv_serv | failed in u_listen: Errno syserror 98: Address already in use
| 2020-11-18 12:17:49 |ERROR|X|UTL|pid=3331.139720659244864| u_ouv_serv | failed in u_listen: Errno syserror 98: Address already in use
| 2020-11-18 12:17:49 |ERROR|X|UTL|pid=3494.140080624981824| u_ouv_serv | failed in u_listen: Errno syserror 98: Address already in use
| 2020-11-18 12:17:49 |ERROR|X|UTL|pid=3545.139650321581888| u_ouv_serv | failed in u_listen: Errno syserror 98: Address already in use
| 2020-11-18 12:17:49 |ERROR|X|UTL|pid=3623.140066815047488| u_ouv_serv | failed in u_listen: Errno syserror 98: Address already in use
| 2020-11-18 12:17:49 |INFO |X|IO |pid=3631.140336259917632| unireorg | End of offline reorg for general area
| 2020-11-18 12:17:49 |INFO |X|IO |pid=3633.140203593914176| unireorg | End of offline reorg for general area
| 2020-11-18 12:17:49 |INFO |X|uni|pid=3634.140351965685568| u_init_srvio | IO server system user: [root]
| 2020-11-18 12:17:49 |INFO |X|uni|pid=3635.140033262761792| u_init_srvio | IO server system user: [univadm]
| 2020-11-18 12:17:49 |INFO |X|IO |pid=3636.140072938424128| unireorg | Offline reorg for area X
| 2020-11-18 12:17:49 |INFO |X|IO |pid=3637.140034240890688| unireorg | Offline reorg for area X
| 2020-11-18 12:17:51 |INFO |X|IO |pid=3993.140087319811904| unireorg | End of offline reorg for area X
Environment
Release : 6.x
Component : DOLLAR UNIVERSE
Cause
System configuration error:
There were two different init scripts for the same application that were executed ( one in /etc/init.d and another one related to Redhat Systemd).
Resolution
Reboot of the system caused that both init.d script and systemd script were launched upon server startup triggering two reorganization offline at the same time that corrupted the data files.
Workaround: restore filesystem to a backup taken before the reboot of the system was performed and launch an offline reorganization.
Solution: remove the Dollar Universe init.d script if you are using an alternative systemd script.
Feedback
