You will not lose any data in the archive by cleaning groups in your perspectives.
All data archives (snapshots of your data) will retain the perspectives as of the moment the snapshot/archive was taken. As a best practice, it is encouraged to remove empty groups in perspectives as empty groups do not have any assets.