You will not lose any data in the archive by cleaning groups in your perspectives.
All data archives (snapshots of your data) will retain the perspectives as of the moment the snapshot/archive was taken. As a best practice, we encourage customers to remove empty groups in their perspectives as empty groups do not have any assets.