The tenant restrictions headers in Google Workspace are designed to force the end user to access a list of whitelisted tenants. However, in the case of the publicly available objects, the restriction is not enforced since it requires the user to be authenticated.
This article describe a way to restrict this type of access.
X-GoogApps-Drive-Deny-Anonymous : true