Large Amount of Binary Unknown Metadata

Products

• Carbon Black EDR (formerly Cb Response)

Issue/Introduction

Large amount of binaries are showing unknown metadata.

Environment

• EDR Server: All Supported Versions
• Microsoft Windows: All Supported Versions
• Linux: All Supported Versions

Cause

The binary file has not been seen for a while and was deleted by a run of the Modulestore_purge cron job. Since a record already exists, the binary metadata is not re-sent by the sensors.

Resolution

• If there are specific binaries from specific endpoints that are problematic, one possible workaround would be to delete the catalog file on those endpoints. This will cause the binary information to get uploaded the next time it is executed.
  • Windows: c:\windows\carbonblack\store\catalog
  • Linux: /var/opt/carbonblack/response/store/md5catalog.dat

Additional Information

• There is a limitation when using the modulestore_purge tool that can lead to binary metadata being missing from the server. Those reporting this problem are either using modulestore_purge currently or have used it in the past.
• Modulestore purge is an optional maintenance task that can be enabled on the EDR server. When enabled, it runs daily and deletes binary metadata information for binaries that have not been referenced in any process data for a given time threshold, typically 60 days. Modulestore purge will also delete the corresponding references to the binary from the Postgres database, and possibly the binary itself, depending on command line options specified and which node in the cluster it is being run on.
• When a sensor observes a new binary, it performs two levels of checks to determine if the binary information needs to be uploaded to the server. First, it checks the md5 against a local catalog file on the endpoint. If the md5 is present, that means it has been seen already and it is assumed that the server already has the information. If it is not present, the sensor makes an API call to the server to see if it already has the binary. If it doesn't, the binary and its metadata are uploaded.
• The problem arises if modulestore purge deletes binary information from the server, and then the binary is run again on an endpoint that already has the md5 in its local catalog. The endpoint is unaware that the server has deleted the binary information, and does not upload it again. The only way that the binary information would get uploaded to the server again would be if the binary is seen on an endpoint where it had not been observed before. There is currently no mechanism in the product that informs the endpoints that their cache may be incorrect.