Unchecking "Enable scanning of non-plain text attachments for words in dictionaries" also prevents scanning of plain text attachments in Symantec Messaging Gateway 10.5.2 and previous. The reference to "non-plain text" can be misleading to how the feature actually works.
This is current working behavior. Symantec has updated the verbiage in SMG 10.5.3 to more accurately describe the function of this feature. The new comment for this feature reads: "Enable scanning of attachments for words in dictionaries"