Postgresql log shows error like ERROR: invalid input syntax for type bytea

Products

• VMware Aria Suite
• SaltStack entitlement to VMware
• VMware SaltStack
• VMware Tanzu Spring Essentials

Issue/Introduction

• RaaS service logs show these errors:
  • sqlalchemy.exc.DataError: (psycopg2.errors.InvalidTextRepresentation) invalid input syntax for type bytea
  • CONTEXT:  PL/pgSQL function minion_cache_grains_sign() line 3 at assignment
• This is usually followed by a "INSERT INTO minion_cache" statement with a large blob of values below.
• You may also see an error in the log file like "Failed to save updated minion cache from master saltmaster.example.com".
• Additional symptoms may include target groups not updating in the UI, or no grain information / values for minions are empty. 

Environment

• Tanzu Salt - all versions
• Aria Config - all versions
• Aria Automation Config - all versions
• SaltProject - all versions

Cause

You or one of your users has likely configured a grain that is able to be rendered as valid YAML, but cannot be saved as bytea (byte array) data in the PSQL DB.

• This can happen if a grain contains a backslash \ character, such as in the newline escape sequence \n
• If a grain contains a multiline value, that is also not compatible with the database schema. An example of such a value might be something like this:

mycustomkey: 'mycustomentry: something
    myenv: envvalue
    anotherkey: theintendedvalue
    '

Notice in the error above that the multiline string itself contains what is probably valid YAML. Nonetheless, the newline character is disallowed in Postgres bytea data type.

Resolution

1. Take note of the minion(s) mentioned in the error, to review their grains first
2. Print out the grain values from the relevant minions:
   1. salt myminion1 grains.items | less
   2. Or for all minions:  salt '*' grains.items | less
3. Search for any backslash character in your custom grain values and remove or alter this value
4. Search for multi-line grains and update the custom grain value to be a single key/value pair that does not contain new line breaks
5. Search for other non-printable characters that may not be parseable into a byte array and edit out such characters.

Note: As an alternative, if the grain is not needed, it can be deleted entirely.

 

Multi-line grain example

Here is the same example as above, but configured properly on the minion.

mycustomkey:
  mycustomentry: something
  myenv: envvalue
  anotherkey: theintendedvalue

• Another valid example might remove the top key altogether like this:

mycustomentry: something
myenv: envvalue
anotherkey: theintendedvalue

• Note: Doing this would make these keys directly visible in the UI as selectable for organizing the target group view by end users.