The invalid character is something apart from the feedback message, so they need to be in quotation marks.
Also, this is very useful if the invalid character is a whitespace character, for example, which will look invisible without quotation marks.