Useless error message with very long pattern

[stpierre]

The error message when an element fails pattern validation is useless when the pattern is extremely long. It would be nice to have a way to override the default message, perhaps by using the description field in the type subschema. E.g.:

'thing': {
    'type': 'string',
    'pattern': '^\S*$',
    'description': 'any number of non-whitespace characters'
}

...might yield:

ValidationError("'foo bar' does not match pattern: any number of non-whitespace characters")

That overloads the use of the description field, though, so I'm not sure that's the best way, either. I'm happy to implement this, I just didn't want to put in a PR with an approach you wouldn't accept.

Thanks!

[Julian]

Hey! Thanks definitely for bringing this up :) I've definitely thought quickly about this before.

My general thoughts have always been that I don't like using the schema to interact with developer-level things, particularly in places where the actual schema value does not literally correspond with whatever we're using it for. I'd rather have those sorts of things be configuration on validator classes, but I also think I've resisted any attempts to add custom error messaging because jsonschema.exceptions.ValidationError is an exception class meant for library users, not end users. If you want error messages for your end users, there's no reason you can't wrap validation errors to look however you want, all the information in the message it gives is readily available for programmatic inspection, so you can construct messages that look exactly as you like depending on anything that fails. Of course if that isn't true, then addressing that is definitely something I'd be for (#119 is the only thing currently I'm aware of.)

However, if it's just a way to make things easier even for library users, that'd be lovely of course. How about just changing reprs to use repr.repr (== reprlib.repr) which knows how to truncate long values, instead of using repr? Would that get you close enough to where you want?

[stpierre]

Using repr.repr would definitely be an improvement. I'm thinking more of a case where you have a large, complex schema (or body of schemas) and you want to handle schema exceptions automatically. It's not reasonable to catch and introspect the ValidationError and then attempt to return a saner error message -- e.g., convert a big nasty regex to an English-language description of it. For instance:

try:
    validator.validate(...)
except jsonschema.exceptions.ValidationError:
    # now to figure out which of 100 regexes failed....

And, ironically, truncating the regex could make the problem worse -- consider two regexes that have the same first N characters, but differ after that. There'd be no way to figure out which one failed and convert the ValidationError to a friendlier error message.

That's why I'd really like to see some way to specify a natural language error within the (sub)schema document itself -- the relationship between the (complex) regex and its description is made obvious, and there's no need to do the kind of crazy inference between stringified regex and human-readable error message that I mocked out above.

That said, if truncating the regex is the only thing that can be done without committing gross JSON Schema abuse, then so be it -- that'd be an improvement other the current state.

[Julian]

Why isn't it practical to do that? What's wrong with:

DESCRIPTIONS = {
    one_regex : "Invalid foo bar",
    another_regex: "Invalid bar baz",
}

try:
    validator.validate(...)
except ValidationError as error:
    if error.validator == "pattern":
        raise InvalidStuff(DESCRIPTIONS.get(error.validator_value))

or if you want it in the schema, you can have it there, and then similarly:

try:
    validator.validate(...)
except ValidationError as error:
     # probably you actually want to traverse error.schema_path into your object, but however you want that to work
     raise MyException(error.schema.get("description or whatever other key you want to use"))

I see that people like doing this seemingly, my objection is basically just that there isn't something that means this in the schema, so I'm reluctant to define our own behavior (although in something like Julian/Seep I'd be probably less reluctant than in here).

Would love to hear more thoughts certainly :)

[thiagomarini]

I published a little package that formats error messages for end users https://pypi.org/project/json_payload_validator/