Support validation/initialization with types without the need to subclass

[davidroeca]

The only work-around I've found for my problem is to subclass types (including primitive/immutable types with the _new_ method). However, this doesn't help with collection or union types, or even boolean types, which cannot be subclassed.

Say I want to map input strings to output variables of explicit types: simplest case validate 'yes' or 'no' strings and convert to a bool.

In my code I might do the following:

from typing import Any, Type, Callable

def ValidatedType(
        name: str,
        parent_type: Type,
        validate_and_initialize: Callable[[Any], Any]
) -> Type:
    '''Follows standard library NewType and runs the validation function'''
    def validated_type(x):  # type: ignore
        return validate_and_initialize(x)
    validated_type.__name__ = name
    validated_type.__supertype__ = parent_type  # type: ignore
    return validated_type  # type: ignore

def validate_yes_no(value: Any) -> bool:
    value_clean = str(value).strip().lower()
    if value_clean == 'yes':
        return True
    elif value_clean == 'no':
        return False
    else:
        raise ValueError('Must be yes or no')

YesNo = ValidatedType('YesNo', bool, validate_yes_no)  # type: Type

def other_func(b: bool) -> None:
    print(b)

def print_yesno(inp: YesNo) -> None: # error: Invalid type "myfile.YesNo"
    other_func(inp)

print_yesno(YesNo('yes'))
print_yesno(YesNo('no'))
print_yesno(YesNo('Yes'))
print_yesno(YesNo('No'))

However, while running mypy with this approach, YesNo is invalid since MyPy follows NewType explicitly.

Here, I have to separate the validation step from the construction of the object. What I'd really like to do is write an extra step in the type/value conversion so that this sort of validation is possible.

[JelleZijlstra]

I'm having a hard time following this. Is the example the code you want to work or the code you're writing to work around your issue? What is the exact error you're getting from mypy?

[davidroeca]

This is a potential work-around to defining non-standard constructors for well-defined types, and I haven't found a general solution that works out of the box. I've commented the line with the error (print_yesno's function definition). YesNo isn't recognized as a type, since mypy merely checks for NewTypes, top-level class definitions, and a few other typing constructs.

If I wanted to work with strings, my solution of YesNo would be:

class YesNo(str):
    '''A string guaranteed to be yes or no'''

    def __new__(cls, value: Any):
        if value not in {'yes', 'no'}:
            raise ValueError('Must be yes or no')
        return super().__new__(cls, value)  # type: ignore

And then I can use the constructor of YesNo like the string constructor with some added validation, knowing anywhere in my code that the string will either be 'yes' or 'no' since there will be a ValueError on instantiation otherwise.

This breaks down if I want to use a non-standard constructor that validates input (e.g. a string), yielding an output for some basic type (e.g. bool, with no subclassing abilities). I'd also like to not have to subclass string, but in either case I'd have to make trade-offs.

An example of such a trade-off: Use NewType across the board. This prevents no one from casting the constructor unsafely, without validation:

from typing import NewType

YesNo = NewType('YesNo', str)
def yes_no_constructor(value: str) -> YesNo:
    '''Here, I am very hopeful no one calls the default YesNo() constructor anywhere
    outside of this function, since I don't know if they will have validated
    the string'''
    if value not in {'yes', 'no'}:
        raise ValueError('Must be yes or no')
    return YesNo(value)

input_value = input() # say it's '42'
unsafe_yes_no = YesNo(input_value) # YesNo('42')
good_yes_no = yes_no_constructor(input_value) # ValueError

[davidroeca]

One thought could be to let the user configure aliases that assign new type variables? That way a custom solution such as my ValidatedType function would just be recognized as generating a new type similarly to NewType.

Not sure if that would be a simple addition, though

[davidroeca]

Realizing that I can use a different design pattern with generics and input/output class properties. Still would be nice if the above were possible, though.

from typing import TypeVar, Generic
from abc import ABCMeta, abstractmethod

InputType = TypeVar('InputType')
OutputType = TypeVar('OutputType')

class Validator(Generic[InputType, OutputType], metaclass=ABCMeta):

    def __init__(self, value: InputType) -> None:
        '''Can also do runtime validation with the is_valid prop'''
        self.value = value

    @property
    @abstractmethod
    def is_valid(self) -> bool:
        '''Check validity of input'''

    @property
    @abstractmethod
    def output(self) -> OutputType:
        '''Handle the output'''

[JelleZijlstra]

Closing as I don't think this issue is going anywhere. You may be interested in https://pypi.org/project/phantom-types/