api: Add new typed_endpoint decorators.

The goal of typed_endpoint is to replicate most features supported by
has_request_variables, and to improve on top of it. There are some
unresolved issues that we don't plan to work on currently. For example,
typed_endpoint does not support ignored_parameters_supported for 400
responses, and it does not run validators on path-only arguments.

Unlike has_request_variables, typed_endpoint supports error handling by
processing validation errors from Pydantic.

Most features supported by has_request_variables are supported by
typed_endpoint in various ways.

To define a function, use a syntax like this with Annotated if there is
any metadata you want to associate with a parameter, do note that
parameters that are not keyword-only are ignored from the request:
```
@typed_endpoint
def view(
    request: HttpRequest,
    user_profile: UserProfile,
    *,
    foo: Annotated[int, ApiParamConfig(path_only=True)],
    bar: Json[int],
    other: Annotated[
        Json[int],
        ApiParamConfig(
            whence="lorem",
            documentation_status=NTENTIONALLY_UNDOCUMENTED
        )
    ] = 10,
) -> HttpResponse:
    ....
```

There are also some shorthands for the commonly used annotated types,
which are encouraged when applicable for better readability and less
typing:
```
WebhookPayload = Annotated[Json[T], ApiParamConfig(argument_type_is_body=True)]
PathOnly = Annotated[T, ApiParamConfig(path_only=True)]
```

Then the view function above can be rewritten as:
```
@typed_endpoint
def view(
    request: HttpRequest,
    user_profile: UserProfile,
    *,
    foo: PathOnly[int],
    bar: Json[int],
    other: Annotated[
        Json[int],
        ApiParamConfig(
            whence="lorem",
            documentation_status=INTENTIONALLY_UNDOCUMENTED
        )
    ] = 10,
) -> HttpResponse:
    ....
```

There are some intentional restrictions:
- A single parameter cannot have more than one ApiParamConfig
- Path-only parameters cannot have default values
- argument_type_is_body is incompatible with whence
- Arguments of name "request", "user_profile", "args", and "kwargs" and
  etc. are ignored by typed_endpoint.
- positional-only arguments are not supported by typed_endpoint. Only
  keyword-only parameters are expected to be parsed from the request.
- Pydantic's strict mode is always enabled, because we don't want to
  coerce input parsed from JSON into other types unnecessarily.
- Using strict mode all the time also means that we should always use
  Json[int] instead of int, because it is only possible for the request
  to have data of type str, and a type annotation of int will always
  reject such data.

typed_endpoint's handling of ignored_parameters_unsupported is mostly
identical to that of has_request_variables.

Author: PIG208

tools/linter_lib/custom_check.py

@@ -23,6 +23,7 @@
     "zerver/lib/email_mirror.py",
     "zerver/lib/email_notifications.py",
     "zerver/lib/send_email.py",
+    "zerver/lib/typed_endpoint.py",
     "zerver/tests/test_new_users.py",
     "zerver/tests/test_email_mirror.py",
     "zerver/tests/test_message_notification_emails.py",

tools/semgrep.yml

@@ -169,3 +169,20 @@ rules:
     message: 'A batched migration should not be atomic. Add "atomic = False" to the Migration class'
     languages: [python]
     severity: ERROR
+
+  - id: typed_endpoint_without_keyword_only_param
+    patterns:
+      - pattern: |
+          @typed_endpoint
+          def $F(...)-> ...:
+            ...
+      - pattern-not-inside: |
+          @typed_endpoint
+          def $F(..., *, ...)-> ...:
+            ...
+    message: |
+      @typed_endpoint should not be used without keyword-only parameters.
+      Make parameters to be parsed from the request as keyword-only,
+      or use @typed_endpoint_without_parameters instead.
+    languages: [python]
+    severity: ERROR

zerver/lib/exceptions.py

@@ -527,3 +527,9 @@ def __init__(self) -> None:
     @staticmethod
     def msg_format() -> str:
         return _("Reaction doesn't exist.")
+
+
+class ApiParamValidationError(JsonableError):
+    def __init__(self, msg: str, error_type: str) -> None:
+        super().__init__(msg)
+        self.error_type = error_type