Redesign client for more flexibility via direct httpx access (#775)

* Prototype tighter httpx integration

* Blacken

* New client template

* Update templates & tests

* Update generated readmes

* Rename `*_client` functions to `*_httpx_client`

* Allow AuthenticatedClient anywhere a Client is allowed

* Use macros to keep Client/AuthenticatedClient the same

* Add changeset notes

* Add integration test for minimal httpx version

* Add mypy to integration tests

* Install lower httpx in the right place for integration tests

* Update every attrs to use new syntax, raise minimum httpx version

* More release dry runs and prerelease action

* Put back missing tabs

* Put back missing tabs

* Update end_to_end_tests/golden-record/my_test_api_client/client.py

Co-authored-by: Ethan Mann <[email protected]>

* Regen

---------

Co-authored-by: Dylan Anthony <[email protected]>
Co-authored-by: Ethan Mann <[email protected]>

Author: dbanty

Diff for: .changeset/allow_customizing_the_underlying_httpx_clients.md

@@ -0,0 +1,47 @@
+---
+default: minor
+---
+
+#### Allow customizing the underlying `httpx` clients
+
+There are many use-cases where customizing the underlying `httpx` client directly is necessary. Some examples are:
+
+- [Event hooks](https://www.python-httpx.org/advanced/#event-hooks)
+- [Proxies](https://www.python-httpx.org/advanced/#http-proxying)
+- [Custom authentication](https://www.python-httpx.org/advanced/#customizing-authentication)
+- [Retries](https://www.python-httpx.org/advanced/#usage_1)
+
+The new `Client` and `AuthenticatedClient` classes come with several methods to customize underlying clients. You can pass arbitrary arguments to `httpx.Client` or `httpx.AsyncClient` when they are constructed:
+
+```python
+client = Client(base_url="https://api.example.com", httpx_args={"proxies": {"https://": "https://proxy.example.com"}})
+```
+
+**The underlying clients are constructed lazily, only when needed. `httpx_args` are stored internally in a dictionary until the first request is made.**
+
+You can force immediate construction of an underlying client in order to edit it directly:
+
+```python
+import httpx
+from my_api import Client
+
+client = Client(base_url="https://api.example.com")
+sync_client: httpx.Client = client.get_httpx_client()
+sync_client.timeout = 10
+async_client = client.get_async_httpx_client()
+async_client.timeout = 15
+```
+
+You can also completely override the underlying clients:
+
+```python
+import httpx
+from my_api import Client
+
+client = Client(base_url="https://api.example.com")
+# The params you put in here ^ are discarded when you call set_httpx_client or set_async_httpx_client
+sync_client = httpx.Client(base_url="https://api.example.com", timeout=10)
+client.set_httpx_client(sync_client)
+async_client = httpx.AsyncClient(base_url="https://api.example.com", timeout=15)
+client.set_async_httpx_client(async_client)
+```

Diff for: .changeset/authenticatedclient_no_longer_inherits_from_client.md

@@ -0,0 +1,7 @@
+---
+default: major
+---
+
+#### `AuthenticatedClient` no longer inherits from `Client`
+
+The API of `AuthenticatedClient` is still a superset of `Client`, but the two classes no longer share a common base class.

Diff for: .changeset/client_and_authenticatedclient_now_use_the_newer_attrs_define_and_field_apis.md

@@ -0,0 +1,7 @@
+---
+default: major
+---
+
+#### Generated clients and models now use the newer attrs `@define` and `field` APIs
+
+See [the attrs docs](https://www.attrs.org/en/stable/names.html#attrs-tng) for more information on how these may affect you.

Diff for: .changeset/clients_now_reuse_connections_between_requests.md

@@ -0,0 +1,7 @@
+---
+default: minor
+---
+
+#### Clients now reuse connections between requests
+
+This happens every time you use the same `Client` or `AuthenticatedClient` instance for multiple requests, however it is best to use a context manager (e.g., `with client as client:`) to ensure the client is closed properly.

Diff for: .changeset/connections_dont_close.md

@@ -0,0 +1,18 @@
+---
+default: major
+---
+
+#### Connections from clients no longer automatically close (PR [#775](https://github.com/openapi-generators/openapi-python-client/pull/775))
+
+`Client` and `AuthenticatedClient` now reuse an internal [`httpx.Client`](https://www.python-httpx.org/advanced/#client-instances) (or `AsyncClient`)—keeping connections open between requests. This will improve performance overall, but may cause resource leaking if clients are not closed properly. The new clients are intended to be used via context managers—though for compatibility they don't _have_ to be used with context managers. If not using a context manager, connections will probably leak. Note that once a client is closed (by leaving the context manager), it can no longer be used—and attempting to do so will raise an exception.
+
+APIs should now be called like:
+
+```python
+with client as client:
+    my_api.sync(client)
+    another_api.sync(client)
+# client is closed here and can no longer be used
+```
+
+Generated READMEs reflect the new syntax, but READMEs for existing generated clients should be updated manually. See [this diff](https://github.com/openapi-generators/openapi-python-client/pull/775/files#diff-62b50316369f84439d58f4981c37538f5b619d344393cb659080dadbda328547) for inspiration.

Diff for: .changeset/minimum_httpx_version_raised_to_020.md

@@ -0,0 +1,7 @@
+---
+default: major
+---
+
+#### Minimum httpx version raised to 0.20
+
+Some features of generated clients already failed at runtime when using httpx < 0.20, but now the minimum version is enforced at generation time.

Diff for: .changeset/removed_public_attributes_for_client_and_authenticatedclient.md

@@ -0,0 +1,14 @@
+---
+default: major
+---
+
+#### Removed public attributes for `Client` and `AuthenticatedClient`
+
+The following attributes have been removed from `Client` and `AuthenticatedClient`:
+
+- `base_url`—this can now only be set via the initializer
+- `cookies`—set at initialization or use `.with_cookies()`
+- `headers`—set at initialization or use `.with_headers()`
+- `timeout`—set at initialization or use `.with_timeout()`
+- `verify_ssl`—this can now only be set via the initializer
+- `follow_redirects`—this can now only be set via the initializer

Diff for: .changeset/stop_showing_poetry_instructions_in_generated_readmes_when_not_appropriate.md

@@ -0,0 +1,5 @@
+---
+default: patch
+---
+
+#### Stop showing Poetry instructions in generated READMEs when not appropriate

Diff for: .changeset/the_timeout_param_and_with_timeout_now_take_an_httpxtimeout_instead_of_a_float.md

@@ -0,0 +1,5 @@
+---
+default: major
+---
+
+#### The `timeout` param and `with_timeout` now take an `httpx.Timeout` instead of a float

Diff for: .github/workflows/checks.yml

@@ -90,6 +90,11 @@ jobs:
   integration:
     name: Integration Tests
     runs-on: ubuntu-latest
+    strategy:
+      matrix:
+        httpx_version:
+          - "0.20.0"
+          - ""
     services:
       openapi-test-server:
         image: ghcr.io/openapi-generators/openapi-test-server:0.0.1
@@ -135,9 +140,15 @@ jobs:
           python -m venv .venv
           poetry run python -m pip install --upgrade pip
           poetry install
+      - name: Set httpx version
+        if: matrix.httpx_version != ''
+        run: |
+          cd integration-tests 
+          poetry run pip install httpx==${{ matrix.httpx_version }}
       - name: Run Tests
         run: |
           cd integration-tests
           poetry run pytest
+          poetry run mypy . --strict
   
 

Diff for: end_to_end_tests/golden-record/README.md

@@ -25,9 +25,10 @@ from my_test_api_client.models import MyDataModel
 from my_test_api_client.api.my_tag import get_my_data_model
 from my_test_api_client.types import Response
 
-my_data: MyDataModel = get_my_data_model.sync(client=client)
-# or if you need more info (e.g. status_code)
-response: Response[MyDataModel] = get_my_data_model.sync_detailed(client=client)
+with client as client:
+    my_data: MyDataModel = get_my_data_model.sync(client=client)
+    # or if you need more info (e.g. status_code)
+    response: Response[MyDataModel] = get_my_data_model.sync_detailed(client=client)
 ```
 
 Or do the same thing with an async version:
@@ -37,8 +38,9 @@ from my_test_api_client.models import MyDataModel
 from my_test_api_client.api.my_tag import get_my_data_model
 from my_test_api_client.types import Response
 
-my_data: MyDataModel = await get_my_data_model.asyncio(client=client)
-response: Response[MyDataModel] = await get_my_data_model.asyncio_detailed(client=client)
+async with client as client:
+    my_data: MyDataModel = await get_my_data_model.asyncio(client=client)
+    response: Response[MyDataModel] = await get_my_data_model.asyncio_detailed(client=client)
 ```
 
 By default, when you're calling an HTTPS API it will attempt to verify that SSL is working correctly. Using certificate verification is highly recommended most of the time, but sometimes you may need to authenticate to a server (especially an internal server) using a custom certificate bundle.
@@ -61,8 +63,6 @@ client = AuthenticatedClient(
 )
 ```
 
-There are more settings on the generated `Client` class which let you control more runtime behavior, check out the docstring on that class for more info.
-
 Things to know:
 1. Every path/method combo becomes a Python module with four functions:
     1. `sync`: Blocking request that returns parsed data (if successful) or `None`
@@ -74,7 +74,42 @@ Things to know:
 1. If your endpoint had any tags on it, the first tag will be used as a module name for the function (my_tag above)
 1. Any endpoint which did not have a tag will be in `my_test_api_client.api.default`
 
-## Building / publishing this Client
+## Advanced customizations
+
+There are more settings on the generated `Client` class which let you control more runtime behavior, check out the docstring on that class for more info. You can also customize the underlying `httpx.Client` or `httpx.AsyncClient` (depending on your use-case):
+
+```python
+from my_test_api_client import Client
+
+def log_request(request):
+    print(f"Request event hook: {request.method} {request.url} - Waiting for response")
+
+def log_response(response):
+    request = response.request
+    print(f"Response event hook: {request.method} {request.url} - Status {response.status_code}")
+
+client = Client(
+    base_url="https://api.example.com",
+    httpx_args={"event_hooks": {"request": [log_request], "response": [log_response]}},
+)
+
+# Or get the underlying httpx client to modify directly with client.get_httpx_client() or client.get_async_httpx_client()
+```
+
+You can even set the httpx client directly, but beware that this will override any existing settings (e.g., base_url):
+
+```python
+import httpx
+from my_test_api_client import Client
+
+client = Client(
+    base_url="https://api.example.com",
+)
+# Note that base_url needs to be re-set, as would any shared cookies, headers, etc.
+client.set_httpx_client(httpx.Client(base_url="https://api.example.com", proxies="http://localhost:8030"))
+```
+
+## Building / publishing this package
 This project uses [Poetry](https://python-poetry.org/) to manage dependencies  and packaging.  Here are the basics:
 1. Update the metadata in pyproject.toml (e.g. authors, version)
 1. If you're using a private repository, configure it with Poetry

Diff for: end_to_end_tests/golden-record/my_test_api_client/api/default/get_common_parameters.py

@@ -4,19 +4,15 @@
 import httpx
 
 from ... import errors
-from ...client import Client
+from ...client import AuthenticatedClient, Client
 from ...types import UNSET, Response, Unset
 
 
 def _get_kwargs(
     *,
-    client: Client,
     common: Union[Unset, None, str] = UNSET,
 ) -> Dict[str, Any]:
-    url = "{}/common_parameters".format(client.base_url)
-
-    headers: Dict[str, str] = client.get_headers()
-    cookies: Dict[str, Any] = client.get_cookies()
+    pass
 
     params: Dict[str, Any] = {}
     params["common"] = common
@@ -25,16 +21,12 @@ def _get_kwargs(
 
     return {
         "method": "get",
-        "url": url,
-        "headers": headers,
-        "cookies": cookies,
-        "timeout": client.get_timeout(),
-        "follow_redirects": client.follow_redirects,
+        "url": "/common_parameters",
         "params": params,
     }
 
 
-def _parse_response(*, client: Client, response: httpx.Response) -> Optional[Any]:
+def _parse_response(*, client: Union[AuthenticatedClient, Client], response: httpx.Response) -> Optional[Any]:
     if response.status_code == HTTPStatus.OK:
         return None
     if client.raise_on_unexpected_status:
@@ -43,7 +35,7 @@ def _parse_response(*, client: Client, response: httpx.Response) -> Optional[Any
         return None
 
 
-def _build_response(*, client: Client, response: httpx.Response) -> Response[Any]:
+def _build_response(*, client: Union[AuthenticatedClient, Client], response: httpx.Response) -> Response[Any]:
     return Response(
         status_code=HTTPStatus(response.status_code),
         content=response.content,
@@ -54,7 +46,7 @@ def _build_response(*, client: Client, response: httpx.Response) -> Response[Any
 
 def sync_detailed(
     *,
-    client: Client,
+    client: Union[AuthenticatedClient, Client],
     common: Union[Unset, None, str] = UNSET,
 ) -> Response[Any]:
     """
@@ -70,12 +62,10 @@ def sync_detailed(
     """
 
     kwargs = _get_kwargs(
-        client=client,
         common=common,
     )
 
-    response = httpx.request(
-        verify=client.verify_ssl,
+    response = client.get_httpx_client().request(
         **kwargs,
     )
 
@@ -84,7 +74,7 @@ def sync_detailed(
 
 async def asyncio_detailed(
     *,
-    client: Client,
+    client: Union[AuthenticatedClient, Client],
     common: Union[Unset, None, str] = UNSET,
 ) -> Response[Any]:
     """
@@ -100,11 +90,9 @@ async def asyncio_detailed(
     """
 
     kwargs = _get_kwargs(
-        client=client,
         common=common,
     )
 
-    async with httpx.AsyncClient(verify=client.verify_ssl) as _client:
-        response = await _client.request(**kwargs)
+    response = await client.get_async_httpx_client().request(**kwargs)
 
     return _build_response(client=client, response=response)