Skip to content

Redesign client for more flexibility via direct httpx access #775

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 19 commits into from
Jul 23, 2023
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
47 changes: 47 additions & 0 deletions .changeset/allow_customizing_the_underlying_httpx_clients.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,47 @@
---
default: minor
---

#### Allow customizing the underlying `httpx` clients

There are many use-cases where customizing the underlying `httpx` client directly is necessary. Some examples are:

- [Event hooks](https://www.python-httpx.org/advanced/#event-hooks)
- [Proxies](https://www.python-httpx.org/advanced/#http-proxying)
- [Custom authentication](https://www.python-httpx.org/advanced/#customizing-authentication)
- [Retries](https://www.python-httpx.org/advanced/#usage_1)

The new `Client` and `AuthenticatedClient` classes come with several methods to customize underlying clients. You can pass arbitrary arguments to `httpx.Client` or `httpx.AsyncClient` when they are constructed:

```python
client = Client(base_url="https://api.example.com", httpx_args={"proxies": {"https://": "https://proxy.example.com"}})
```

**The underlying clients are constructed lazily, only when needed. `httpx_args` are stored internally in a dictionary until the first request is made.**

You can force immediate construction of an underlying client in order to edit it directly:

```python
import httpx
from my_api import Client

client = Client(base_url="https://api.example.com")
sync_client: httpx.Client = client.get_httpx_client()
sync_client.timeout = 10
async_client = client.get_async_httpx_client()
async_client.timeout = 15
```

You can also completely override the underlying clients:

```python
import httpx
from my_api import Client

client = Client(base_url="https://api.example.com")
# The params you put in here ^ are discarded when you call set_httpx_client or set_async_httpx_client
sync_client = httpx.Client(base_url="https://api.example.com", timeout=10)
client.set_httpx_client(sync_client)
async_client = httpx.AsyncClient(base_url="https://api.example.com", timeout=15)
client.set_async_httpx_client(async_client)
```
Original file line number Diff line number Diff line change
@@ -0,0 +1,7 @@
---
default: major
---

#### `AuthenticatedClient` no longer inherits from `Client`

The API of `AuthenticatedClient` is still a superset of `Client`, but the two classes no longer share a common base class.
Original file line number Diff line number Diff line change
@@ -0,0 +1,7 @@
---
default: major
---

#### Generated clients and models now use the newer attrs `@define` and `field` APIs

See [the attrs docs](https://www.attrs.org/en/stable/names.html#attrs-tng) for more information on how these may affect you.
7 changes: 7 additions & 0 deletions .changeset/clients_now_reuse_connections_between_requests.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,7 @@
---
default: minor
---

#### Clients now reuse connections between requests

This happens every time you use the same `Client` or `AuthenticatedClient` instance for multiple requests, however it is best to use a context manager (e.g., `with client as client:`) to ensure the client is closed properly.
18 changes: 18 additions & 0 deletions .changeset/connections_dont_close.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,18 @@
---
default: major
---

#### Connections from clients no longer automatically close (PR [#775](https://github.com/openapi-generators/openapi-python-client/pull/775))

`Client` and `AuthenticatedClient` now reuse an internal [`httpx.Client`](https://www.python-httpx.org/advanced/#client-instances) (or `AsyncClient`)—keeping connections open between requests. This will improve performance overall, but may cause resource leaking if clients are not closed properly. The new clients are intended to be used via context managers—though for compatibility they don't _have_ to be used with context managers. If not using a context manager, connections will probably leak. Note that once a client is closed (by leaving the context manager), it can no longer be used—and attempting to do so will raise an exception.

APIs should now be called like:

```python
with client as client:
my_api.sync(client)
another_api.sync(client)
# client is closed here and can no longer be used
```

Generated READMEs reflect the new syntax, but READMEs for existing generated clients should be updated manually. See [this diff](https://github.com/openapi-generators/openapi-python-client/pull/775/files#diff-62b50316369f84439d58f4981c37538f5b619d344393cb659080dadbda328547) for inspiration.
7 changes: 7 additions & 0 deletions .changeset/minimum_httpx_version_raised_to_020.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,7 @@
---
default: major
---

#### Minimum httpx version raised to 0.20

Some features of generated clients already failed at runtime when using httpx < 0.20, but now the minimum version is enforced at generation time.
Original file line number Diff line number Diff line change
@@ -0,0 +1,14 @@
---
default: major
---

#### Removed public attributes for `Client` and `AuthenticatedClient`

The following attributes have been removed from `Client` and `AuthenticatedClient`:

- `base_url`—this can now only be set via the initializer
- `cookies`—set at initialization or use `.with_cookies()`
- `headers`—set at initialization or use `.with_headers()`
- `timeout`—set at initialization or use `.with_timeout()`
- `verify_ssl`—this can now only be set via the initializer
- `follow_redirects`—this can now only be set via the initializer
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
---
default: patch
---

#### Stop showing Poetry instructions in generated READMEs when not appropriate
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
---
default: major
---

#### The `timeout` param and `with_timeout` now take an `httpx.Timeout` instead of a float
11 changes: 11 additions & 0 deletions .github/workflows/checks.yml
Original file line number Diff line number Diff line change
Expand Up @@ -90,6 +90,11 @@ jobs:
integration:
name: Integration Tests
runs-on: ubuntu-latest
strategy:
matrix:
httpx_version:
- "0.20.0"
- ""
services:
openapi-test-server:
image: ghcr.io/openapi-generators/openapi-test-server:0.0.1
Expand Down Expand Up @@ -135,9 +140,15 @@ jobs:
python -m venv .venv
poetry run python -m pip install --upgrade pip
poetry install
- name: Set httpx version
if: matrix.httpx_version != ''
run: |
cd integration-tests
poetry run pip install httpx==${{ matrix.httpx_version }}
- name: Run Tests
run: |
cd integration-tests
poetry run pytest
poetry run mypy . --strict


51 changes: 43 additions & 8 deletions end_to_end_tests/golden-record/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -25,9 +25,10 @@ from my_test_api_client.models import MyDataModel
from my_test_api_client.api.my_tag import get_my_data_model
from my_test_api_client.types import Response

my_data: MyDataModel = get_my_data_model.sync(client=client)
# or if you need more info (e.g. status_code)
response: Response[MyDataModel] = get_my_data_model.sync_detailed(client=client)
with client as client:
my_data: MyDataModel = get_my_data_model.sync(client=client)
# or if you need more info (e.g. status_code)
response: Response[MyDataModel] = get_my_data_model.sync_detailed(client=client)
```

Or do the same thing with an async version:
Expand All @@ -37,8 +38,9 @@ from my_test_api_client.models import MyDataModel
from my_test_api_client.api.my_tag import get_my_data_model
from my_test_api_client.types import Response

my_data: MyDataModel = await get_my_data_model.asyncio(client=client)
response: Response[MyDataModel] = await get_my_data_model.asyncio_detailed(client=client)
async with client as client:
my_data: MyDataModel = await get_my_data_model.asyncio(client=client)
response: Response[MyDataModel] = await get_my_data_model.asyncio_detailed(client=client)
```

By default, when you're calling an HTTPS API it will attempt to verify that SSL is working correctly. Using certificate verification is highly recommended most of the time, but sometimes you may need to authenticate to a server (especially an internal server) using a custom certificate bundle.
Expand All @@ -61,8 +63,6 @@ client = AuthenticatedClient(
)
```

There are more settings on the generated `Client` class which let you control more runtime behavior, check out the docstring on that class for more info.

Things to know:
1. Every path/method combo becomes a Python module with four functions:
1. `sync`: Blocking request that returns parsed data (if successful) or `None`
Expand All @@ -74,7 +74,42 @@ Things to know:
1. If your endpoint had any tags on it, the first tag will be used as a module name for the function (my_tag above)
1. Any endpoint which did not have a tag will be in `my_test_api_client.api.default`

## Building / publishing this Client
## Advanced customizations

There are more settings on the generated `Client` class which let you control more runtime behavior, check out the docstring on that class for more info. You can also customize the underlying `httpx.Client` or `httpx.AsyncClient` (depending on your use-case):

```python
from my_test_api_client import Client

def log_request(request):
print(f"Request event hook: {request.method} {request.url} - Waiting for response")

def log_response(response):
request = response.request
print(f"Response event hook: {request.method} {request.url} - Status {response.status_code}")

client = Client(
base_url="https://api.example.com",
httpx_args={"event_hooks": {"request": [log_request], "response": [log_response]}},
)

# Or get the underlying httpx client to modify directly with client.get_httpx_client() or client.get_async_httpx_client()
```

You can even set the httpx client directly, but beware that this will override any existing settings (e.g., base_url):

```python
import httpx
from my_test_api_client import Client

client = Client(
base_url="https://api.example.com",
)
# Note that base_url needs to be re-set, as would any shared cookies, headers, etc.
client.set_httpx_client(httpx.Client(base_url="https://api.example.com", proxies="http://localhost:8030"))
```

## Building / publishing this package
This project uses [Poetry](https://python-poetry.org/) to manage dependencies and packaging. Here are the basics:
1. Update the metadata in pyproject.toml (e.g. authors, version)
1. If you're using a private repository, configure it with Poetry
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -4,19 +4,15 @@
import httpx

from ... import errors
from ...client import Client
from ...client import AuthenticatedClient, Client
from ...types import UNSET, Response, Unset


def _get_kwargs(
*,
client: Client,
common: Union[Unset, None, str] = UNSET,
) -> Dict[str, Any]:
url = "{}/common_parameters".format(client.base_url)

headers: Dict[str, str] = client.get_headers()
cookies: Dict[str, Any] = client.get_cookies()
pass

params: Dict[str, Any] = {}
params["common"] = common
Expand All @@ -25,16 +21,12 @@ def _get_kwargs(

return {
"method": "get",
"url": url,
"headers": headers,
"cookies": cookies,
"timeout": client.get_timeout(),
"follow_redirects": client.follow_redirects,
"url": "/common_parameters",
"params": params,
}


def _parse_response(*, client: Client, response: httpx.Response) -> Optional[Any]:
def _parse_response(*, client: Union[AuthenticatedClient, Client], response: httpx.Response) -> Optional[Any]:
if response.status_code == HTTPStatus.OK:
return None
if client.raise_on_unexpected_status:
Expand All @@ -43,7 +35,7 @@ def _parse_response(*, client: Client, response: httpx.Response) -> Optional[Any
return None


def _build_response(*, client: Client, response: httpx.Response) -> Response[Any]:
def _build_response(*, client: Union[AuthenticatedClient, Client], response: httpx.Response) -> Response[Any]:
return Response(
status_code=HTTPStatus(response.status_code),
content=response.content,
Expand All @@ -54,7 +46,7 @@ def _build_response(*, client: Client, response: httpx.Response) -> Response[Any

def sync_detailed(
*,
client: Client,
client: Union[AuthenticatedClient, Client],
common: Union[Unset, None, str] = UNSET,
) -> Response[Any]:
"""
Expand All @@ -70,12 +62,10 @@ def sync_detailed(
"""

kwargs = _get_kwargs(
client=client,
common=common,
)

response = httpx.request(
verify=client.verify_ssl,
response = client.get_httpx_client().request(
**kwargs,
)

Expand All @@ -84,7 +74,7 @@ def sync_detailed(

async def asyncio_detailed(
*,
client: Client,
client: Union[AuthenticatedClient, Client],
common: Union[Unset, None, str] = UNSET,
) -> Response[Any]:
"""
Expand All @@ -100,11 +90,9 @@ async def asyncio_detailed(
"""

kwargs = _get_kwargs(
client=client,
common=common,
)

async with httpx.AsyncClient(verify=client.verify_ssl) as _client:
response = await _client.request(**kwargs)
response = await client.get_async_httpx_client().request(**kwargs)

return _build_response(client=client, response=response)
Loading