Skip to content

Commit c51af4b

Browse files
tomchristiekarpetrosyan
tomchristie
and
karpetrosyan
authored
Extensions docs (#3080)
* Deprecate app=... in favour of explicit WSGITransport/ASGITransport * Linting * Linting * Update WSGITransport and ASGITransport docs * Deprecate app * Drop deprecation tests * Add CHANGELOG * Deprecate 'app=...' shortcut, rather than removing it. * Update CHANGELOG * Fix test_asgi.test_deprecated_shortcut * Extensions docs * Include 'extensions' in docs index * Update docs/advanced/extensions.md Co-authored-by: Kar Petrosyan <[email protected]> --------- Co-authored-by: Kar Petrosyan <[email protected]>
1 parent cabd1c0 commit c51af4b

File tree

2 files changed

+202
-0
lines changed

2 files changed

+202
-0
lines changed

‎docs/advanced/extensions.md

+201
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,201 @@
1+
# Extensions
2+
3+
Request and response extensions provide a untyped space where additional information may be added.
4+
5+
Extensions should be used for features that may not be available on all transports, and that do not fit neatly into [the simplified request/response model](https://www.encode.io/httpcore/extensions/) that the underlying `httpcore` pacakge uses as it's API.
6+
7+
Several extensions are supported on the request:
8+
9+
```python
10+
# Request timeouts actually implemented as an extension on
11+
# the request, ensuring that they are passed throughout the
12+
# entire call stack.
13+
client = httpx.Client()
14+
response = client.get(
15+
"https://www.example.com",
16+
extensions={"timeout": {"connect": 5.0}}
17+
)
18+
response.request.extensions["timeout"]
19+
{"connect": 5.0}
20+
```
21+
22+
And on the response:
23+
24+
```python
25+
client = httpx.Client()
26+
response = client.get("https://www.example.com")
27+
print(response.extensions["http_version"]) # b"HTTP/1.1"
28+
# Other server responses could have been
29+
# b"HTTP/0.9", b"HTTP/1.0", or b"HTTP/1.1"
30+
```
31+
32+
## Request Extensions
33+
34+
### `"trace"`
35+
36+
The trace extension allows a callback handler to be installed to monitor the internal
37+
flow of events within the underlying `httpcore` transport.
38+
39+
The simplest way to explain this is with an example:
40+
41+
```python
42+
import httpx
43+
44+
def log(event_name, info):
45+
print(event_name, info)
46+
47+
client = httpx.Client()
48+
response = client.get("https://www.example.com/", extensions={"trace": log})
49+
# connection.connect_tcp.started {'host': 'www.example.com', 'port': 443, 'local_address': None, 'timeout': None}
50+
# connection.connect_tcp.complete {'return_value': <httpcore.backends.sync.SyncStream object at 0x1093f94d0>}
51+
# connection.start_tls.started {'ssl_context': <ssl.SSLContext object at 0x1093ee750>, 'server_hostname': b'www.example.com', 'timeout': None}
52+
# connection.start_tls.complete {'return_value': <httpcore.backends.sync.SyncStream object at 0x1093f9450>}
53+
# http11.send_request_headers.started {'request': <Request [b'GET']>}
54+
# http11.send_request_headers.complete {'return_value': None}
55+
# http11.send_request_body.started {'request': <Request [b'GET']>}
56+
# http11.send_request_body.complete {'return_value': None}
57+
# http11.receive_response_headers.started {'request': <Request [b'GET']>}
58+
# http11.receive_response_headers.complete {'return_value': (b'HTTP/1.1', 200, b'OK', [(b'Age', b'553715'), (b'Cache-Control', b'max-age=604800'), (b'Content-Type', b'text/html; charset=UTF-8'), (b'Date', b'Thu, 21 Oct 2021 17:08:42 GMT'), (b'Etag', b'"3147526947+ident"'), (b'Expires', b'Thu, 28 Oct 2021 17:08:42 GMT'), (b'Last-Modified', b'Thu, 17 Oct 2019 07:18:26 GMT'), (b'Server', b'ECS (nyb/1DCD)'), (b'Vary', b'Accept-Encoding'), (b'X-Cache', b'HIT'), (b'Content-Length', b'1256')])}
59+
# http11.receive_response_body.started {'request': <Request [b'GET']>}
60+
# http11.receive_response_body.complete {'return_value': None}
61+
# http11.response_closed.started {}
62+
# http11.response_closed.complete {'return_value': None}
63+
```
64+
65+
The `event_name` and `info` arguments here will be one of the following:
66+
67+
* `{event_type}.{event_name}.started`, `<dictionary of keyword arguments>`
68+
* `{event_type}.{event_name}.complete`, `{"return_value": <...>}`
69+
* `{event_type}.{event_name}.failed`, `{"exception": <...>}`
70+
71+
Note that when using async code the handler function passed to `"trace"` must be an `async def ...` function.
72+
73+
The following event types are currently exposed...
74+
75+
**Establishing the connection**
76+
77+
* `"connection.connect_tcp"`
78+
* `"connection.connect_unix_socket"`
79+
* `"connection.start_tls"`
80+
81+
**HTTP/1.1 events**
82+
83+
* `"http11.send_request_headers"`
84+
* `"http11.send_request_body"`
85+
* `"http11.receive_response"`
86+
* `"http11.receive_response_body"`
87+
* `"http11.response_closed"`
88+
89+
**HTTP/2 events**
90+
91+
* `"http2.send_connection_init"`
92+
* `"http2.send_request_headers"`
93+
* `"http2.send_request_body"`
94+
* `"http2.receive_response_headers"`
95+
* `"http2.receive_response_body"`
96+
* `"http2.response_closed"`
97+
98+
The exact set of trace events may be subject to change across different versions of `httpcore`. If you need to rely on a particular set of events it is recommended that you pin installation of the package to a fixed version.
99+
100+
### `"sni_hostname"`
101+
102+
The server's hostname, which is used to confirm the hostname supplied by the SSL certificate.
103+
104+
If you want to connect to an explicit IP address rather than using the standard DNS hostname lookup, then you'll need to use this request extension.
105+
106+
For example:
107+
108+
``` python
109+
# Connect to '185.199.108.153' but use 'www.encode.io' in the Host header,
110+
# and use 'www.encode.io' when SSL verifying the server hostname.
111+
client = httpx.Client()
112+
headers = {"Host": "www.encode.io"}
113+
extensions = {"sni_hostname": "www.encode.io"}
114+
response = client.get(
115+
"https://185.199.108.153/path",
116+
headers=headers,
117+
extensions=extensions
118+
)
119+
```
120+
121+
### `"timeout"`
122+
123+
A dictionary of `str: Optional[float]` timeout values.
124+
125+
May include values for `'connect'`, `'read'`, `'write'`, or `'pool'`.
126+
127+
For example:
128+
129+
```python
130+
# Timeout if a connection takes more than 5 seconds to established, or if
131+
# we are blocked waiting on the connection pool for more than 10 seconds.
132+
client = httpx.Client()
133+
response = client.get(
134+
"https://www.example.com",
135+
extensions={"timeout": {"connect": 5.0, "pool": 10.0}}
136+
)
137+
```
138+
139+
This extension is how the `httpx` timeouts are implemented, ensuring that the timeout values are associated with the request instance and passed throughout the stack. You shouldn't typically be working with this extension directly, but use the higher level `timeout` API instead.
140+
141+
## Response Extensions
142+
143+
### `"http_version"`
144+
145+
The HTTP version, as bytes. Eg. `b"HTTP/1.1"`.
146+
147+
When using HTTP/1.1 the response line includes an explicit version, and the value of this key could feasibly be one of `b"HTTP/0.9"`, `b"HTTP/1.0"`, or `b"HTTP/1.1"`.
148+
149+
When using HTTP/2 there is no further response versioning included in the protocol, and the value of this key will always be `b"HTTP/2"`.
150+
151+
### `"reason_phrase"`
152+
153+
The reason-phrase of the HTTP response, as bytes. For example `b"OK"`. Some servers may include a custom reason phrase, although this is not recommended.
154+
155+
HTTP/2 onwards does not include a reason phrase on the wire.
156+
157+
When no key is included, a default based on the status code may be used.
158+
159+
### `"stream_id"`
160+
161+
When HTTP/2 is being used the `"stream_id"` response extension can be accessed to determine the ID of the data stream that the response was sent on.
162+
163+
### `"network_stream"`
164+
165+
The `"network_stream"` extension allows developers to handle HTTP `CONNECT` and `Upgrade` requests, by providing an API that steps outside the standard request/response model, and can directly read or write to the network.
166+
167+
The interface provided by the network stream:
168+
169+
* `read(max_bytes, timeout = None) -> bytes`
170+
* `write(buffer, timeout = None)`
171+
* `close()`
172+
* `start_tls(ssl_context, server_hostname = None, timeout = None) -> NetworkStream`
173+
* `get_extra_info(info) -> Any`
174+
175+
This API can be used as the foundation for working with HTTP proxies, WebSocket upgrades, and other advanced use-cases.
176+
177+
See the [network backends documentation](https://www.encode.io/httpcore/network-backends/) for more information on working directly with network streams.
178+
179+
**Extra network information**
180+
181+
The network stream abstraction also allows access to various low-level information that may be exposed by the underlying socket:
182+
183+
```python
184+
response = httpx.get("https://www.example.com")
185+
network_stream = response.extensions["network_stream"]
186+
187+
client_addr = network_stream.get_extra_info("client_addr")
188+
server_addr = network_stream.get_extra_info("server_addr")
189+
print("Client address", client_addr)
190+
print("Server address", server_addr)
191+
```
192+
193+
The socket SSL information is also available through this interface, although you need to ensure that the underlying connection is still open, in order to access it...
194+
195+
```python
196+
with httpx.stream("GET", "https://www.example.com") as response:
197+
network_stream = response.extensions["network_stream"]
198+
199+
ssl_object = network_stream.get_extra_info("ssl_object")
200+
print("TLS version", ssl_object.version())
201+
```

‎mkdocs.yml

+1
Original file line numberDiff line numberDiff line change
@@ -34,6 +34,7 @@ nav:
3434
- Event Hooks: 'advanced/event-hooks.md'
3535
- Transports: 'advanced/transports.md'
3636
- Text Encodings: 'advanced/text-encodings.md'
37+
- Extensions: 'advanced/extensions.md'
3738
- Guides:
3839
- Async Support: 'async.md'
3940
- HTTP/2 Support: 'http2.md'

0 commit comments

Comments
 (0)