Description
Surf Architecture
This is a (quick) design document for Surf, an HTTP client with middleware
support in Rust.
Goals
- concise API for most common cases
- builder API for more complex cases
- backend agnostic (e.g. swap Hyper for libcurl or browser fetch)
- extendable using middleware
- enable some middleware by default (compression, redirects, etc)
- provide a way to opt-out of default middleware
API Example
// text
let res = xhr::get("/endpoint").send().await?;
dbg!(res);
// json
#[derive(Deserialize)]
struct User { name: String }
let user: User = xhr::get("/endpoint").send_json().await?;
dbg!(user);Rust types are nice in that send_json can take a type parameter (User in
this case) which allows it to try and deserialize the type into that struct, and
if it can't it'll return an error. This allows for super concise JSON APIs!
Another implication of the way this is done is that the struct definitions can
be shared between servers & clients. This means that we can guarantee a server
and corresponding client can work with each other, even if the wire protocol is
something like JSON. And this can further be enhance using something like
session-types.
Architecture
surfas the top-level framework crate with sane defaults & middlewarehttp-clientto provide the swappable backend layer- the middleware layer probably should live here too
- contains types also
- a collection of useful middleware in the same repo as
surf- we can probably move some core bits out as we progress to make it more
widely useful. See
rustasync/async-compression.
- we can probably move some core bits out as we progress to make it more
-----------------
| Surf middleware | = `cargo add surf`
| Surf core |
-----------------
^
|-----------------
|
------------------- |
| http-client-curl | |
| http-client-fetch | |
------------------- |
| http-client-hyper | --|
| http-client | = `cargo add http-client`
-------------------Middleware
let user: User = xhr::get("/endpoint")
.middleware(some_middleware::new("value"))
.send_json().await?;We should take a look at make-fetch-happen and start by building out the features it implements. We should probably define some sort of prioritization here (:
Connection Pooling
A core feature of surf should be the ability to perform connection pooling.
I'm not sure yet how that works, as I've never used & researched it. But it
seems very important, and we should come up with a design here. I'm thinking
in terms of API it may be something like this:
let pool = surf::Pool::new()
.max_sockets(512)
.max_free_sockets(256)
.build();
let res = pool.get("some_url").send().await?;I somehow suspect that a similar API may be relevant for performing H2 requests
down the line too, as it's yet another form of multiplexing. It's a bit early
right now, but if we can get the API right we can add this functionality at a
later stage without too many surface changes.
Conclusion
This is about it for now. I hope I covered the rough outlines here!