Skip to content

It is recommended to optimize the encode_document and encode_array functions. #146

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
danclive opened this issue Feb 25, 2020 · 2 comments
Closed

Comments

@danclive
Copy link

In our implementation of the function that encodes the Document and Array as bytes, we need to allocate a Vec and encode the elements in it to calculate the length. This causes excess memory allocation.

pub fn encode_document<
'a,
S: AsRef<str> + 'a,
W: Write + ?Sized,
D: IntoIterator<Item = (&'a S, &'a Bson)>,
>(
writer: &mut W,
doc: D,
) -> EncoderResult<()> {
let mut buf = Vec::new();
for (key, val) in doc.into_iter() {
encode_bson(&mut buf, key.as_ref(), val)?;
}
write_i32(
writer,
(buf.len() + mem::size_of::<i32>() + mem::size_of::<u8>()) as i32,
)?;
writer.write_all(&buf)?;
writer.write_u8(0)?;
Ok(())
}

I looked at RawBson idea, but RawBson is immutable, and we often need to change documents, so it's not particularly useful in practice.

That's why I recommend whether you can maintain a length message when you're building a Document and an Array, just like that:

pub struct Document {
    map: IndexMap<String, Bson>,
    len: usize
}

pub struct Array {
    vec: Vec<Bson>,
    len: usize
}

This allows you to use this length directly when encode, without having to encode to vec first.

@koshkin-kna
Copy link

Encoding and decoding of embedded objects and arrays, in principle, work very slowly.
A flat document is processed on the order of ~ 0.002 seconds, while its counterpart with nesting is ~ 0.030 seconds

@jcdyer
Copy link
Contributor

jcdyer commented Mar 12, 2020

Length of a Bson object can be calculated without encoding it. See the doc_size_bytes and size_bytes functions in https://github.com/mongodb/mongo-rust-driver/blob/master/src/bson_util.rs

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

3 participants