Skip to content

Commit 5bbdf65

Browse files
authored
Rollup merge of #99703 - dtolnay:tokenstreamsizehint, r=petrochenkov
Expose size_hint() for TokenStream's iterator The iterator for `proc_macro::TokenStream` is a wrapper around a `Vec` iterator: https://github.com/rust-lang/rust/blob/babff2211e3ae9ef52852dc1b01f3eacdd94c12e/library/proc_macro/src/lib.rs#L363-L371 so it can cheaply provide a perfectly precise size hint, with just a pointer subtraction: https://github.com/rust-lang/rust/blob/babff2211e3ae9ef52852dc1b01f3eacdd94c12e/library/alloc/src/vec/into_iter.rs#L170-L177 I need the size hint in syn (https://github.com/dtolnay/syn/blob/1.0.98/src/buffer.rs) to reduce allocations when converting TokenStream into syn's internal TokenBuffer representation. Aside from `size_hint`, the other non-default methods in `std::vec::IntoIter`'s `Iterator` impl are `advance_by`, `count`, and `__iterator_get_unchecked`. I've included `count` in this PR since it is trivial. I did not include `__iterator_get_unchecked` because it is spoopy and I did not feel like dealing with that. Lastly, I did not include `advance_by` because that requires `feature(iter_advance_by)` (#77404) and I noticed this comment at the top of libproc_macro: https://github.com/rust-lang/rust/blob/babff2211e3ae9ef52852dc1b01f3eacdd94c12e/library/proc_macro/src/lib.rs#L20-L22
2 parents 2973b00 + 63e74ab commit 5bbdf65

File tree

1 file changed

+8
-0
lines changed

1 file changed

+8
-0
lines changed

library/proc_macro/src/lib.rs

+8
Original file line numberDiff line numberDiff line change
@@ -382,6 +382,14 @@ pub mod token_stream {
382382
bridge::TokenTree::Literal(tt) => TokenTree::Literal(Literal(tt)),
383383
})
384384
}
385+
386+
fn size_hint(&self) -> (usize, Option<usize>) {
387+
self.0.size_hint()
388+
}
389+
390+
fn count(self) -> usize {
391+
self.0.count()
392+
}
385393
}
386394

387395
#[stable(feature = "proc_macro_lib2", since = "1.29.0")]

0 commit comments

Comments
 (0)