Add an index setting to limit the maximum number of slices allowed in a scroll request.

[jimczi]

This is a follow up for #18237 (comment)

[s1monw]

I wonder if we should default it to the number of shards :)

[s1monw]

I think we should be very conservative here by default

[s1monw]

left some comments! thanks for doing this!

[jimczi]

@s1monw I pushed another commit to address your comments. The default maximum number of slices is now equal to the number of shards in the requested index.

[s1monw]

@s1monw I pushed another commit to address your comments. The default maximum number of slices is now equal to the number of shards in the requested index.

this might actually not work 😞 if we go through an alias we may have way more slices than shards in a single index. and then we fail half way. So I think we need something different (as you had)? Maybe 1024 ? @jpountz WDTY - sorry for not thinking about it earlier

[jimczi]

Don't _shrink my scroll ! ;)
Yep aliases are dangerous. The simple fact that each slice is an independent request is dangerous.
You may also argue that the index could be deleted and re-created between two sliced scroll.
+1 for 1024 and I'll add a note in the docs about aliases and index creation

[jpountz]

+1 to a fixed limit, I think it is easier to reason about

[jimczi]

@jpountz @s1monw I changed the default to 1024.

[s1monw]

LGTM