ConnectionField loads the entire query set when paginating

[Dandi91]

I've noticed that when I query Relay-based pagination it doesn't limit the database query, but loads everything into memory and then does slicing. I understand that for a database table of moderate size it might not be a problem, but it becomes a concern when you've got a few hundred thousands records and want to display only a hundred of them at a time.
Is it something that was missed or this behavior was deliberately omitted for some reason?

Also, so far I got this workaround:

class Query(ObjectType):
	all_users = relay.ConnectionField(UserConnection)

	def resolve_all_users(self, info, **args):
		_len = User.objects.count()
		return connection_from_list_slice(User.objects.all(), args, 
			connection_type=UserConnection,
			edge_type=UserConnection.Edge, pageinfo_type=relay.PageInfo,
			list_length=_len, list_slice_length=_len)

It is possible to hide this inside a class derived from ConnectionField and override resolve_connection() which would call connection_from_list_slice() instead of connection_from_list(). However, this workaround still relies on the current implementation of connection_from_list_slice(), which fortunately doesn't do anything that evaluates the passed QuerySet until it actually slices it (and only if you provide list_slice_length argument to it). From the description of the function it's unsure whether it was specifically designed not to evaluate QuerySet or it's just a coincidence. Another downside to it is a separate query that counts rows, but I guess it is a trade off for not fetching everything at once.

So, my question really is: is it a useful feature and if it is, could it be done cleaner (in particular, not relying on connection_from_list_slice() implementation)?

[jkimbo]

Hi @Dandi91 . We're currently going through old issues that appear to have gone stale (ie. not updated in about the last 3 months) to try and clean up the issue tracker. If this is still important to you please comment and we'll re-open this.

Thanks!