Do not fetch /preupload if already done in upload-large-folder

[Wauplin]

Originally from @Kakulukian on slack (private):

From the end of last weekend until yesterday, we experienced increased latency on the hub. This appears to be connected to how the upload_large_folder method works. I've identified that one user uploaded approximately ~200,000 files using this method.
According to my understanding of the code, uploading via this method requires a call to the preupload endpoint for each file (right?). The issue is that as a repository grows more significant (commits or files), this method becomes progressively slower, particularly when the calls aren't batched
Would it be possible to also implement file batching for this endpoint with a minimum threshold?

[HuggingFaceDocBuilderDev]

The docs for this PR live here. All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

[hanouticelina]

exactly the changes I was going to commit 😄
thank you!

[hanouticelina]

cood catch! I completely missed that we weren’t storing the upload modes too. This is a much more complete fix, thank you!
I tested it locally as well, it works as expected ✅

[hanouticelina]

We don't need target_chunk() function anymore, right? I'm a bit confused, I don't recall exactly why we had it in the first place in #3016, the dynamic sizing was only meant for the commit job and it was not strictly necessary for the get upload mode job

[Wauplin]

well, we reviewed #3016 a bit too quickly 🙈 target_chunk must be used for commit chunks (but not done at the moment). Let's fix this in this PR

[Wauplin]

done in ce1e35d

[hanouticelina]

I'm merging this, thanks again @Wauplin for the fix!