Improve Workers Sites asset sync reliability (#3098)

* Improve Workers Sites asset sync reliability

- When splitting upload into buckets, just store the file path, not
  the full content. This means we have to read the contents twice,
  but avoids buffering all content before uploading, causing OOMs.
- Limit in-flight bulk upload requests to 5, avoiding
  `Too many bulk operations already in progress.` error.
- Fix logging of upload progress. Previous, progress was logged per
  upload bucket, which doesn't really make sense to end users. Now,
  upload progress is across all files in all buckets.
- Only log first 100 changed assets by default. The rest can be shown
  by setting `WRANGLER_LOG=debug`. This avoids console spam when
  uploading sites with 1000s of files. A little bit of colour has
  also been added to the diff. :)

Closes #2223
Closes #2245

* fixup! Improve Workers Sites asset sync reliability

Move fetching list log

* fixup! Improve Workers Sites asset sync reliability

Read files for upload in serial

* fixup! Improve Workers Sites asset sync reliability

Add test for upload failing

* fixup! Improve Workers Sites asset sync reliability

Ensure publish tests not dependent on bucket upload order

Author: mrbbot

.changeset/clever-radios-cover.md

@@ -0,0 +1,10 @@
+---
+"wrangler": minor
+---
+
+fix: improve Workers Sites asset upload reliability
+
+- Wrangler no longer buffers all assets into memory before uploading. This should prevent out-of-memory errors when publishing sites with many large files.
+- Wrangler now limits the number of in-flight asset upload requests to 5, fixing the `Too many bulk operations already in progress` error.
+- Wrangler now correctly logs upload progress. Previously, the reported percentage was per upload request group, not across all assets.
+- Wrangler no longer logs all assets to the console by default. Instead, it will just log the first 100. The rest can be shown by setting the `WRANGLER_LOG=debug` environment variable. A splash of colour has also been added.