Compress-ZipArchive still hits 2GB limit when using -Update

[ssilvidi]

Running into the same issue as referenced here: PowerShell/Microsoft.PowerShell.Archive#19 (comment)

I've just tried this on a 3.7GB file and it cuts the original file short as the CopyTo has the same Stream was too long issue. The archive completes but only with 2GB of the original file.

I found this module in hopes of solving a strange case where I've only got remote PowerShell access to a server and I need to compress some big log files. I don't have enough free space to compress the entire folder, so my plan was to add a few files at a time to the archive.

Unfortunately, I'm getting the usual "Stream was too long" error when using -Update. Note that I don't have a problem when creating a new archive - it only seems to happen with -Update.

PS E:\Services\logs\AppService> ls *2023-03-0*.svclog | select -First 3


    Directory: E:\Services\logs\AppService


Mode                LastWriteTime         Length Name
----                -------------         ------ ----
-a----         3/4/2023  11:59 PM      899010559 AppServiceLog-2023-03-04-00-00-01.svclog
-a----         3/5/2023  11:59 PM     5206177620 AppServiceLog-2023-03-05-00-00-00.svclog
-a----         3/6/2023   5:47 PM     3460504997 AppServiceLog-2023-03-06-00-00-01.svclog

...

PS E:\Services\logs\AppService> ls *2023-03-0*.svclog | select -First 3 | Compress-ZipArchive -DestinationPath $dest -Update
Compress-ZipArchive : Exception calling "CopyTo" with "1" argument(s): "Stream was too long."
At line:1 char:43
+ ...  select -First 3 | Compress-ZipArchive -DestinationPath $dest -Update
+                        ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
    + CategoryInfo          : NotSpecified: (:) [Compress-ZipArchive], MethodInvocationException
    + FullyQualifiedErrorId : IOException,Compress-ZipArchive

Compress-ZipArchive : Exception calling "CopyTo" with "1" argument(s): "Stream was too long."
At line:1 char:43
+ ...  select -First 3 | Compress-ZipArchive -DestinationPath $dest -Update
+                        ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
    + CategoryInfo          : NotSpecified: (:) [Compress-ZipArchive], MethodInvocationException
    + FullyQualifiedErrorId : IOException,Compress-ZipArchive

...

PS E:\Services\logs\AppService> $archive = [System.IO.Compression.ZipFile]::OpenRead("$pwd\AppServiceLog_backup_2023-03-21.zip")
PS E:\Services\logs\AppService> $archive.Entries | ft fullname, compressedlength, length

FullName                                 CompressedLength     Length
--------                                 ----------------     ------
AppServiceLog-2023-02-18-00-00-00.svclog         11079552  862175799
AppServiceLog-2023-02-25-00-00-00.svclog         12047838  970221159
AppServiceLog-2023-03-04-00-00-01.svclog         11284310  899010559
AppServiceLog-2023-03-05-00-00-00.svclog         28428997 2147450880
AppServiceLog-2023-03-06-00-00-01.svclog         29660913 2147450880

This was with PSCompression 1.0.4 from PSGallery and PowerShell 5.1.17763.2931.
Hopefully the output above tells the story, but let me know what I can do to help.

Thanks!

[santisq]

Hi @ssilvidi will test this later but looking at the .NET Docs the exception you might be seeing is this one:

In which case I'm afraid I don't have a solution for this.

[santisq]

Seems like this is simply not possible using ZipArchive class unfortunately, at least for me if the Zip is a new one I can compress files no matter their size @ssilvidi. Might need to find other way to do this, either use an external program (7-zip or other) or instead of appending to an existing Zip File create new ones.

[ssilvidi]

Thanks for the quick response!

I was able to make it work in my scenario by creating separate archives in a few steps. Like you said, creating new files worked for me regardless of the size.

[santisq]

@ssilvidi good to know and happy you could sort it out. I'll close this issue for now, feel free to re-open if you have any related question