Convert existing models with cache duplication to symlinks

[quickgrid]

Is your feature request related to a problem? Please describe.
I have duplication of multiple models in windows with same large model files stored on both blobs and snapshots folder that was downloaded in past. This is taking lots of extra space which can be reduced by converting to symlink versions.

For new models I run python terminal as admin and it creates 0KB symlink in snapshots folder and I see actual files are stored on blobs folder. There is no duplicate extra space taken.

I have also seen some models store actual files on snapshots folder and the blobs folder is empty.

Describe the solution you'd like
Convert existing model folder that have duplications to symlink versions.

Describe alternatives you've considered
Delete cache for duplicated models and redownload with admin privilege.

Additional context

[Wauplin]

Hi @quickgrid 👋

Symlinks on windows have indeed be a topic so far. The behavior you mentioned (some duplicated files, some not, some in blobs, some in snapshots,...) comes from the fact that symlinks are not supported by default on Windows. What we really encourage is to set developer mode on your machine or always run as admin (less ideal). You should normally get a warning message about this.

For a more detailed explanation, please read #1062 (comment) (and the full issue). The behavior has been introduced in by PR #1067. It is not optimal but that's "the best we can do" to get huggingface_hub working on all setups.

Now, if you want to fix your situation the best solution is to enable developer mode and clean your existing cache. Hope this is fine with you :)