Skip to content

feat: Cache model and engine downloads so users can restart download #777

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
Tracked by #1077
freelerobot opened this issue Jun 25, 2024 · 5 comments
Closed
Tracked by #1077
Assignees
Labels
category: model management Model pull, yaml, model state P1: important Important feature / fix type: feature request A new feature
Milestone

Comments

@freelerobot
Copy link
Contributor

freelerobot commented Jun 25, 2024

Problem:

  1. cortex run trt/onnx model
  2. wait for download
  3. 10 minutes in, something might happen, internet drops/windows goes to sleep, etc.
  4. Lose entire download progress.
  5. Sadness

Happened for me 2/3 times for larger model folders

Suggested solution

  • cache downloads, host a mirror, open to better suggestions
@freelerobot freelerobot added the type: feature request A new feature label Jun 25, 2024
@louis-menlo louis-menlo self-assigned this Jun 28, 2024
@imtuyethan imtuyethan moved this to Icebox in Menlo Sep 2, 2024
@freelerobot freelerobot changed the title feat: Model download caches would be nice (onnx/trt downloads are slow and often fail) feat: Cache model downloads so users can restart download Sep 6, 2024
@freelerobot freelerobot added the category: model management Model pull, yaml, model state label Sep 6, 2024
@freelerobot freelerobot added the P1: important Important feature / fix label Sep 6, 2024
@dan-menlo
Copy link
Contributor

Linking this to main issue #1077

@dan-menlo dan-menlo moved this from Icebox to Scheduled in Menlo Sep 8, 2024
@hiento09 hiento09 self-assigned this Sep 13, 2024
@hiento09
Copy link
Contributor

Hien will help to add checksum and version to cortex.tensorrtllm, cortex.llamacpp, cortex.onnx

@namchuai
Copy link
Contributor

proposed:
file name: version.txt
content: yaml compatible

name: full_name (vulkan/os/arch for us to troubleshoot issue in the future)
version: x.y.z
description: (optional)

@hiento09
Copy link
Contributor

hiento09 commented Sep 13, 2024

For checksum.yml file, use format similar to Jan app
Content: yaml compatible

version: 0.5.3
files:
  - url: jan-mac-arm64-0.5.3.zip
    sha512: >-
      GE3eaRoDi/fuqzdC0SQBvKvwjPGVPPBeulCo4JkjZuZ+0FLx0AiT/LgoUIMiOLh5LSR0Q73emkMsSBIcQco4bg==
  - url: jan-mac-arm64-0.5.3.dmg
    sha512: >-
      bpB8x0UajDWvshJ37NmuQVLl9MqUd3J/7yydKWtpuEJBsu2TxXB8qJSta1mvxXGBke7SS13V2y3syP/qerHvoA==
  - url: jan-mac-x64-0.5.3.zip
    sha512: >-
      oX0mTvuqshXoy3TEVmw+JQGJaeJK//42j+UL6VTJus/637LeLzf3m/8TtIZMe7Ac6k5zpTox8bWqjIz5mKp6Bg==
  - url: jan-mac-x64-0.5.3.dmg
    sha512: >-
      Hd9SPyJM9TL2CVG0Is4lQzCSAXcn4WhVmra6hSBx0PgVN9L6utdc+yiYnhtd1L6x9BR1mwlZ3UCSJ8Ffr1wwJA==

@dan-menlo dan-menlo changed the title feat: Cache model downloads so users can restart download feat: Cache model and engine downloads so users can restart download Sep 20, 2024
@dan-menlo dan-menlo moved this from In Review to Review + QA in Menlo Sep 29, 2024
@gabrielle-ong gabrielle-ong moved this from Review + QA to Completed in Menlo Oct 3, 2024
@gabrielle-ong gabrielle-ong added this to the v1.0.0 milestone Oct 3, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
category: model management Model pull, yaml, model state P1: important Important feature / fix type: feature request A new feature
Projects
Archived in project
Development

No branches or pull requests

7 participants