Skip to content

Commit 06c7001

Browse files
authored
[VSSL-7701] Demo LLM API with VESSL Run and vLLM (#29)
* Add vllm-online-serving * Add prom metrics * Update monitoring * remove logging * Add labels * Use vllm directly from upstream latest to pick up vllm-project/vllm#2316 * Roll back vllm to 0.3.0 * Get patch files for metrics in vllm-project/vllm#2316 * Update llm_engine.py * Write documents * Add vllm-online-serving/README-ko.md * write README.md
1 parent 969bc96 commit 06c7001

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

47 files changed

+3207
-1526
lines changed

mistral_7B/LICENSE

Lines changed: 0 additions & 201 deletions
This file was deleted.

mistral_7B/README.md

Lines changed: 0 additions & 147 deletions
This file was deleted.
-22.3 KB
Binary file not shown.

mistral_7B/assets/chunking.png

-27.3 KB
Binary file not shown.

mistral_7B/assets/full_attention.png

-17.7 KB
Binary file not shown.

mistral_7B/assets/kv_padding.png

-25.1 KB
Binary file not shown.

mistral_7B/assets/padding.png

-17.5 KB
Binary file not shown.

mistral_7B/assets/rolling_cache.png

-22.1 KB
Binary file not shown.
-18.6 KB
Binary file not shown.

mistral_7B/deploy/Dockerfile

Lines changed: 0 additions & 31 deletions
This file was deleted.

mistral_7B/deploy/entrypoint.sh

Lines changed: 0 additions & 11 deletions
This file was deleted.

0 commit comments

Comments
 (0)