[Model] Support Skywork-R1V #15397

pengyuange · 2025-03-24T13:09:23Z

fix vllm's support for skyworkr1v SkyworkAI/Skywork-R1V#6
FIX #15186

github-actions · 2025-03-24T13:09:34Z

👋 Hi! Thank you for contributing to the vLLM project.

💬 Join our developer Slack at https://slack.vllm.ai to discuss your PR in #pr-reviews, coordinate on features in #feat- channels, or join special interest groups in #sig- channels.

Just a reminder: PRs would not trigger full CI run by default. Instead, it would only run fastcheck CI which starts running only a small and essential subset of CI tests to quickly catch errors. You can run other CI tests on top of those by going to your fastcheck build on Buildkite UI (linked in the PR checks section) and unblock them. If you do not have permission to unblock, ping simon-mo or khluu to add you in our Buildkite org.

Once the PR is approved and ready to go, your PR reviewer(s) can run CI to test the changes comprehensively before merging.

To run CI, PR reviewers can either: Add ready label to the PR or enable auto-merge.

🚀

DarkLight1337 · 2025-03-24T13:59:42Z

Thanks for implementing this! Can you apply the fix #15086 to this model so it can work correctly in V1?

DarkLight1337 · 2025-03-24T14:01:06Z

Also, please add this model to the following files so CI can pass:

Supported Models page
tests/models/registry.py
tests/models/multimodal/processing/test_common.py
tests/models/decoder_only/vision_language/test_models.py

vllm/model_executor/models/registry.py

vllm/model_executor/models/skywork_vit.py

vllm/model_executor/models/skyworkr1v.py

Isotr0py

Hmmm, with a quick glance, this model is quite similar to original InternVL2 models...

If the difference is minor, I prefer to inherit from existing InternVL2 implementation instead of making a highly similar code copy...

vllm/model_executor/models/skywork_vit.py

vllm/model_executor/models/skyworkr1v.py

DarkLight1337 · 2025-03-26T08:29:33Z

Can you merge in the changes from latest main and apply the changes from #15443?

vllm/transformers_utils/configs/__init__.py

tests/models/decoder_only/vision_language/test_models.py

mergify · 2025-03-27T03:38:05Z

This pull request has merge conflicts that must be resolved before it can be
merged. Please rebase the PR, @pengyuange.

https://docs.github.com/en/pull-requests/collaborating-with-pull-requests/working-with-forks/syncing-a-fork

tests/models/registry.py

DarkLight1337 · 2025-03-28T06:54:19Z

I think the code looks pretty good now. Finally, can you add this model to the List of Supported Models page in the docs as well?

DarkLight1337 · 2025-03-28T06:54:43Z

And also add it to examples/offline_inference/vision_language*.py

pengyuange · 2025-03-28T12:10:21Z

I think the code looks pretty good now. Finally, can you add this model to the List of Supported Models page in the docs as well?

And also add it to examples/offline_inference/vision_language*.py

I have added these information.

docs/source/models/supported_models.md

Signed-off-by: jiacai.liu <[email protected]>

DarkLight1337 · 2025-03-28T14:58:30Z

There is a trailing comma after chat template field in https://huggingface.co/Skywork/Skywork-R1V-38B/blob/main/tokenizer_config.json which prevents the JSON from being loaded

pengyuange · 2025-03-28T15:35:15Z

There is a trailing comma after chat template field in https://huggingface.co/Skywork/Skywork-R1V-38B/blob/main/tokenizer_config.json which prevents the JSON from being loaded

It should be fixed right now.

DarkLight1337 · 2025-03-28T16:01:46Z

I can successfully run the example script, thanks for your effort and let's get this merged!

pengyuange · 2025-03-29T00:10:46Z

Ci has failed. how should I fix this?

DarkLight1337 · 2025-03-29T03:38:56Z

I can force merge

Signed-off-by: jiacai.liu <[email protected]> Co-authored-by: jiacai.liu <[email protected]>

Signed-off-by: jiacai.liu <[email protected]> Co-authored-by: jiacai.liu <[email protected]> Signed-off-by: Kyle Sayers <[email protected]>

Signed-off-by: jiacai.liu <[email protected]> Co-authored-by: jiacai.liu <[email protected]> Signed-off-by: xinyuxiao <[email protected]>

Signed-off-by: jiacai.liu <[email protected]> Co-authored-by: jiacai.liu <[email protected]> Signed-off-by: Louis Ulmer <[email protected]>

Signed-off-by: jiacai.liu <[email protected]> Co-authored-by: jiacai.liu <[email protected]>

Signed-off-by: jiacai.liu <[email protected]> Co-authored-by: jiacai.liu <[email protected]> Signed-off-by: Mu Huai <[email protected]>

mergify bot added the frontend label Mar 24, 2025

pengyuange force-pushed the skywork81 branch 2 times, most recently from fd29e4a to 6a8f898 Compare March 24, 2025 13:25

jeejeelee changed the title ~~skyworkr1v update~~ [Model] Support Skywork-R1V Mar 24, 2025

jeejeelee added the new-model Requests to new models label Mar 24, 2025

jeejeelee reviewed Mar 24, 2025

View reviewed changes

vllm/model_executor/models/registry.py Outdated Show resolved Hide resolved

jeejeelee reviewed Mar 24, 2025

View reviewed changes

vllm/model_executor/models/skywork_vit.py Outdated Show resolved Hide resolved

jeejeelee reviewed Mar 24, 2025

View reviewed changes

vllm/model_executor/models/skyworkr1v.py Outdated Show resolved Hide resolved

Isotr0py reviewed Mar 24, 2025

View reviewed changes

vllm/model_executor/models/skywork_vit.py Outdated Show resolved Hide resolved

vllm/model_executor/models/skyworkr1v.py Show resolved Hide resolved

Isotr0py added this to Multi-modal Model Requests Mar 25, 2025

Isotr0py moved this to In Progress in Multi-modal Model Requests Mar 25, 2025

pengyuange force-pushed the skywork81 branch from 6a8f898 to 2360898 Compare March 26, 2025 06:51

pengyuange requested review from DarkLight1337 and ywang96 as code owners March 26, 2025 06:51

mergify bot added the multi-modality Related to multi-modality (#4194) label Mar 26, 2025

pengyuange force-pushed the skywork81 branch 3 times, most recently from ff0d3b2 to e84cb1e Compare March 26, 2025 08:24

pengyuange force-pushed the skywork81 branch 3 times, most recently from 30e87c9 to 19a139e Compare March 26, 2025 13:00

pengyuange requested review from jeejeelee and Isotr0py March 27, 2025 02:01

Isotr0py reviewed Mar 27, 2025

View reviewed changes

vllm/transformers_utils/configs/__init__.py Outdated Show resolved Hide resolved

tests/models/decoder_only/vision_language/test_models.py Outdated Show resolved Hide resolved

tests/models/decoder_only/vision_language/test_models.py Outdated Show resolved Hide resolved

Isotr0py changed the base branch from v0.8.1 to main March 27, 2025 03:37

DarkLight1337 reviewed Mar 28, 2025

View reviewed changes

tests/models/registry.py Outdated Show resolved Hide resolved

pengyuange force-pushed the skywork81 branch from d70cbd3 to 15e8838 Compare March 28, 2025 06:47

pengyuange force-pushed the skywork81 branch from 15e8838 to 1a65457 Compare March 28, 2025 12:08

mergify bot added the documentation Improvements or additions to documentation label Mar 28, 2025

DarkLight1337 reviewed Mar 28, 2025

View reviewed changes

docs/source/models/supported_models.md Show resolved Hide resolved

pengyuange force-pushed the skywork81 branch from 1a65457 to d3cf2f7 Compare March 28, 2025 12:45

[Model] Support Skywork-R1V

fcb3f4b

Signed-off-by: jiacai.liu <[email protected]>

pengyuange force-pushed the skywork81 branch from d3cf2f7 to fcb3f4b Compare March 28, 2025 14:43

DarkLight1337 approved these changes Mar 28, 2025

View reviewed changes

DarkLight1337 enabled auto-merge (squash) March 28, 2025 16:02

github-actions bot added the ready ONLY add when PR is ready to merge/full CI is needed label Mar 28, 2025

vllm-bot merged commit de1cb38 into vllm-project:main Mar 29, 2025
42 of 44 checks passed

github-project-automation bot moved this from In Progress to Done in Multi-modal Model Requests Mar 29, 2025

lengrongfu pushed a commit to lengrongfu/vllm that referenced this pull request Apr 2, 2025

[Model] Support Skywork-R1V (vllm-project#15397)

3c4fd34

Signed-off-by: jiacai.liu <[email protected]> Co-authored-by: jiacai.liu <[email protected]>

kylesayrs pushed a commit to neuralmagic/vllm that referenced this pull request Apr 2, 2025

[Model] Support Skywork-R1V (vllm-project#15397)

008a60b

Signed-off-by: jiacai.liu <[email protected]> Co-authored-by: jiacai.liu <[email protected]> Signed-off-by: Kyle Sayers <[email protected]>

lulmer pushed a commit to lulmer/vllm that referenced this pull request Apr 7, 2025

[Model] Support Skywork-R1V (vllm-project#15397)

072be34

Signed-off-by: jiacai.liu <[email protected]> Co-authored-by: jiacai.liu <[email protected]> Signed-off-by: Louis Ulmer <[email protected]>

nishith-fujitsu pushed a commit to nishith-fujitsu/vllm that referenced this pull request Apr 9, 2025

[Model] Support Skywork-R1V (vllm-project#15397)

9ace520

Signed-off-by: jiacai.liu <[email protected]> Co-authored-by: jiacai.liu <[email protected]>

ckhordiasma mentioned this pull request Apr 17, 2025

[do not merge] pr test for nm changes into 2.20 red-hat-data-services/vllm#107

Closed

lk-chen pushed a commit to lk-chen/vllm that referenced this pull request Apr 29, 2025

[Model] Support Skywork-R1V (vllm-project#15397)

07db3e9

Signed-off-by: jiacai.liu <[email protected]> Co-authored-by: jiacai.liu <[email protected]>

shreyankg pushed a commit to shreyankg/vllm that referenced this pull request May 3, 2025

[Model] Support Skywork-R1V (vllm-project#15397)

f3af86d

Signed-off-by: jiacai.liu <[email protected]> Co-authored-by: jiacai.liu <[email protected]>

RichardoMrMu pushed a commit to RichardoMrMu/vllm that referenced this pull request May 12, 2025

[Model] Support Skywork-R1V (vllm-project#15397)

963702a

Signed-off-by: jiacai.liu <[email protected]> Co-authored-by: jiacai.liu <[email protected]> Signed-off-by: Mu Huai <[email protected]>

Uh oh!

[Model] Support Skywork-R1V #15397

[Model] Support Skywork-R1V #15397

Uh oh!

Conversation

pengyuange commented Mar 24, 2025 • edited by github-actions bot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

github-actions bot commented Mar 24, 2025

Uh oh!

DarkLight1337 commented Mar 24, 2025

Uh oh!

DarkLight1337 commented Mar 24, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Isotr0py left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

DarkLight1337 commented Mar 26, 2025

Uh oh!

Uh oh!

Uh oh!

Uh oh!

mergify bot commented Mar 27, 2025

Uh oh!

Uh oh!

DarkLight1337 commented Mar 28, 2025

Uh oh!

DarkLight1337 commented Mar 28, 2025

Uh oh!

pengyuange commented Mar 28, 2025

Uh oh!

Uh oh!

DarkLight1337 commented Mar 28, 2025

Uh oh!

pengyuange commented Mar 28, 2025

Uh oh!

DarkLight1337 commented Mar 28, 2025

Uh oh!

pengyuange commented Mar 29, 2025

Uh oh!

DarkLight1337 commented Mar 29, 2025

Uh oh!

Uh oh!

Uh oh!

pengyuange commented Mar 24, 2025 •

edited by github-actions bot

Loading

DarkLight1337 commented Mar 24, 2025 •

edited

Loading