Skip to content

test: add kv cache event tests for disagg workers #3602

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 3 commits into from
Apr 18, 2025

Conversation

zhengd-nv
Copy link
Collaborator

This PR introduces new tests for some features required by kv cache aware routing on disaggregate serving.

  • Getting KV cache events from workers
  • KV cache blocks matching based on hash

This PR also fixed the issue that LLM.get_kv_cache_events returns List[List[dict]] instead of type annotated List[dict] and updated tests accordingly.

@zhengd-nv
Copy link
Collaborator Author

/bot run

@tensorrt-cicd
Copy link
Collaborator

PR_Github #2429 [ run ] triggered by Bot

@tensorrt-cicd
Copy link
Collaborator

PR_Github #2429 [ run ] completed with state SUCCESS
/LLM/main/L0_MergeRequest_PR pipeline #1745 completed with status: 'FAILURE'

@zhengd-nv zhengd-nv force-pushed the disagg-cache-event branch from 7502595 to 1c81fab Compare April 16, 2025 09:02
@zhengd-nv
Copy link
Collaborator Author

/bot run

@tensorrt-cicd
Copy link
Collaborator

PR_Github #2452 [ run ] triggered by Bot

@tensorrt-cicd
Copy link
Collaborator

PR_Github #5 [ run ] triggered by Bot

@tensorrt-cicd
Copy link
Collaborator

PR_Github #5 [ run ] completed with state ABORTED

@tensorrt-cicd
Copy link
Collaborator

PR_Github #2452 [ run ] completed with state SUCCESS
/LLM/main/L0_MergeRequest_PR pipeline #1758 completed with status: 'FAILURE'

@zhengd-nv
Copy link
Collaborator Author

/bot run

@tensorrt-cicd
Copy link
Collaborator

PR_Github #2540 [ run ] triggered by Bot

@tensorrt-cicd
Copy link
Collaborator

PR_Github #2540 [ run ] completed with state SUCCESS
/LLM/main/L0_MergeRequest_PR pipeline #1824 completed with status: 'FAILURE'

@zhengd-nv
Copy link
Collaborator Author

/bot run

@tensorrt-cicd
Copy link
Collaborator

PR_Github #2579 [ run ] triggered by Bot

@tensorrt-cicd
Copy link
Collaborator

PR_Github #2579 [ run ] completed with state SUCCESS
/LLM/main/L0_MergeRequest_PR pipeline #1841 completed with status: 'FAILURE'

@zhengd-nv
Copy link
Collaborator Author

/bot run

@tensorrt-cicd
Copy link
Collaborator

PR_Github #2624 [ run ] triggered by Bot

@tensorrt-cicd
Copy link
Collaborator

PR_Github #2624 [ run ] completed with state SUCCESS
/LLM/main/L0_MergeRequest_PR pipeline #1868 completed with status: 'FAILURE'

@zhengd-nv
Copy link
Collaborator Author

/bot run

@tensorrt-cicd
Copy link
Collaborator

PR_Github #2695 [ run ] triggered by Bot

@tensorrt-cicd
Copy link
Collaborator

PR_Github #2695 [ run ] completed with state SUCCESS
/LLM/main/L0_MergeRequest_PR pipeline #1924 completed with status: 'SUCCESS'

@Shixiaowei02 Shixiaowei02 enabled auto-merge (squash) April 18, 2025 10:16
@Shixiaowei02
Copy link
Collaborator

/bot reuse-pipeline

@tensorrt-cicd
Copy link
Collaborator

PR_Github #2761 [ reuse-pipeline ] triggered by Bot

@tensorrt-cicd
Copy link
Collaborator

PR_Github #2761 [ reuse-pipeline ] completed with state SUCCESS
Reusing PR_Github #2695 for commit 419586e

@Shixiaowei02 Shixiaowei02 merged commit bce7ea8 into NVIDIA:main Apr 18, 2025
3 checks passed
@zhengd-nv zhengd-nv deleted the disagg-cache-event branch April 21, 2025 01:33
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants