Which tests do I have to run before submitting a pool request for a new feature?

[Settheworldonfireiii]

Which tests must the new feature pass before the pull request is made?

I locally implemented output of hidden states from LLM's generate() function based on a new flag attribute that I added to SamplingParams in a separate sandboxed version of vLLM.

Before I even tried to merge/code it into the official dev version, I tried to run tests as described here:

pip install -r requirements/dev.txt

# Linting, formatting and static type checking
pre-commit install --hook-type pre-commit --hook-type commit-msg

# You can manually run pre-commit with
pre-commit run --all-files

# Unit tests
pytest tests/

on an editable vLLM version that I installed using these commands, before implementing my changes:

git clone https://github.com/vllm-project/vllm.git
cd vllm
VLLM_USE_PRECOMPILED=1 pip install --editable .

and I got a ton of errors, and that's the official dev repo's code. When I run all my University research scripts, and when I run all the commands I mentioned above **before pre-commit run --all-files ** within the same environment(that is, with dev version of vllm), I do not get any errors, and pre-commit run --all-files produces only a few ones related to lint. When I run models as a part of my research in the same environment, I never have any issues. Apparently, some of the errors are due to peculiarities of my environment, or lack of access to certain Huggingface models that I never use. Anyways, those errors pertain to some aspects that I never encounter and the feature I implemented never interacts with those.

Therefore, I wonder do I really have to pass all the tests in pytest/ directory, especially given that currently all of the errors are get are related to my conda environment rather than to the code I wrote/ feature I implemented, since I haven't yet written/merged them into the repository in question.

In addition, tests are notoriously slow to run, especially with limited compute resources.

If so, I wonder which subset of the tests do I need to pass before I create pull request? How do I determine those tests that are related to my feature (apart from those that I written myself)?

Also, how many tests of my own should I write to extensively test my feature? I know the criteria is that they should cover the usage of the feature, but still, on the average, how many tests usually does it take quantitatively?

Just for reference, the errors and the environment snapshot. I did not put it into "issues" or on Stackoverflow because my primary question is not asking to help with these errors below (although, if I have to pass all the tests, that would be appreciated), but rather to figure out which, if not all, tests does my code have to pass :

===================================================================================================================================== ERRORS =====================================================================================================================================
___________________________________________________________________________________________________________ ERROR collecting tests/compile/piecewise/test_toy_llama.py ___________________________________________________________________________________________________________
tests/compile/piecewise/test_toy_llama.py:39: in <module>
    direct_register_custom_op(
vllm/utils.py:1888: in direct_register_custom_op
    my_lib.define(op_name + schema_str)
../anaconda3/envs/dev_vllm/lib/python3.12/site-packages/torch/library.py:156: in define
    result = self.m.define(schema, alias_analysis, tuple(tags))
E   RuntimeError: Tried to register an operator (silly::attention(Tensor q, Tensor k, Tensor v, Tensor(a3!) out) -> ()) with the same name and overload name multiple times. Each overload's schema should only be registered with a single call to def(). Duplicate registration: registered at /dev/null:184. Original registration: registered at /dev/null:184
______________________________________________________________________________________________________________ ERROR collecting tests/compile/test_pass_manager.py _______________________________________________________________________________________________________________
ImportError while importing test module '/users/2/radke149/vllm/tests/compile/test_pass_manager.py'.
Hint: make sure your test modules/packages have valid Python names.
Traceback:
../anaconda3/envs/dev_vllm/lib/python3.12/importlib/__init__.py:90: in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
tests/compile/test_pass_manager.py:9: in <module>
    from vllm.compilation.config import CompilationConfig
E   ModuleNotFoundError: No module named 'vllm.compilation.config'
__________________________________________________________________________________________________________ ERROR collecting tests/distributed/test_ca_buffer_sharing.py __________________________________________________________________________________________________________
tests/distributed/test_ca_buffer_sharing.py:17: in <module>
    dist.init_process_group(backend="gloo")
../anaconda3/envs/dev_vllm/lib/python3.12/site-packages/torch/distributed/c10d_logger.py:81: in wrapper
    return func(*args, **kwargs)
../anaconda3/envs/dev_vllm/lib/python3.12/site-packages/torch/distributed/c10d_logger.py:95: in wrapper
    func_return = func(*args, **kwargs)
../anaconda3/envs/dev_vllm/lib/python3.12/site-packages/torch/distributed/distributed_c10d.py:1714: in init_process_group
    store, rank, world_size = next(rendezvous_iterator)
../anaconda3/envs/dev_vllm/lib/python3.12/site-packages/torch/distributed/rendezvous.py:263: in _env_rendezvous_handler
    rank = int(_get_env_or_raise("RANK"))
../anaconda3/envs/dev_vllm/lib/python3.12/site-packages/torch/distributed/rendezvous.py:248: in _get_env_or_raise
    raise _env_error(env_var)
E   ValueError: Error initializing torch.distributed using env:// rendezvous: environment variable RANK expected, but not set
__________________________________________________________________________________________________________ ERROR collecting tests/distributed/test_torchrun_example.py ___________________________________________________________________________________________________________
tests/distributed/test_torchrun_example.py:24: in <module>
    llm = LLM(model="facebook/opt-125m",
vllm/utils.py:1031: in inner
    return fn(*args, **kwargs)
vllm/entrypoints/llm.py:242: in __init__
    self.llm_engine = LLMEngine.from_engine_args(
vllm/engine/llm_engine.py:520: in from_engine_args
    return engine_cls.from_vllm_config(
vllm/v1/engine/llm_engine.py:111: in from_vllm_config
    return cls(vllm_config=vllm_config,
vllm/v1/engine/llm_engine.py:86: in __init__
    self.engine_core = EngineCoreClient.make_client(
vllm/v1/engine/core_client.py:68: in make_client
    return InprocClient(vllm_config, executor_class, log_stats)
vllm/v1/engine/core_client.py:164: in __init__
    self.engine_core = EngineCore(*args, **kwargs)
vllm/v1/engine/core.py:59: in __init__
    self.model_executor = executor_class(vllm_config)
vllm/executor/executor_base.py:52: in __init__
    self._init_executor()
vllm/executor/uniproc_executor.py:110: in _init_executor
    rank = int(os.environ["RANK"])
<frozen os>:714: in __getitem__
    ???
E   KeyError: 'RANK'
-------------------------------------------------------------------------------------------------------------------------------- Captured stdout ---------------------------------------------------------------------------------------------------------------------------------
INFO 03-19 14:01:32 [config.py:583] This model supports multiple tasks: {'generate', 'reward', 'embed', 'classify', 'score'}. Defaulting to 'generate'.
INFO 03-19 14:01:32 [config.py:1474] Disabling V1 multiprocessing for external launcher.
INFO 03-19 14:01:32 [config.py:1693] Chunked prefill is enabled with max_num_batched_tokens=8192.
INFO 03-19 14:01:33 [core.py:53] Initializing a V1 LLM engine (v0.8.0rc3.dev25+g228b768d) with config: model='facebook/opt-125m', speculative_config=None, tokenizer='facebook/opt-125m', skip_tokenizer_init=False, tokenizer_mode=auto, revision=None, override_neuron_config=None, tokenizer_revision=None, trust_remote_code=False, dtype=torch.float16, max_seq_len=2048, download_dir=None, load_format=LoadFormat.AUTO, tensor_parallel_size=2, pipeline_parallel_size=1, disable_custom_all_reduce=False, quantization=None, enforce_eager=False, kv_cache_dtype=auto,  device_config=cuda, decoding_config=DecodingConfig(guided_decoding_backend='xgrammar', reasoning_backend=None), observability_config=ObservabilityConfig(show_hidden_metrics=False, otlp_traces_endpoint=None, collect_model_forward_time=False, collect_model_execute_time=False), seed=0, served_model_name=facebook/opt-125m, num_scheduler_steps=1, multi_step_stream_outputs=True, enable_prefix_caching=True, chunked_prefill_enabled=True, use_async_output_proc=True, disable_mm_preprocessor_cache=False, mm_processor_kwargs=None, pooler_config=None, compilation_config={"level":3,"custom_ops":["none"],"splitting_ops":["vllm.unified_attention","vllm.unified_attention_with_output"],"use_inductor":true,"compile_sizes":[],"use_cudagraph":true,"cudagraph_num_of_warmups":1,"cudagraph_capture_sizes":[512,504,496,488,480,472,464,456,448,440,432,424,416,408,400,392,384,376,368,360,352,344,336,328,320,312,304,296,288,280,272,264,256,248,240,232,224,216,208,200,192,184,176,168,160,152,144,136,128,120,112,104,96,88,80,72,64,56,48,40,32,24,16,8,4,2,1],"max_capture_size":512}
__________________________________________________________________________________________________________ ERROR collecting tests/entrypoints/llm/test_lazy_outlines.py __________________________________________________________________________________________________________
ImportError while importing test module '/users/2/radke149/vllm/tests/entrypoints/llm/test_lazy_outlines.py'.
Hint: make sure your test modules/packages have valid Python names.
Traceback:
../anaconda3/envs/dev_vllm/lib/python3.12/importlib/__init__.py:90: in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
tests/entrypoints/llm/test_lazy_outlines.py:7: in <module>
    from vllm_test_utils import BlameResult, blame
E   ModuleNotFoundError: No module named 'vllm_test_utils'
_______________________________________________________________________________________________________________ ERROR collecting tests/kernels/test_flashinfer.py ________________________________________________________________________________________________________________
ImportError while importing test module '/users/2/radke149/vllm/tests/kernels/test_flashinfer.py'.
Hint: make sure your test modules/packages have valid Python names.
Traceback:
../anaconda3/envs/dev_vllm/lib/python3.12/importlib/__init__.py:90: in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
tests/kernels/test_flashinfer.py:5: in <module>
    import flashinfer
E   ModuleNotFoundError: No module named 'flashinfer'
____________________________________________________________________________________________________________ ERROR collecting tests/neuron/1_core/test_block_table.py ____________________________________________________________________________________________________________
ImportError while importing test module '/users/2/radke149/vllm/tests/neuron/1_core/test_block_table.py'.
Hint: make sure your test modules/packages have valid Python names.
Traceback:
../anaconda3/envs/dev_vllm/lib/python3.12/importlib/__init__.py:90: in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
tests/neuron/1_core/test_block_table.py:3: in <module>
    import neuronxcc.nki.language as nl
E   ModuleNotFoundError: No module named 'neuronxcc'
_______________________________________________________________________________________________________________ ERROR collecting tests/neuron/1_core/test_cache.py _______________________________________________________________________________________________________________
ImportError while importing test module '/users/2/radke149/vllm/tests/neuron/1_core/test_cache.py'.
Hint: make sure your test modules/packages have valid Python names.
Traceback:
../anaconda3/envs/dev_vllm/lib/python3.12/importlib/__init__.py:90: in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
tests/neuron/1_core/test_cache.py:6: in <module>
    from vllm.attention.ops.nki_flash_attn import reshape_and_cache
vllm/attention/ops/nki_flash_attn.py:3: in <module>
    import neuronxcc.nki.isa as nisa
E   ModuleNotFoundError: No module named 'neuronxcc'
_____________________________________________________________________________________________________________ ERROR collecting tests/neuron/2_core/test_comm_ops.py ______________________________________________________________________________________________________________
ImportError while importing test module '/users/2/radke149/vllm/tests/neuron/2_core/test_comm_ops.py'.
Hint: make sure your test modules/packages have valid Python names.
Traceback:
../anaconda3/envs/dev_vllm/lib/python3.12/importlib/__init__.py:90: in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
tests/neuron/2_core/test_comm_ops.py:8: in <module>
    import torch_xla.distributed.xla_multiprocessing as xmp
E   ModuleNotFoundError: No module named 'torch_xla'
______________________________________________________________________________________________________________________ ERROR collecting tests/test_utils.py ______________________________________________________________________________________________________________________
ImportError while importing test module '/users/2/radke149/vllm/tests/test_utils.py'.
Hint: make sure your test modules/packages have valid Python names.
Traceback:
../anaconda3/envs/dev_vllm/lib/python3.12/importlib/__init__.py:90: in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
tests/test_utils.py:11: in <module>
    from vllm_test_utils import monitor
E   ModuleNotFoundError: No module named 'vllm_test_utils'
_________________________________________________________________________________________________________________ ERROR collecting tests/tpu/test_compilation.py _________________________________________________________________________________________________________________
tests/tpu/test_compilation.py:64: in <module>
    assert len(compiled_codes) == 4
E   assert 0 == 4
E    +  where 0 = len([])
-------------------------------------------------------------------------------------------------------------------------------- Captured stdout ---------------------------------------------------------------------------------------------------------------------------------
INFO 03-19 14:02:35 [config.py:583] This model supports multiple tasks: {'generate', 'reward', 'embed', 'classify', 'score'}. Defaulting to 'generate'.
INFO 03-19 14:02:35 [config.py:1693] Chunked prefill is enabled with max_num_batched_tokens=8192.
WARNING 03-19 14:02:35 [cuda.py:95] To see benefits of async output processing, enable CUDA graph. Since, enforce-eager is enabled, async output processor cannot be used
INFO 03-19 14:02:36 [core.py:53] Initializing a V1 LLM engine (v0.8.0rc3.dev25+g228b768d) with config: model='Qwen/Qwen2.5-1.5B-Instruct', speculative_config=None, tokenizer='Qwen/Qwen2.5-1.5B-Instruct', skip_tokenizer_init=False, tokenizer_mode=auto, revision=None, override_neuron_config=None, tokenizer_revision=None, trust_remote_code=False, dtype=torch.bfloat16, max_seq_len=512, download_dir=None, load_format=LoadFormat.AUTO, tensor_parallel_size=1, pipeline_parallel_size=1, disable_custom_all_reduce=False, quantization=None, enforce_eager=True, kv_cache_dtype=auto,  device_config=cuda, decoding_config=DecodingConfig(guided_decoding_backend='xgrammar', reasoning_backend=None), observability_config=ObservabilityConfig(show_hidden_metrics=False, otlp_traces_endpoint=None, collect_model_forward_time=False, collect_model_execute_time=False), seed=None, served_model_name=Qwen/Qwen2.5-1.5B-Instruct, num_scheduler_steps=1, multi_step_stream_outputs=True, enable_prefix_caching=True, chunked_prefill_enabled=True, use_async_output_proc=False, disable_mm_preprocessor_cache=False, mm_processor_kwargs=None, pooler_config=None, compilation_config={"level":1,"splitting_ops":[],"compile_sizes":[],"cudagraph_capture_sizes":[],"max_capture_size":0}
WARNING 03-19 14:02:36 [utils.py:2282] Methods determine_num_available_blocks,device_config,get_cache_block_size_bytes,initialize_cache not implemented in <vllm.v1.worker.gpu_worker.Worker object at 0x7f0a33530590>
INFO 03-19 14:02:37 [parallel_state.py:967] rank 0 in world size 1 is assigned as DP rank 0, PP rank 0, TP rank 0
INFO 03-19 14:02:37 [cuda.py:215] Using Flash Attention backend on V1 engine.
INFO 03-19 14:02:37 [gpu_model_runner.py:1164] Starting to load model Qwen/Qwen2.5-1.5B-Instruct...
WARNING 03-19 14:02:38 [topk_topp_sampler.py:63] FlashInfer is not available. Falling back to the PyTorch-native implementation of top-p & top-k sampling. For the best performance, please install FlashInfer.
INFO 03-19 14:02:38 [weight_utils.py:257] Using model weights format ['*.safetensors']
INFO 03-19 14:02:38 [weight_utils.py:307] No model.safetensors.index.json found in remote.
INFO 03-19 14:02:43 [loader.py:429] Loading weights took 4.81 seconds
INFO 03-19 14:02:43 [gpu_model_runner.py:1176] Model loading took 2.8875 GB and 5.441957 seconds
INFO 03-19 14:02:45 [kv_cache_utils.py:537] GPU KV cache size: 1,175,232 tokens
INFO 03-19 14:02:45 [kv_cache_utils.py:540] Maximum concurrency for 512 tokens per request: 2295.38x
INFO 03-19 14:02:45 [core.py:138] init engine (profile, create kv cache, warmup model) took 1.59 seconds
Prompt: 'A robot may not injure a human being', Generated text: ' or, through inaction, allow a human being to come to harm.\nThe'
Prompt: 'It is only with the heart that one can see rightly;', Generated text: ' what is essential is invisible to the eye.\nJudith S. Martin\nIn'
Prompt: 'The greatest glory in living lies not in never falling,', Generated text: ' but in rising every time we fall. Which of the following is closest in meaning'
-------------------------------------------------------------------------------------------------------------------------------- Captured stderr ---------------------------------------------------------------------------------------------------------------------------------
Loading safetensors checkpoint shards:   0% Completed | 0/1 [00:00<?, ?it/s]
Loading safetensors checkpoint shards: 100% Completed | 1/1 [00:04<00:00,  4.69s/it]
Loading safetensors checkpoint shards: 100% Completed | 1/1 [00:04<00:00,  4.69s/it]

Processed prompts: 100%|██████████| 3/3 [00:00<00:00,  6.87it/s, est. speed input: 73.26 toks/s, output: 109.88 toks/s]
_________________________________________________________________________________________________________________ ERROR collecting tests/tracing/test_tracing.py _________________________________________________________________________________________________________________
ImportError while importing test module '/users/2/radke149/vllm/tests/tracing/test_tracing.py'.
Hint: make sure your test modules/packages have valid Python names.
Traceback:
../anaconda3/envs/dev_vllm/lib/python3.12/importlib/__init__.py:90: in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
tests/tracing/test_tracing.py:11: in <module>
    import grpc
E   ModuleNotFoundError: No module named 'grpc'
______________________________________________________________________________________________________________ ERROR collecting tests/v1/engine/test_engine_core.py ______________________________________________________________________________________________________________
../anaconda3/envs/dev_vllm/lib/python3.12/site-packages/huggingface_hub/utils/_http.py:406: in hf_raise_for_status
    response.raise_for_status()
../anaconda3/envs/dev_vllm/lib/python3.12/site-packages/requests/models.py:1024: in raise_for_status
    raise HTTPError(http_error_msg, response=self)
E   requests.exceptions.HTTPError: 401 Client Error: Unauthorized for url: https://huggingface.co/meta-llama/Llama-3.2-1B-Instruct/resolve/main/config.json

The above exception was the direct cause of the following exception:
../anaconda3/envs/dev_vllm/lib/python3.12/site-packages/transformers/utils/hub.py:403: in cached_file
    resolved_file = hf_hub_download(
../anaconda3/envs/dev_vllm/lib/python3.12/site-packages/huggingface_hub/utils/_validators.py:114: in _inner_fn
    return fn(*args, **kwargs)
../anaconda3/envs/dev_vllm/lib/python3.12/site-packages/huggingface_hub/file_download.py:862: in hf_hub_download
    return _hf_hub_download_to_cache_dir(
../anaconda3/envs/dev_vllm/lib/python3.12/site-packages/huggingface_hub/file_download.py:969: in _hf_hub_download_to_cache_dir
    _raise_on_head_call_error(head_call_error, force_download, local_files_only)
../anaconda3/envs/dev_vllm/lib/python3.12/site-packages/huggingface_hub/file_download.py:1484: in _raise_on_head_call_error
    raise head_call_error
../anaconda3/envs/dev_vllm/lib/python3.12/site-packages/huggingface_hub/file_download.py:1376: in _get_metadata_or_catch_error
    metadata = get_hf_file_metadata(
../anaconda3/envs/dev_vllm/lib/python3.12/site-packages/huggingface_hub/utils/_validators.py:114: in _inner_fn
    return fn(*args, **kwargs)
../anaconda3/envs/dev_vllm/lib/python3.12/site-packages/huggingface_hub/file_download.py:1296: in get_hf_file_metadata
    r = _request_wrapper(
../anaconda3/envs/dev_vllm/lib/python3.12/site-packages/huggingface_hub/file_download.py:277: in _request_wrapper
    response = _request_wrapper(
../anaconda3/envs/dev_vllm/lib/python3.12/site-packages/huggingface_hub/file_download.py:301: in _request_wrapper
    hf_raise_for_status(response)
../anaconda3/envs/dev_vllm/lib/python3.12/site-packages/huggingface_hub/utils/_http.py:423: in hf_raise_for_status
    raise _format(GatedRepoError, message, response) from e
E   huggingface_hub.errors.GatedRepoError: 401 Client Error. (Request ID: Root=1-67db14d6-6e1836ef5596aafa510426fe;bcdae3fd-8185-4e30-81af-a1194bba76a0)
E   
E   Cannot access gated repo for url https://huggingface.co/meta-llama/Llama-3.2-1B-Instruct/resolve/main/config.json.
E   Access to model meta-llama/Llama-3.2-1B-Instruct is restricted. You must have access to it and be authenticated to access it. Please log in.

The above exception was the direct cause of the following exception:
tests/v1/engine/test_engine_core.py:28: in <module>
    TOKENIZER = AutoTokenizer.from_pretrained(MODEL_NAME)
../anaconda3/envs/dev_vllm/lib/python3.12/site-packages/transformers/models/auto/tokenization_auto.py:891: in from_pretrained
    config = AutoConfig.from_pretrained(
../anaconda3/envs/dev_vllm/lib/python3.12/site-packages/transformers/models/auto/configuration_auto.py:1054: in from_pretrained
    config_dict, unused_kwargs = PretrainedConfig.get_config_dict(pretrained_model_name_or_path, **kwargs)
../anaconda3/envs/dev_vllm/lib/python3.12/site-packages/transformers/configuration_utils.py:591: in get_config_dict
    config_dict, kwargs = cls._get_config_dict(pretrained_model_name_or_path, **kwargs)
../anaconda3/envs/dev_vllm/lib/python3.12/site-packages/transformers/configuration_utils.py:650: in _get_config_dict
    resolved_config_file = cached_file(
../anaconda3/envs/dev_vllm/lib/python3.12/site-packages/transformers/utils/hub.py:421: in cached_file
    raise EnvironmentError(
E   OSError: You are trying to access a gated repo.
E   Make sure to have access to it at https://huggingface.co/meta-llama/Llama-3.2-1B-Instruct.
E   401 Client Error. (Request ID: Root=1-67db14d6-6e1836ef5596aafa510426fe;bcdae3fd-8185-4e30-81af-a1194bba76a0)
E   
E   Cannot access gated repo for url https://huggingface.co/meta-llama/Llama-3.2-1B-Instruct/resolve/main/config.json.
E   Access to model meta-llama/Llama-3.2-1B-Instruct is restricted. You must have access to it and be authenticated to access it. Please log in.
__________________________________________________________________________________________________________ ERROR collecting tests/v1/engine/test_engine_core_client.py ___________________________________________________________________________________________________________
../anaconda3/envs/dev_vllm/lib/python3.12/site-packages/huggingface_hub/utils/_http.py:406: in hf_raise_for_status
    response.raise_for_status()
../anaconda3/envs/dev_vllm/lib/python3.12/site-packages/requests/models.py:1024: in raise_for_status
    raise HTTPError(http_error_msg, response=self)
E   requests.exceptions.HTTPError: 401 Client Error: Unauthorized for url: https://huggingface.co/meta-llama/Llama-3.2-1B-Instruct/resolve/main/config.json

The above exception was the direct cause of the following exception:
../anaconda3/envs/dev_vllm/lib/python3.12/site-packages/transformers/utils/hub.py:403: in cached_file
    resolved_file = hf_hub_download(
../anaconda3/envs/dev_vllm/lib/python3.12/site-packages/huggingface_hub/utils/_validators.py:114: in _inner_fn
    return fn(*args, **kwargs)
../anaconda3/envs/dev_vllm/lib/python3.12/site-packages/huggingface_hub/file_download.py:862: in hf_hub_download
    return _hf_hub_download_to_cache_dir(
../anaconda3/envs/dev_vllm/lib/python3.12/site-packages/huggingface_hub/file_download.py:969: in _hf_hub_download_to_cache_dir
    _raise_on_head_call_error(head_call_error, force_download, local_files_only)
../anaconda3/envs/dev_vllm/lib/python3.12/site-packages/huggingface_hub/file_download.py:1484: in _raise_on_head_call_error
    raise head_call_error
../anaconda3/envs/dev_vllm/lib/python3.12/site-packages/huggingface_hub/file_download.py:1376: in _get_metadata_or_catch_error
    metadata = get_hf_file_metadata(
../anaconda3/envs/dev_vllm/lib/python3.12/site-packages/huggingface_hub/utils/_validators.py:114: in _inner_fn
    return fn(*args, **kwargs)
../anaconda3/envs/dev_vllm/lib/python3.12/site-packages/huggingface_hub/file_download.py:1296: in get_hf_file_metadata
    r = _request_wrapper(
../anaconda3/envs/dev_vllm/lib/python3.12/site-packages/huggingface_hub/file_download.py:277: in _request_wrapper
    response = _request_wrapper(
../anaconda3/envs/dev_vllm/lib/python3.12/site-packages/huggingface_hub/file_download.py:301: in _request_wrapper
    hf_raise_for_status(response)
../anaconda3/envs/dev_vllm/lib/python3.12/site-packages/huggingface_hub/utils/_http.py:423: in hf_raise_for_status
    raise _format(GatedRepoError, message, response) from e
E   huggingface_hub.errors.GatedRepoError: 401 Client Error. (Request ID: Root=1-67db14d6-447f6b6a78d6125642d8d5d9;6f0d11a9-b866-44fc-b831-0fb37f332c7f)
E   
E   Cannot access gated repo for url https://huggingface.co/meta-llama/Llama-3.2-1B-Instruct/resolve/main/config.json.
E   Access to model meta-llama/Llama-3.2-1B-Instruct is restricted. You must have access to it and be authenticated to access it. Please log in.

The above exception was the direct cause of the following exception:
tests/v1/engine/test_engine_core_client.py:28: in <module>
    TOKENIZER = AutoTokenizer.from_pretrained(MODEL_NAME)
../anaconda3/envs/dev_vllm/lib/python3.12/site-packages/transformers/models/auto/tokenization_auto.py:891: in from_pretrained
    config = AutoConfig.from_pretrained(
../anaconda3/envs/dev_vllm/lib/python3.12/site-packages/transformers/models/auto/configuration_auto.py:1054: in from_pretrained
    config_dict, unused_kwargs = PretrainedConfig.get_config_dict(pretrained_model_name_or_path, **kwargs)
../anaconda3/envs/dev_vllm/lib/python3.12/site-packages/transformers/configuration_utils.py:591: in get_config_dict
    config_dict, kwargs = cls._get_config_dict(pretrained_model_name_or_path, **kwargs)
../anaconda3/envs/dev_vllm/lib/python3.12/site-packages/transformers/configuration_utils.py:650: in _get_config_dict
    resolved_config_file = cached_file(
../anaconda3/envs/dev_vllm/lib/python3.12/site-packages/transformers/utils/hub.py:421: in cached_file
    raise EnvironmentError(
E   OSError: You are trying to access a gated repo.
E   Make sure to have access to it at https://huggingface.co/meta-llama/Llama-3.2-1B-Instruct.
E   401 Client Error. (Request ID: Root=1-67db14d6-447f6b6a78d6125642d8d5d9;6f0d11a9-b866-44fc-b831-0fb37f332c7f)
E   
E   Cannot access gated repo for url https://huggingface.co/meta-llama/Llama-3.2-1B-Instruct/resolve/main/config.json.
E   Access to model meta-llama/Llama-3.2-1B-Instruct is restricted. You must have access to it and be authenticated to access it. Please log in.
================================================================================================================================ warnings summary ================================================================================================================================
../anaconda3/envs/dev_vllm/lib/python3.12/site-packages/pytest_asyncio/plugin.py:997
../anaconda3/envs/dev_vllm/lib/python3.12/site-packages/pytest_asyncio/plugin.py:997
../anaconda3/envs/dev_vllm/lib/python3.12/site-packages/pytest_asyncio/plugin.py:997
../anaconda3/envs/dev_vllm/lib/python3.12/site-packages/pytest_asyncio/plugin.py:997
  /users/2/radke149/anaconda3/envs/dev_vllm/lib/python3.12/site-packages/pytest_asyncio/plugin.py:997: PytestDeprecationWarning: The "scope" keyword argument to the asyncio marker has been deprecated. Please use the "loop_scope" argument instead.
  
    warnings.warn(PytestDeprecationWarning(_MARKER_SCOPE_KWARG_DEPRECATION_WARNING))

tests/compile/test_basic_correctness.py:14
  /users/2/radke149/vllm/tests/compile/test_basic_correctness.py:14: PytestCollectionWarning: cannot collect test class 'TestSetting' because it has a __init__ constructor (from: tests/compile/test_basic_correctness.py)
    @dataclasses.dataclass

tests/compile/backend.py:12
  /users/2/radke149/vllm/tests/compile/backend.py:12: PytestCollectionWarning: cannot collect test class 'TestBackend' because it has a __init__ constructor (from: tests/compile/test_functionalization.py)
    class TestBackend:

tests/compile/backend.py:12
  /users/2/radke149/vllm/tests/compile/backend.py:12: PytestCollectionWarning: cannot collect test class 'TestBackend' because it has a __init__ constructor (from: tests/compile/test_fusion.py)
    class TestBackend:

tests/compile/test_fusion.py:21
  /users/2/radke149/vllm/tests/compile/test_fusion.py:21: PytestCollectionWarning: cannot collect test class 'TestModel' because it has a __init__ constructor (from: tests/compile/test_fusion.py)
    class TestModel(torch.nn.Module):

tests/entrypoints/openai/test_chat_echo.py:36
  /users/2/radke149/vllm/tests/entrypoints/openai/test_chat_echo.py:36: PytestCollectionWarning: cannot collect test class 'TestCase' because it has a __new__ constructor (from: tests/entrypoints/openai/test_chat_echo.py)
    class TestCase(NamedTuple):

tests/entrypoints/openai/test_root_path.py:38
  /users/2/radke149/vllm/tests/entrypoints/openai/test_root_path.py:38: PytestCollectionWarning: cannot collect test class 'TestCase' because it has a __new__ constructor (from: tests/entrypoints/openai/test_root_path.py)
    class TestCase(NamedTuple):

../anaconda3/envs/dev_vllm/lib/python3.12/site-packages/triton/runtime/autotuner.py:108
../anaconda3/envs/dev_vllm/lib/python3.12/site-packages/triton/runtime/autotuner.py:108
../anaconda3/envs/dev_vllm/lib/python3.12/site-packages/triton/runtime/autotuner.py:108
../anaconda3/envs/dev_vllm/lib/python3.12/site-packages/triton/runtime/autotuner.py:108
  /users/2/radke149/anaconda3/envs/dev_vllm/lib/python3.12/site-packages/triton/runtime/autotuner.py:108: DeprecationWarning: warmup, rep, and use_cuda_graph parameters are deprecated. See https://github.com/triton-lang/triton/pull/4496 for details.
    warnings.warn(("warmup, rep, and use_cuda_graph parameters are deprecated. See "

tests/kernels/test_encoder_decoder_attn.py:53
  /users/2/radke149/vllm/tests/kernels/test_encoder_decoder_attn.py:53: PytestCollectionWarning: cannot collect test class 'TestPoint' because it has a __new__ constructor (from: tests/kernels/test_encoder_decoder_attn.py)
    class TestPoint(NamedTuple):

tests/kernels/test_encoder_decoder_attn.py:80
  /users/2/radke149/vllm/tests/kernels/test_encoder_decoder_attn.py:80: PytestCollectionWarning: cannot collect test class 'TestResources' because it has a __new__ constructor (from: tests/kernels/test_encoder_decoder_attn.py)
    class TestResources(NamedTuple):

tests/lora/test_qwen2vl.py:23
  /users/2/radke149/vllm/tests/lora/test_qwen2vl.py:23: PytestCollectionWarning: cannot collect test class 'TestConfig' because it has a __init__ constructor (from: tests/lora/test_qwen2vl.py)
    @dataclass

tests/lora/test_utils.py:88
  /users/2/radke149/vllm/tests/lora/test_utils.py:88: PytestCollectionWarning: cannot collect test class 'TestLRUCache' because it has a __init__ constructor (from: tests/lora/test_utils.py)
    class TestLRUCache(LRUCache):

../anaconda3/envs/dev_vllm/lib/python3.12/site-packages/mistral_common/tokens/tokenizers/mistral.py:152
  /users/2/radke149/anaconda3/envs/dev_vllm/lib/python3.12/site-packages/mistral_common/tokens/tokenizers/mistral.py:152: FutureWarning: Calling `MistralTokenizer.from_model(..., strict=False)` is deprecated as it can lead to incorrect tokenizers. It is strongly recommended to use MistralTokenizer.from_model(..., strict=True)` which will become the default in `mistral_common=1.6.0`.If you are using `mistral_common` for open-sourced model weights, we recommend using `MistralTokenizer.from_file('<path/to/tokenizer/file>')` instead.
    warnings.warn(

../anaconda3/envs/dev_vllm/lib/python3.12/site-packages/librosa/core/intervals.py:15
  /users/2/radke149/anaconda3/envs/dev_vllm/lib/python3.12/site-packages/librosa/core/intervals.py:15: DeprecationWarning: path is deprecated. Use files() instead. Refer to https://importlib-resources.readthedocs.io/en/latest/using.html#migrating-from-legacy for migration advice.
    with resources.path("librosa.core", "intervals.msgpack") as imsgpack:

../anaconda3/envs/dev_vllm/lib/python3.12/site-packages/audioread/rawread.py:16
  /users/2/radke149/anaconda3/envs/dev_vllm/lib/python3.12/site-packages/audioread/rawread.py:16: DeprecationWarning: 'aifc' is deprecated and slated for removal in Python 3.13
    import aifc

../anaconda3/envs/dev_vllm/lib/python3.12/site-packages/audioread/rawread.py:17
  /users/2/radke149/anaconda3/envs/dev_vllm/lib/python3.12/site-packages/audioread/rawread.py:17: DeprecationWarning: 'audioop' is deprecated and slated for removal in Python 3.13
    import audioop

../anaconda3/envs/dev_vllm/lib/python3.12/site-packages/audioread/rawread.py:19
  /users/2/radke149/anaconda3/envs/dev_vllm/lib/python3.12/site-packages/audioread/rawread.py:19: DeprecationWarning: 'sunau' is deprecated and slated for removal in Python 3.13
    import sunau

tests/multimodal/test_utils.py:205
  /users/2/radke149/vllm/tests/multimodal/test_utils.py:205: PytestCollectionWarning: cannot collect test class 'TestCase' because it has a __new__ constructor (from: tests/multimodal/test_utils.py)
    class TestCase(NamedTuple):

../anaconda3/envs/dev_vllm/lib/python3.12/site-packages/astor/op_util.py:92
  /users/2/radke149/anaconda3/envs/dev_vllm/lib/python3.12/site-packages/astor/op_util.py:92: DeprecationWarning: ast.Num is deprecated and will be removed in Python 3.14; use ast.Constant instead
    precedence_data = dict((getattr(ast, x, None), z) for x, y, z in op_data)

.:0
  :0: UserWarning: /users/2/radke149/anaconda3/envs/dev_vllm/lib/python3.12/site-packages/depyf/explain/enable_debugging.py:163: You are trying to debug `torch.compile`. Please make sure the code runs multiple times to cover all the possible branches.

-- Docs: https://docs.pytest.org/en/stable/how-to/capture-warnings.html
============================================================================================================================ short test summary info =============================================================================================================================
ERROR tests/compile/piecewise/test_toy_llama.py - RuntimeError: Tried to register an operator (silly::attention(Tensor q, Tensor k, Tensor v, Tensor(a3!) out) -> ()) with the same name and overload name multiple times. Each overload's schema should only be registered with a single call to def(). Duplicate registration...
ERROR tests/compile/test_pass_manager.py
ERROR tests/distributed/test_ca_buffer_sharing.py - ValueError: Error initializing torch.distributed using env:// rendezvous: environment variable RANK expected, but not set
ERROR tests/distributed/test_torchrun_example.py - KeyError: 'RANK'
ERROR tests/entrypoints/llm/test_lazy_outlines.py
ERROR tests/kernels/test_flashinfer.py
ERROR tests/neuron/1_core/test_block_table.py
ERROR tests/neuron/1_core/test_cache.py
ERROR tests/neuron/2_core/test_comm_ops.py
ERROR tests/test_utils.py
ERROR tests/tpu/test_compilation.py - assert 0 == 4
ERROR tests/tracing/test_tracing.py
ERROR tests/v1/engine/test_engine_core.py - OSError: You are trying to access a gated repo.
ERROR tests/v1/engine/test_engine_core_client.py - OSError: You are trying to access a gated repo.
!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!! Interrupted: 14 errors during collection !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
============================================================================================================= 3 skipped, 26 warnings, 14 errors in 101.21s (0:01:41) =============================================================================================================
[rank0]:[W319 14:02:50.562728881 ProcessGroupNCCL.cpp:1496] Warning: WARNING: destroy_process_group() was not called before program exit, which can leak resources. For more info, please see https://pytorch.org/docs/stable/distributed.html#shutdown (function operator())

Environment snapshot:

INFO 03-19 14:44:54 [__init__.py:256] Automatically detected platform cuda.
Collecting environment information...
PyTorch version: 2.6.0+cu124
Is debug build: False
CUDA used to build PyTorch: 12.4
ROCM used to build PyTorch: N/A

OS: Rocky Linux release 8.10 (Green Obsidian) (x86_64)
GCC version: (GCC) 8.5.0 20210514 (Red Hat 8.5.0-23)
Clang version: Could not collect
CMake version: version 3.26.5
Libc version: glibc-2.28

Python version: 3.12.9 | packaged by Anaconda, Inc. | (main, Feb  6 2025, 18:56:27) [GCC 11.2.0] (64-bit runtime)
Python platform: Linux-4.18.0-553.8.1.el8_10.x86_64-x86_64-with-glibc2.28
Is CUDA available: True
CUDA runtime version: 12.4.131
CUDA_MODULE_LOADING set to: LAZY
GPU models and configuration: 
GPU 0: NVIDIA A100-SXM4-40GB
GPU 1: NVIDIA A100-SXM4-40GB

Nvidia driver version: 550.144.03
cuDNN version: Could not collect
HIP runtime version: N/A
MIOpen runtime version: N/A
Is XNNPACK available: True

CPU:
Architecture:        x86_64
CPU op-mode(s):      32-bit, 64-bit
Byte Order:          Little Endian
CPU(s):              64
On-line CPU(s) list: 0-63
Thread(s) per core:  1
Core(s) per socket:  64
Socket(s):           1
NUMA node(s):        4
Vendor ID:           AuthenticAMD
CPU family:          25
Model:               1
Model name:          AMD EPYC 7763 64-Core Processor
Stepping:            1
CPU MHz:             2393.433
BogoMIPS:            4890.83
Virtualization:      AMD-V
L1d cache:           32K
L1i cache:           32K
L2 cache:            512K
L3 cache:            32768K
NUMA node0 CPU(s):   0-15
NUMA node1 CPU(s):   16-31
NUMA node2 CPU(s):   32-47
NUMA node3 CPU(s):   48-63
Flags:               fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush mmx fxsr sse sse2 ht syscall nx mmxext fxsr_opt pdpe1gb rdtscp lm constant_tsc rep_good nopl nonstop_tsc cpuid extd_apicid aperfmperf pni pclmulqdq monitor ssse3 fma cx16 pcid sse4_1 sse4_2 movbe popcnt aes xsave avx f16c rdrand lahf_lm cmp_legacy svm extapic cr8_legacy abm sse4a misalignsse 3dnowprefetch osvw ibs skinit wdt tce topoext perfctr_core perfctr_nb bpext perfctr_llc mwaitx cpb cat_l3 cdp_l3 invpcid_single hw_pstate ssbd mba ibrs ibpb stibp vmmcall fsgsbase bmi1 avx2 smep bmi2 erms invpcid cqm rdt_a rdseed adx smap clflushopt clwb sha_ni xsaveopt xsavec xgetbv1 xsaves cqm_llc cqm_occup_llc cqm_mbm_total cqm_mbm_local clzero irperf xsaveerptr wbnoinvd amd_ppin brs arat npt lbrv svm_lock nrip_save tsc_scale vmcb_clean flushbyasid decodeassists pausefilter pfthreshold v_vmsave_vmload vgif v_spec_ctrl umip pku ospke vaes vpclmulqdq rdpid overflow_recov succor smca fsrm

Versions of relevant libraries:
[pip3] mypy-extensions==1.0.0
[pip3] numpy==1.26.4
[pip3] nvidia-cublas-cu12==12.4.5.8
[pip3] nvidia-cuda-cupti-cu12==12.4.127
[pip3] nvidia-cuda-nvrtc-cu12==12.4.127
[pip3] nvidia-cuda-runtime-cu12==12.4.127
[pip3] nvidia-cudnn-cu12==9.1.0.70
[pip3] nvidia-cufft-cu12==11.2.1.3
[pip3] nvidia-curand-cu12==10.3.5.147
[pip3] nvidia-cusolver-cu12==11.6.1.9
[pip3] nvidia-cusparse-cu12==12.3.1.170
[pip3] nvidia-cusparselt-cu12==0.6.2
[pip3] nvidia-nccl-cu12==2.21.5
[pip3] nvidia-nvjitlink-cu12==12.4.127
[pip3] nvidia-nvtx-cu12==12.4.127
[pip3] pyzmq==26.3.0
[pip3] sentence-transformers==3.2.1
[pip3] torch==2.6.0
[pip3] torchaudio==2.6.0
[pip3] torchvision==0.21.0
[pip3] transformers==4.48.2
[pip3] transformers-stream-generator==0.0.5
[pip3] triton==3.2.0
[pip3] tritonclient==2.51.0
[pip3] vector-quantize-pytorch==1.21.2
[conda] cuda                      12.4.0                        0    nvidia/label/cuda-12.4.0
[conda] cuda-demo-suite           12.4.99                       0    nvidia/label/cuda-12.4.0
[conda] numpy                     1.26.4                   pypi_0    pypi
[conda] nvidia-cublas-cu12        12.4.5.8                 pypi_0    pypi
[conda] nvidia-cuda-cupti-cu12    12.4.127                 pypi_0    pypi
[conda] nvidia-cuda-nvrtc-cu12    12.4.127                 pypi_0    pypi
[conda] nvidia-cuda-runtime-cu12  12.4.127                 pypi_0    pypi
[conda] nvidia-cudnn-cu12         9.1.0.70                 pypi_0    pypi
[conda] nvidia-cufft-cu12         11.2.1.3                 pypi_0    pypi
[conda] nvidia-curand-cu12        10.3.5.147               pypi_0    pypi
[conda] nvidia-cusolver-cu12      11.6.1.9                 pypi_0    pypi
[conda] nvidia-cusparse-cu12      12.3.1.170               pypi_0    pypi
[conda] nvidia-cusparselt-cu12    0.6.2                    pypi_0    pypi
[conda] nvidia-nccl-cu12          2.21.5                   pypi_0    pypi
[conda] nvidia-nvjitlink-cu12     12.4.127                 pypi_0    pypi
[conda] nvidia-nvtx-cu12          12.4.127                 pypi_0    pypi
[conda] pyzmq                     26.3.0                   pypi_0    pypi
[conda] sentence-transformers     3.2.1                    pypi_0    pypi
[conda] torch                     2.6.0                    pypi_0    pypi
[conda] torchaudio                2.6.0                    pypi_0    pypi
[conda] torchvision               0.21.0                   pypi_0    pypi
[conda] transformers              4.48.2                   pypi_0    pypi
[conda] transformers-stream-generator 0.0.5                    pypi_0    pypi
[conda] triton                    3.2.0                    pypi_0    pypi
[conda] tritonclient              2.51.0                   pypi_0    pypi
[conda] vector-quantize-pytorch   1.21.2                   pypi_0    pypi
ROCM Version: Could not collect
Neuron SDK Version: N/A
vLLM Version: 0.8.0rc3.dev25+g228b768d
vLLM Build Flags:
CUDA Archs: Not Set; ROCm: Disabled; Neuron: Disabled
GPU Topology:
GPU0	GPU1	NIC0	CPU Affinity	NUMA Affinity	GPU NUMA ID
GPU0	 X 	NV4	SYS	48,55-56	3		N/A
GPU1	NV4	 X 	PHB	0,7	0		N/A
NIC0	SYS	PHB	 X 				

Legend:

  X    = Self
  SYS  = Connection traversing PCIe as well as the SMP interconnect between NUMA nodes (e.g., QPI/UPI)
  NODE = Connection traversing PCIe as well as the interconnect between PCIe Host Bridges within a NUMA node
  PHB  = Connection traversing PCIe as well as a PCIe Host Bridge (typically the CPU)
  PXB  = Connection traversing multiple PCIe bridges (without traversing the PCIe Host Bridge)
  PIX  = Connection traversing at most a single PCIe bridge
  NV#  = Connection traversing a bonded set of # NVLinks

NIC Legend:

  NIC0: mlx5_0

CUDA_HOME=/users/2/radke149/anaconda3/envs/dev_vllm
CUDA_HOME=/users/2/radke149/anaconda3/envs/dev_vllm
CUDA_VISIBLE_DEVICES=0,1
CUDA_VISIBLE_DEVICES=0,1
NCCL_CUMEM_ENABLE=0
TORCHINDUCTOR_COMPILE_THREADS=1
CUDA_MODULE_LOADING=LAZY