Improve configs - SchedulerConfig

[hmellor]

Follows #16332 and #16422

Notable changes:

• Updates ParallelConfig.distributed_executor_backend to use Literal[...] instead of str
• Rename nullable_str to optional_str (which sounds more Pythonic) and add other optional type methods
• Make the conversion of config docstring to argparse arguments much more robust
• The default for multi_step_stream_outputs was True in EngineArgs and False in SchedulerConfig. I chose the config from EngineArgs

[github-actions]

👋 Hi! Thank you for contributing to the vLLM project.

💬 Join our developer Slack at https://slack.vllm.ai to discuss your PR in #pr-reviews, coordinate on features in #feat- channels, or join special interest groups in #sig- channels.

Just a reminder: PRs would not trigger full CI run by default. Instead, it would only run fastcheck CI which starts running only a small and essential subset of CI tests to quickly catch errors. You can run other CI tests on top of those by going to your fastcheck build on Buildkite UI (linked in the PR checks section) and unblock them. If you do not have permission to unblock, ping simon-mo or khluu to add you in our Buildkite org.

Once the PR is approved and ready to go, your PR reviewer(s) can run CI to test the changes comprehensively before merging.

To run CI, PR reviewers can either: Add ready label to the PR or enable auto-merge.

🚀

[hmellor]

Failures are relevant, I'll look into it

[DarkLight1337]

Since None is resolved inside EngineArgs or ModelConfig, I feel that we should not allow it to be None when initializing SchedulerConfig

[hmellor]

The default is None so that the default in EngineArgs is None.

We have the same problem with the max_num_batched_tokens and max_model_length. It can't be optional in the config because maths is done on them in __post_init__, but they must have None defaults so that EngineArgs can have them as optional.

Where would you change behaviour so that the defaults here are not None?

[DarkLight1337]

Maybe we can have a from_optional method just like the one for SamplingParams to resolve the None values

[DarkLight1337]

Using post init to overwrite None values leads to a bunch of issues with type checking since downstream access of these attributes will unnecessarily need to consider the None case in order to avoid type checking errors.

[hmellor]

Maybe we can have a from_optional method just like the one for SamplingParams to resolve the None values

Would we then need to change everywhere SchedulingConfig is instantiated? And then remember to instantiate it that way?

Using post init to overwrite None values leads to a bunch of issues with type checking since downstream access of these attributes will unnecessarily need to consider the None case in order to avoid type checking errors.

I agree, but that's why there in SchedulingConfig the type is int but in EngineArgs the type is Optional[int], so that the type cheking is happy.

[hmellor]

I think it's worth noting that prior to this PR, this is how we handled max _num_batched_tokens.

I changed the other two here so that they would no longer have arbitrary defaults that could be inherited by EngineArgs

[DarkLight1337]

As per offline discussion, this is the cleanest way given how the configs are set up right now. We can address this later