Skip to content

A conversion method of quant_state from 0.41.0's List to 0.42.0's QuantState as a class method of QuantState #991

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
sxndqc opened this issue Jan 27, 2024 · 0 comments
Labels
Question Further information is requested

Comments

@sxndqc
Copy link
Contributor

sxndqc commented Jan 27, 2024

Feature request

0.41.0 represents quant_state under Params4bits as a nested list with 7 elements, but in 0.42.0 this is drastically changed to a new QuantState class, and it seems some features are discarded (e.g data_type in 0.41.0), and some features are added (QuantState.code, state2 (what's its type?) ) and I'm not very sure what that means.

Motivation

Lots of local quantized model used 0.41.0's format, which makes it hard to run them on 0.42.0.

Your contribution

I still need time to figure out what state2 is (in QuantState.as_dict it seems like another nested QuantState, but how is it constructed?). Any help or guidance is appreciated on this topic!

@akx akx mentioned this issue Jan 30, 2024
@matthewdouglas matthewdouglas added the Question Further information is requested label Feb 28, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Question Further information is requested
Projects
None yet
Development

No branches or pull requests

2 participants