Skip to content

Server deps.sh unreadable xxd files to string literals. #5221

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 4 commits into from
Jan 30, 2024

Conversation

jboero
Copy link
Contributor

@jboero jboero commented Jan 30, 2024

No description provided.

HPP files are much more readable as multiline literals rather than hex arrays.
Replace . and - with _ in file names -> variable names.
XXD-> string literals
Replaced these unreadable headers with string literal versions using new deps.sh.
@ggerganov ggerganov merged commit 4003be0 into ggml-org:master Jan 30, 2024
@jboero
Copy link
Contributor Author

jboero commented Jan 30, 2024

Wow that was quick. Thanks!

@stduhpf
Copy link
Contributor

stduhpf commented Jan 30, 2024

This trows compiler error C2026 when compiling index.html.hpp and index.js.hpp with Visual Studio.
Spamming a bunch of )LITERAL" R"LITERAL( in the middle of the strings fixes it.

ggerganov added a commit that referenced this pull request Jan 30, 2024
@ggerganov
Copy link
Member

Ah, so we'll have to stick with the byte arrays then - that's unfortunate

@jboero
Copy link
Contributor Author

jboero commented Jan 30, 2024

Wait what? This should be a part of the C++11 standard which is how this is compiled. Is your Visual Studio env set to C++ version 11 or up? Please verify as it's a shame to leave this because of an IDE misconfiguration.

@stduhpf
Copy link
Contributor

stduhpf commented Jan 30, 2024

@jboero https://developercommunity.visualstudio.com/t/c-string-literal-max-length-much-shorter-than-docu/758957
It seems to be a known bug with Visual Studio's implementation sadly.

@jboero
Copy link
Contributor Author

jboero commented Jan 30, 2024

That's a shame and odd because VS Code has no such issues.

I'd love to chunk it up with the )LITERAL" R"LITERAL( blocks somehow but not sure how to automate it in deps.sh.

@cebtenzzre
Copy link
Collaborator

cebtenzzre commented Jan 30, 2024

VS Code has no such issues

We're talking about MSVC here, it's a whole separate compiler developed by Microsoft.

I'd love to chunk it up with the )LITERAL" R"LITERAL( blocks somehow but not sure how to automate it in deps.sh.

I could try to implement that if you'd like.

@jboero
Copy link
Contributor Author

jboero commented Jan 30, 2024

We're talking about MSVC here, it's a whole separate compiler developed by Microsoft.

Yes I understand and it's a great IDE actually.

If you can find a quick way to do that then great yes. I was thinking of cutting the file up into blocks of 1K or so but that may be unpredictable depending on where the file ends. Open to suggestions.

jordankanter pushed a commit to jordankanter/llama.cpp that referenced this pull request Feb 3, 2024
* Changed ugly xxd to literals.

HPP files are much more readable as multiline literals rather than hex arrays.

* Dashes in literal variable names.

Replace . and - with _ in file names -> variable names.

* Comment on removing xxd.

XXD-> string literals

* XXD to string literals.

Replaced these unreadable headers with string literal versions using new deps.sh.
jordankanter pushed a commit to jordankanter/llama.cpp that referenced this pull request Feb 3, 2024
hodlen pushed a commit to hodlen/llama.cpp that referenced this pull request Apr 1, 2024
* Changed ugly xxd to literals.

HPP files are much more readable as multiline literals rather than hex arrays.

* Dashes in literal variable names.

Replace . and - with _ in file names -> variable names.

* Comment on removing xxd.

XXD-> string literals

* XXD to string literals.

Replaced these unreadable headers with string literal versions using new deps.sh.
hodlen pushed a commit to hodlen/llama.cpp that referenced this pull request Apr 1, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants