Skip to content

Improve syntax highlighting in code samples a bit! #160

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Conversation

thennothinghappened
Copy link
Contributor

I was having a look at the manual and again noticed how bare-bones the syntax highlighting is, how it is missing keywords such as new and static, how it doesn't highlight variable colours, struct members... and so on. I decided today to take it upon myself to have a go learning some of highlight.js's s way of defining language highlighers and use that to improve the manual's highlighting, as it uses this library for it.

The changes are as follows, with before and after images!

  1. new keyword is highlighted:

Before:
image

After:
image

  1. #macro, #region, #endregion are highlighted, including comment highlighting for the latter two, and constant colouring for the identifier for macros:

Before:
image
image

After:
image
image

  1. Multi-line strings:

Before:
image

...note that it didn't even pick up GML as the language here as it didn't see it as valid syntax.

After:
image

  1. Variable declarations, and struct literal members:

Before:
image
image

After:
image
image

  1. Function declarations and call sites (for user functions):

Before:
image

After:
image

  1. throw keyword is highlighted:

Before:
image

After:
image

  1. Property accesses highlighted, with special colouration for global:

Before:
image
image

After:
image
image

  1. Sneaky highlighting for obj_*, spr_* and shader_* as asset constant colours:

Before:
image

After:
image

@thennothinghappened
Copy link
Contributor Author

I've also just added try, catch and finally which were missing. Another thing I'd like to point out is that since functions are now detected by call signature, that means anything that looks like this(/*...*/) will be highlighted, also applying to built-in functions of course.

There's a massive master list of every function in the engine in the gml.js file, which I can't tell if it is used for any other purpose than to manually highlight them. If that's the only reason it exists, this can now be safely removed, to the death of much manual data-entry!

@thennothinghappened
Copy link
Contributor Author

Added support for enums!

image image

I've also gone and added some documentation for the various modes I've added, as they're not too self-explanatory at first glance.

Finally, I added some negative look-behind and look-ahead assertions to the VALID_IDENTIFIER_REG regex, because it was getting false-positives on <br />, and was matching the br in there as a variable which isn't correct obviously. I might have a look at a better way of handling this as I'm not too comfortable with it working that way, since these assertions to my knowledge are only supported in the latest browser versions, which might break the manual(!) for people on outdated browsers. This false positive issue has come up a few times and I'd worked around it, but it was a big issue for enum entries so I added those assertions in.

I'm going to have a look if I can do some sort of pre-processing on the input to steal all the <br /> instances and replace them with proper line breaks, but really the solution to this would be if the manual switched its code sample blocks to use white-space: pre-wrap, ditched all the non-breaking spaces and formatted code samples more properly in the HTML. My understanding however is that RoboHelp is some sort of WYSIWYG editor(?) which makes this not really an option as it would be responsible for the incorrect formatting.

If I can't find a good solution, and it seems apparent that the assertions are not widely supported enough, then I'll remove them and turn off the enum entry colouring, alongside any additional more complex state-based colouring I'd otherwise like to add.

Removed the need for the negative look-behind and look-ahead assertions. This means we don't have to worry about the difference between `\n` and `<br>` during highlighting, and should make it faster and more supported across browsers again.
@thennothinghappened
Copy link
Contributor Author

Alrighty, I'm back again!

I've made a slight modification to the highlight.js script - which by the looks of things someone had already made a similar, though slightly different modification at some point earlier - and made sure that the language definition regex only gets passed \n instead of <br>. This was pretty easy to do and means that nasty lookbehind assertion is gone, and the whole thing is a lot simpler and more stable.

@thennothinghappened
Copy link
Contributor Author

Added support for GML's underscores in number literals, and $ and # colour hex!
image
image
image
image

@gnysek
Copy link

gnysek commented Aug 19, 2024

I made few pull requests for highlight.js from time to time, updating GML ( https://github.com/highlightjs/highlight.js/pulls?q=is%3Apr+author%3Agnysek+is%3Aclosed ), but with it rapid updates it's for sure out of date again. Latest version I've updated was 2024.2 - so if you have any newer definitions file, would be nice to update it in hightlight.js repo too! :)

@thennothinghappened
Copy link
Contributor Author

I made few pull requests for highlight.js from time to time, updating GML ( https://github.com/highlightjs/highlight.js/pulls?q=is%3Apr+author%3Agnysek+is%3Aclosed ), but with it rapid updates it's for sure out of date again. Latest version I've updated was 2024.2 - so if you have any newer definitions file, would be nice to update it in hightlight.js repo too! :)

Heya! I thought about porting it across but I'm unfamiliar with contributing to that project - I've made substantial changes to how the language definition for GML works and might have a few more edge cases to sort out I've not found. I'm now quite sure the manual list of function names in there is made unnecessary by my changes so will go ahead with removing it.

If/when this PR is accepted, I'll happily open one in highlight.js itself to backport my changes, which should make the highlighting in Discord etc. much better (eventually!) :)

We now detect function call sites by signature, not by a manual list of what functions exist, so this is no longer necessary!
@thennothinghappened
Copy link
Contributor Author

Just gone ahead with removing that massive function list as I'm now quite certain it was only used for manually highlighting function calls. Since that's now done by signature, I've removed that list and it won't need manual updating anymore :)

@gurpreetsinghmatharoo gurpreetsinghmatharoo merged commit 72a85a4 into YoYoGames:develop Aug 20, 2024
1 check failed
@gurpreetsinghmatharoo
Copy link
Collaborator

Merged. Thank you so much for your work!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants