Merge from upstream #36
Merged
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
win32/win32.c: In function 'rtc_error_handler':
win32/win32.c:691:5: warning: function 'rtc_error_handler' might be a candidate for 'gnu_printf' format attribute [-Wsuggest-attribute=format]
691 | rb_str_vcatf(str, fmt, ap);
| ^~~~~~~~~~~~
and
win32/win32.c:683:1: warning: 'rtc_error_handler' defined but not used [-Wunused-function]
683 | rtc_error_handler(int e, const char *src, int line, const char *exe, const char *fmt, ...)
| ^~~~~~~~~~~~~~~~~
* Add edit command * Make find_source a public singleton method * Add document for the edit command * Make find_end private * Remove duplicated private ruby/irb@4321674aa7 Co-authored-by: Takashi Kokubun <[email protected]>
This doesn't seem to stably work on mswin: https://github.com/ruby/ruby/actions/runs/3505363753/jobs/5871633211 For CI stability, it generally seems like a bad idea to run druby tests on Windows, given that it's pretty much unstable on MinGW as well.
You can't take rubygems for granted in a default gem. https://github.com/ruby/ruby/actions/runs/3506561943/jobs/5873689466 ruby/irb@58bb3954d0
The timeout seems too short for some CIs. http://rubyci.s3.amazonaws.com/debian11-aarch64/ruby-master/log/20221120T012840Z.fail.html.gz
As there should be no modified files just affter `git cherry-pick` succeeded in `sync_default_gems_with_commits`, reset to the previous revision once to pick up the committed files.
So that `SyntaxError#detailed_message` will be used also in the case exiting by such syntax error.
sync_default_gems.rb sometimes syncs too much.
…hods
Implementation for Language Server Protocol (LSP) sometimes needs token information.
For example both `m(1)` and `m(1, )` has same AST structure other than node locations
then it's impossible to check the existence of `,` from AST. However in later case,
it might be better to suggest variables list for the second argument.
Token information is important for such case.
This commit adds these methods.
* Add `keep_tokens` option for `RubyVM::AbstractSyntaxTree.parse`, `.parse_file` and `.of`
* Add `RubyVM::AbstractSyntaxTree::Node#tokens` which returns tokens for the node including tokens for descendants nodes.
* Add `RubyVM::AbstractSyntaxTree::Node#all_tokens` which returns all tokens for the input script regardless the receiver node.
[Feature #19070]
Impacts on memory usage and performance are below:
Memory usage:
```
$ cat test.rb
root = RubyVM::AbstractSyntaxTree.parse_file(File.expand_path('../test/ruby/test_keyword.rb', __FILE__), keep_tokens: true)
$ /usr/bin/time -f %Mkb /usr/local/bin/ruby -v
ruby 3.2.0dev (2022-11-19T09:41:54Z 19070-keep_tokens d3af1b8) [x86_64-linux]
11408kb
# keep_tokens :false
$ /usr/bin/time -f %Mkb /usr/local/bin/ruby test.rb
17508kb
# keep_tokens :true
$ /usr/bin/time -f %Mkb /usr/local/bin/ruby test.rb
30960kb
```
Performance:
```
$ cat ../ast_keep_tokens.yml
prelude: |
src = <<~SRC
module M
class C
def m1(a, b)
1 + a + b
end
end
end
SRC
benchmark:
without_keep_tokens: |
RubyVM::AbstractSyntaxTree.parse(src, keep_tokens: false)
with_keep_tokens: |
RubyVM::AbstractSyntaxTree.parse(src, keep_tokens: true)
$ make benchmark COMPARE_RUBY="./ruby" ARGS=../ast_keep_tokens.yml
/home/kaneko.y/.rbenv/shims/ruby --disable=gems -rrubygems -I../benchmark/lib ../benchmark/benchmark-driver/exe/benchmark-driver \
--executables="compare-ruby::./ruby -I.ext/common --disable-gem" \
--executables="built-ruby::./miniruby -I../lib -I. -I.ext/common ../tool/runruby.rb --extout=.ext -- --disable-gems --disable-gem" \
--output=markdown --output-compare -v ../ast_keep_tokens.yml
compare-ruby: ruby 3.2.0dev (2022-11-19T09:41:54Z 19070-keep_tokens d3af1b8) [x86_64-linux]
built-ruby: ruby 3.2.0dev (2022-11-19T09:41:54Z 19070-keep_tokens d3af1b8) [x86_64-linux]
warming up..
| |compare-ruby|built-ruby|
|:--------------------|-----------:|---------:|
|without_keep_tokens | 21.659k| 21.303k|
| | 1.02x| -|
|with_keep_tokens | 6.220k| 5.691k|
| | 1.09x| -|
```
* s/Innteger/Integer/ * s/diretory/directory/ * s/Bufer/Buffer/ * s/defalt/default/ * s/covearge/coverage/
Asked by ko1 to release ruby/irb#444 for simplifying ruby/debug#808, and hsbt made me a gem owner for this. Stan said 1.4.3 should have been 1.5.0, but now that it's already released and it's not worth yanking it, we're not doing that change. However, now that this release includes `debug` and `edit`, I think it's a good opportunity to hit the version 1.5.0. ruby/irb@85937d71f6
(ruby/irb#449) * Seamlessly integrate a few debug commands * Improve the break command support * Utilize skip_src option if available * Add step and delete commands * Write end-to-end tests for each debugger command * Add documentation * Add backtrace, info, catch commands ruby/irb@976100c1c2
obj_ivar_set and vm_setivar_slowpath is essentially doing the same thing, but the code is duplicated and not quite implemented in the same way, which could cause bugs. This commit refactors vm_setivar_slowpath to use obj_ivar_set.
This commit significantly speeds up shape transitions as it changes
get_next_shape_internal to not perform a lookup (and instead require
the caller to perform the lookup). This avoids double lookups during
shape transitions.
There is a significant (~2x) speedup in the following micro-benchmark:
puts(Benchmark.measure do
o = Object.new
100_000.times do |i|
o.instance_variable_set(:"@A#{i}", 0)
end
end)
Before:
22.393194 0.201639 22.594833 ( 22.684237)
After:
11.323086 0.022284 11.345370 ( 11.389346)
Build and store the error message with `#detailed_message` before terminating all Ractors, then show the message later.
Raise a `SyntaxError` with the parser error message, in the case reading from a file instead of the `-e` option or standard input. So syntax_suggest can get the message from the caught error.
This commit adds RVALUE_OVERHEAD for storing metadata at the end of the slot. This commit moves the ractor_belonging_id in debug builds from the flags to RVALUE_OVERHEAD which frees the 16 bits in the headers for object shapes.
The ractor_belonging_id has been moved out of the headers, so object shapes can take the top 32 bits of the flags on debug builds.
Throw a RuntimeError if the HTTP response header contains CR or LF to prevent HTTP response splitting. https://hackerone.com/reports/1204695 ruby/cgi@64c5045c0a
…ze (ruby#6788) We can loosely predict the number of ivar sets on a class based on the number of iv set instructions in the initialize method. This should give us a more accurate estimate to use for initial size pool allocation, which should in turn give us more cache hits.
YJIT: Skip padding jumps to side exits Co-authored-by: Maxime Chevalier-Boisvert <[email protected]> Co-authored-by: Alan Wu <[email protected]> Co-authored-by: Maxime Chevalier-Boisvert <[email protected]> Co-authored-by: Alan Wu <[email protected]>
The removed code is a duplicate of the code above.
We need to make sure the name table is freed otherwise we have a memory leak.
wks
pushed a commit
that referenced
this pull request
Aug 4, 2023
[Bug #19793]
Dummy frames are created at the top level when requiring another file.
While requiring a file, it will try to convert using encodings. Some of
these encodings will not respond to to_str. If method_missing is
redefined on Object, then it will call method_missing and attempt raise
an error. However, the iseq is invalid as it's a dummy frame so it will
write an invalid iseq to the created NoMethodError.
The following script crashes:
```
GC.stress = true
class Object
public :method_missing
end
File.write("/tmp/empty.rb", "")
require "/tmp/empty.rb"
```
With the following backtrace:
```
frame #0: 0x00000001000fa8b8 miniruby`RVALUE_MARKED(obj=4308637824) at gc.c:1638:12
frame #1: 0x00000001000fb440 miniruby`RVALUE_BLACK_P(obj=4308637824) at gc.c:1763:12
frame #2: 0x00000001000facdc miniruby`gc_writebarrier_incremental(a=4308637824, b=4308332208, objspace=0x000000010180b000) at gc.c:8822:9
frame #3: 0x00000001000faad8 miniruby`rb_gc_writebarrier(a=4308637824, b=4308332208) at gc.c:8864:17
frame #4: 0x000000010016aff0 miniruby`rb_obj_written(a=4308637824, oldv=36, b=4308332208, filename="../iseq.c", line=1279) at gc.h:804:9
frame #5: 0x0000000100162a60 miniruby`rb_obj_write(a=4308637824, slot=0x0000000100d09888, b=4308332208, filename="../iseq.c", line=1279) at gc.h:837:5
frame #6: 0x0000000100165b0c miniruby`iseqw_new(iseq=0x0000000100d09880) at iseq.c:1279:9
frame #7: 0x0000000100165a64 miniruby`rb_iseqw_new(iseq=0x0000000100d09880) at iseq.c:1289:12
frame #8: 0x00000001000d8324 miniruby`name_err_init_attr(exc=4309777920, recv=4304780496, method=827660) at error.c:1830:35
frame #9: 0x00000001000d1b80 miniruby`name_err_init(exc=4309777920, mesg=4308332496, recv=4304780496, method=827660) at error.c:1869:12
frame #10: 0x00000001000d1bd4 miniruby`rb_nomethod_err_new(mesg=4308332496, recv=4304780496, method=827660, args=4308332448, priv=0) at error.c:1957:5
frame #11: 0x000000010039049c miniruby`rb_make_no_method_exception(exc=4304914512, format=4308332496, obj=4304780496, argc=1, argv=0x000000016fdfab00, priv=0) at vm_eval.c:959:16
frame #12: 0x00000001003b3274 miniruby`raise_method_missing(ec=0x0000000100b06f40, argc=1, argv=0x000000016fdfab00, obj=4304780496, last_call_status=MISSING_NOENTRY) at vm_eval.c:999:15
frame #13: 0x00000001003945d4 miniruby`rb_method_missing(argc=1, argv=0x000000016fdfab00, obj=4304780496) at vm_eval.c:944:5
...
frame #23: 0x000000010038f5e4 miniruby`rb_vm_call_kw(ec=0x0000000100b06f40, recv=4304780496, id=2865, argc=1, argv=0x000000016fdfab00, me=0x0000000100cbfcf0, kw_splat=0) at vm_eval.c:326:12
frame #24: 0x00000001003c18e4 miniruby`call_method_entry(ec=0x0000000100b06f40, defined_class=4304927952, obj=4304780496, id=2865, cme=0x0000000100cbfcf0, argc=1, argv=0x000000016fdfab00, kw_splat=0) at vm_method.c:2720:20
frame #25: 0x00000001003c440c miniruby`check_funcall_exec(v=6171896792) at vm_eval.c:589:12
frame #26: 0x00000001000dec00 miniruby`rb_vrescue2(b_proc=(miniruby`check_funcall_exec at vm_eval.c:587), data1=6171896792, r_proc=(miniruby`check_funcall_failed at vm_eval.c:596), data2=6171896792, args="Pȗ") at eval.c:919:18
frame #27: 0x00000001000deab0 miniruby`rb_rescue2(b_proc=(miniruby`check_funcall_exec at vm_eval.c:587), data1=6171896792, r_proc=(miniruby`check_funcall_failed at vm_eval.c:596), data2=6171896792) at eval.c:900:17
frame #28: 0x000000010039008c miniruby`check_funcall_missing(ec=0x0000000100b06f40, klass=4304923536, recv=4304780496, mid=3233, argc=0, argv=0x0000000000000000, respond=-1, def=36, kw_splat=0) at vm_eval.c:666:15
frame #29: 0x000000010038fa60 miniruby`rb_check_funcall_default_kw(recv=4304780496, mid=3233, argc=0, argv=0x0000000000000000, def=36, kw_splat=0) at vm_eval.c:703:21
frame #30: 0x000000010038fb04 miniruby`rb_check_funcall(recv=4304780496, mid=3233, argc=0, argv=0x0000000000000000) at vm_eval.c:685:12
frame #31: 0x00000001001c469c miniruby`convert_type_with_id(val=4304780496, tname="String", method=3233, raise=0, index=-1) at object.c:3061:15
frame #32: 0x00000001001c4a4c miniruby`rb_check_convert_type_with_id(val=4304780496, type=5, tname="String", method=3233) at object.c:3153:9
frame #33: 0x00000001002d59f8 miniruby`rb_check_string_type(str=4304780496) at string.c:2571:11
frame #34: 0x000000010014b7b0 miniruby`io_encoding_set(fptr=0x0000000100d09ca0, v1=4304780496, v2=4, opt=4) at io.c:11655:19
frame #35: 0x0000000100139a58 miniruby`rb_io_set_encoding(argc=1, argv=0x000000016fdfb450, io=4308334032) at io.c:13497:5
frame #36: 0x00000001003c0004 miniruby`ractor_safe_call_cfunc_m1(recv=4308334032, argc=1, argv=0x000000016fdfb450, func=(miniruby`rb_io_set_encoding at io.c:13487)) at vm_insnhelper.c:3271:12
...
frame #43: 0x0000000100390b08 miniruby`rb_funcall(recv=4308334032, mid=16593, n=1) at vm_eval.c:1137:12
frame #44: 0x00000001002a43d8 miniruby`load_file_internal(argp_v=6171899936) at ruby.c:2500:5
...
```
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Resolves merge conflict in
gc.c