Remove java-ification of objects in the Java Event class from JRuby

[talevy]

Currently, anytime you set or get values from the Event class, you either set or receive copies of your data which have been deeply copied and converted into the correct types for their respective runtime contexts. The main reason this was done was to create a strict division between what is in ruby-land and java-land. This meant two things; 1) it is simpler to reason about types from within the Event object and 2) any future plugins that used a raw java API to interact with the pipeline would not be contaminated with Ruby objects that it does not know what to do with.

The Proposal made in this issue is to stop doing this conversion in the context of JRuby. In short, there is no reason why the Event object's internal Map cannot contain references to IRubyObjects. There may be some data structures that need some wrapping, but both RubyArray and RubyHash implement Java library interfaces like List and Map (respectively). Since this is the case, we mainly have to only worry about the keys of these objects. For example, internal RubyHash objects need to become compatible for FieldReference retrieval by the Accessors class.

Changes needed in Accessors/PathCache/FieldReference

• Ability to extract the responsibility of parsing the string references (e.g. "[foo][bar]") into RubyString components will enable the nested key lookups that occur in Accessors against RubyHashes.
  • I propose that this responsibility of FieldReference.parse be moved up into ruby_set_field and ruby_get_field of JrubyEventExtLibrary. This way we remove the need to pass a Ruby Runtime Context to javaland.

TODO

• continue evaluating what steps and situations are missing from the above plan
• introduce ruby-eval tests against Event from within logstash-core-event-java instead of just relying on rspec tests outside of the project.
  • involves: introducing src/test/resources/fixtures ruby scripts to be executed against jruby runtimes from within junit tests

[guyboertje]

@talevy - I have already started this...
Issue: #4834
PR: #4999

[colinsurprenant]

@talevy can you see if what is proposed in #4834 / #4999 solves (some of) your concerns and we can move the discussion there?

[talevy]

@guyboertje @colinsurprenant , didn't see that! I'll take a look

[guyboertje]

@colinsurprenant, @talevy :

• It does not solve the concerns of treating an event like a Ruby Hash.
• It does provide a glimpse into what is possible - using a value object holding a Ruby value and a Java value.
• It begs the question: in the Java Event, what if data (and metadata) was a dual value object that held the ruby hash and the java map (initially empty) and we had accessors that operated on one or the other? With a clever way of switching to the applicable accessor and a means to invalidate the other if one mutates.

[andrewvc]

@guyboertje your third point makes a lot of sense to me!

I don't know how helpful this is, but we do, in Logstash, have the capability of knowing when we're in Javaland vs. Rubyland. We could have the pipeline 'switch' the event between one of two types or modes depending on the plugin language.

[colinsurprenant]

I created issue #5035 to discuss the field reference syntax API & design and the support of implementation and language specific assumptions. Or in other words, Event should definitely not be treated like a Ruby Hash, IMO it's bad design moving forward.

[guyboertje]

In JrubyEventExtLibrary we defined a wrapper for Java Event called RubyEvent - we can switch in that.

[suyograo]

@talevy can we close this now in lieu of #5035 and #5141

[talevy]

yes. closing for reasons stated above ^^^