Add Es Spark Accumulators #2159
Conversation
Counters was present for Hadoop Map Reduce, but Accumulators for Spark not.
|
Hello, could you review it and give me some feedback to progress (or not) on this feature? I suggested this pull request (2 months ago), and I haven't received any comments on it yet. But an answer on the Elastic discussion platform here with a potential acceptation (automatically closed due to 28 days of inactivity now). Could you confirm that this pull request/feature is accepted, and if yes, how to progress to merge it? Note: I mentioned 8.12 version in the doc, but this version has already been released ... |
|
Heya @glegoux, thanks so much for submitting this contribution. My sincere apologies that this hasn't been looked at, it seems to have gotten lost in the shuffle before the holidays. I will happily get a review on here soon. Thanks again so much for your patience and for the reminder ping. |
|
Any news? |
|
Having looked through the accumulator implementation, there are some concerns from us regarding counter accuracy in the event of task failures. Based on these concerns, I'm going to close this PR out. We deeply apologize that this PR has languished for so long without a proper review. It's been hard to prioritize effort toward new feature development and review on the Hadoop project compared to other endeavors that we've been undertaking at Elastic. We hope to have more time going forward to dedicate to these sorts of requests and appreciate your patience in the mean time. |
Counters was present for Hadoop Map Reduce, but Accumulators for Spark
not.
The goal of this pull request is to add it.