Do not exclude empty arrays in source filtering with Jackson streaming

Fixes elastic#109668.

This fixes the specific case outlined in the above linked Github issue -
namely this:

* Document: `{ "myArray": [] }`
* Source filter: `myArray.myField`
* Expected filtered result: `{ "myArray": [] }`
* Actual filtered result: `{}`

When we switched to from map-based source filtering to
Jackson-streaming-based source filtering in some code paths as part of
filtering (using the automata, etc.) correctly included the empty array
from the parent. Unfortunately, it's not clear when these two code paths
for source filtering diverge in behavior.

Note also the long comment in `FilterPathBasedFilter`. It's not clear what
the correct behavior is when we are filtering out all contents of arrays
via source filter path exclusion.

Author: mccheah

libs/x-content/impl/src/main/java/org/elasticsearch/xcontent/provider/filtering/FilterPathBasedFilter.java

@@ -95,6 +95,31 @@ public TokenFilter includeProperty(String name) {
         return filter;
     }
 
+    /**
+     * This is overridden in order to keep empty arrays in nested exclusions - see #109668.
+     * <p>
+     * If we are excluding contents, we only want to exclude based on property name - but empty arrays in themselves do not have a property
+     * name. If the empty array were to be excluded, it should be done by excluding the parent.
+     * <p>
+     * Note though that the expected behavior seems to be ambiguous if contentsFiltered is true - that is, that the filter has pruned all
+     * the contents of a given array, such that we are left with the empty array. The behavior below drops that array, for at the time of
+     * writing, not doing so would cause assertions in JsonXContentFilteringTests to fail, which expect this behavior. Yet it is not obvious
+     * if dropping the empty array in this case is correct. For example, one could expect this sort of behavior:
+     * <ul>
+     *     <li>Document: <pre>{ "myArray": [ { "myField": "myValue" } ]}</pre></li>
+     *     <li>Filter: <pre>{ "exclude": "myArray.myField" }</pre></li>
+     * </ul>
+     * From the user's perspective, this could reasonably yield either of:
+     * <ol>
+     *     <li><pre>{ "myArray": []}</pre></li>
+     *     <li>Removing {@code myArray} entirely.</li>
+     * </ol>
+     */
+    @Override
+    public boolean includeEmptyArray(boolean contentsFiltered) {
+        return !inclusive && !contentsFiltered;
+    }
+
     @Override
     protected boolean _includeScalar() {
         return inclusive == false;

server/src/test/java/org/elasticsearch/search/lookup/SourceFilterTests.java

@@ -13,6 +13,7 @@
 import org.elasticsearch.test.ESTestCase;
 import org.elasticsearch.xcontent.XContentType;
 
+import java.util.Arrays;
 import java.util.List;
 import java.util.Map;
 
@@ -111,4 +112,23 @@ public Source filter(SourceFilter sourceFilter) {
 
     }
 
+    // Verification for issue #109668
+    public void testIncludeParentAndExcludeChildEmptyArray() {
+        Source fromMap = Source.fromMap(Map.of("myArray", List.of()), XContentType.JSON);
+        Source filteredMap = fromMap.filter(new SourceFilter(new String[] {"myArray"}, new String[]{"myArray.myField"}));
+        assertEquals(filteredMap.source(), Map.of("myArray", List.of()));
+        Source fromBytes = Source.fromBytes(new BytesArray("{\"myArray\": []}"), XContentType.JSON);
+        Source filteredBytes = fromBytes.filter(new SourceFilter(new String[] {"myArray"}, new String[]{"myArray.myField"}));
+        assertEquals(filteredBytes.source(), Map.of("myArray", List.of()));
+    }
+
+    public void testIncludeParentAndExcludeChildSubFields() {
+        Source fromMap = Source.fromMap(Map.of("myArray", List.of(Map.<String, Object>of("myField", "myValue", "other", "otherValue"))), XContentType.JSON);
+        Source filteredMap = fromMap.filter(new SourceFilter(new String[] {"myArray"}, new String[]{"myArray.myField"}));
+        assertEquals(filteredMap.source(), Map.of("myArray", List.of(Map.of("other", "otherValue"))));
+        Source fromBytes = Source.fromBytes(new BytesArray("""
+            { "myArray": [ { "myField": "myValue", "other": "otherValue" } ] }"""), XContentType.JSON);
+        Source filteredBytes = fromBytes.filter(new SourceFilter(new String[] {"myArray"}, new String[]{"myArray.myField"}));
+        assertEquals(filteredBytes.source(), Map.of("myArray", List.of(Map.of("other", "otherValue"))));
+    }
 }