PutMappingAsync fails with Bad Request when PutMappingRequest.DynamicTemplates contains just one element

[yansklyarenko]

Elastic.Clients.Elasticsearch version: 8.10.0

Elasticsearch version: 8.2.2

.NET runtime version: net6.0, net7.0

Operating system version: Windows 10

Description of the problem including expected versus actual behavior:
Consider the following code sample:

var myTemplate = new DynamicTemplate
{
    PathMatch = "testPathMatch",
    Mapping = new KeywordProperty()
};

var putMappingRequest = new PutMappingRequest("my-index")
{
    DynamicTemplates = new[]
    {
        new Dictionary<string, DynamicTemplate>
        {
            { "testTemplateName", myTemplate }
        }
    }
};

var response = await client.Indices.PutMappingAsync(putMappingRequest).ConfigureAwait(false);

Actual behavior
The response is invalid, containing 400 bad request:

Invalid Elasticsearch response built from a unsuccessful (400) low level call on PUT: /repository-integration-tests/_mapping?pretty=true&error_trace=true
 Exception: Request failed to execute. Call: Status code 400 from: PUT /repository-integration-tests/_mapping?pretty=true&error_trace=true. ServerError: Type: mapper_parsing_exception Reason: "Failed to parse mapping: Dynamic template syntax error. An array of named objects is expected." CausedBy: "Type: mapper_parsing_exception Reason: "Dynamic template syntax error. An array of named objects is expected.""

Expected behavior
The appropriate mapping is added to the index.

Some details which might be useful
As soon as I add another element to the DynamicTemplates array, the request passes just fine and the mapping is created. This is because the DynamicTemplates property is decorated with [SingleOrManyCollectionConverter(typeof(IDictionary<string, DynamicTemplate>))], which serializes the array with a single element as an object, not as an array.

Based on this comment from another issue, I make a conclusion that it was done on purpose. So, it is not clear whether it is the 'put mapping' endpoint problem that it doesn't understand the new body format, or the serialization logic.

Please, let me know if you need some more details.

[flobernd]

Hi @yansklyarenko, thanks for the detailed report.

In the specification, the DynamicTemplates property is explicitly defined like this:

dynamic_templates?: Record<string, MappingDynamicTemplate> | Record<string, MappingDynamicTemplate>[]

This might be incorrect. I'll try to confirm by looking in the ES server code.

[anghelnicolae]

I've also encountered the same bug. Not fixed in version 8.12.

[yansklyarenko]

I have found a workaround for this issue. When issuing a put mapping request, you can check whether the DynamicTemplates property contains just one template, and in this case add a dummy one, which doesn't change the overall result. For example:

if (putMappingRequest.DynamicTemplates != null
 && putMappingRequest.DynamicTemplates.Count == 1)
{
    // DynamicTemplates property is serialized as an object in case the collection has just one element
    // This results in a bad request: https://github.com/elastic/elasticsearch-net/issues/7968
    // This workaround is to add a dummy "influence nothing" dynamic template to let it serialize as array
    putMappingRequest.DynamicTemplates.Add(
        new Dictionary<string, DynamicTemplate>
        {
            {
                "testTemplateName",
                new DynamicTemplate
                {
                    PathMatch = "dummynonexistingproperty",
                    Mapping = new KeywordProperty()
                }
            }
        });
}

[durlandd]

#7337 addresses the defect related to the TypeMapping generated code. It does not address the same issue found in the PutMappingRequest code. The [SingleOrManyCollectionConverter(typeof(IReadOnlyDictionary<string, Elastic.Clients.Elasticsearch.Mapping.DynamicTemplate>))] attribute is still applied to the DynamicTemplates property.

[lucacremonesi]

What is the state of this bug?
It has been reported more than one year ago and the suggested workaround is definitely ugly.
That means that a simple PUT mapping request is broken using the new .NET client.

[flobernd]

@l-trotta Do you have the same issue with the Java client?

[l-trotta]

@flobernd tested the same request with the java client, seems to be working normally:

var template = DynamicTemplate.of(d -> d.mapping(m -> m.keyword(k -> k)).pathMatch("test"));
var putMappingRequest = PutMappingRequest.of(p -> p.index("my-index").dynamicTemplates(Map.of("testTemplate",template)));
var response = esClient.indices().putMapping(putMappingRequest);

[flobernd]

We just confirmed that the specification is indeed wrong about this one. This will be fixed in the client after the specification PR is merged and a new version of the client is released.