Skip to content

JSON decoding issues after upgrade to Spring Cloud 2023.0.2 #2158

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
kschlesselmann opened this issue Jun 3, 2024 · 18 comments
Closed

JSON decoding issues after upgrade to Spring Cloud 2023.0.2 #2158

kschlesselmann opened this issue Jun 3, 2024 · 18 comments

Comments

@kschlesselmann
Copy link

After upgrading to Spring Cloud 2023.0.2 a lot of tests in our services fail with

`JSON decoding error: Trailing token (of type FIELD_NAME) found after value (bound as `org.springframework.hateoas.Link`): not allowed as per `DeserializationFeature.FAIL_ON_TRAILING_TOKENS`

if they try to read the response body to a class that inherits from RepresentationModel.

As far as I can tell the HypermediaWebTestClientConfigurer is somehow involved. If I reomve it from our WebTestClient the response body can be read and the test fails due to class mismatches.

Everything here is on the reactive stack. Do you have any suggestions what's going on here?

@pcornelissen
Copy link

pcornelissen commented Jun 6, 2024

Using WebMVC the same error occured. I reverted back to 2023.0.1 until I know hot to fix that.
And I'm using the generic wrappers and not the inheritance

@odrotbohm
Copy link
Member

Can you provide a minimal reproducer that shows the error in a test case?

@pcornelissen
Copy link

pcornelissen commented Jul 8, 2024

As it prevents me from upgrading, I started to dig deeper and it gets weirder. When the service runs in intelliJ I can't trigger the exception. It just happen
s when the service is run via mvn outside of the IDE and sadly when it runs in the cluster...
(I can reproduce it in intelliJ now as well)

@pcornelissen
Copy link

pcornelissen commented Jul 8, 2024

OK, more digging.
In my case it was triggered by a request in a webMVC application using webclient (as this service is older than restclient...). This I had no helpful context in the exception. I rewrote the part to use restclient and I could find that this is a request to another service, where I map a part of the response to an entity.

The response is:

{"id":"bfd1d723-0656-4b34-96a4-ce736c018eb3","version":14,"name":"Hydraulik Schneider Demo","logoUrl":"fd70beb2-c64e-4551-8607-afb7adaf1a3e","logoAssetId":null,"homepageUrl":"http://hydraulik-schneider.de","publicContentDomain":"http://hydraulikschneider.public.localhost","billingAddress":{"name":"Hydraulik Schneider AG","additionalNameRow":null,"street":"Beispielstraße","houseIdentifier":"42z","zipCode":"54321","city":"Beispielhausen","state":null,"country":"Deutschland"},"presets":{"colors":{"primary":"#00589C"},"general":{},"blockDefaults":null,"activeCss":null,"draftCss":null,"cssClassNames":null,"imprint":null,"dataProtectionOfficer":null},"auditMetaData":{"createdBy":"demo","createdDate":"2024-05-28T18:15:21.255939Z","modifiedBy":"[email protected]","modificationDate":"2024-07-03T11:02:09.989323Z"},"_links":{"self":{"href":"http://localhost:9087/bfd1d723-0656-4b34-96a4-ce736c018eb3"},"fetchCustomCss":{"href":"http://localhost:9087/bfd1d723-0656-4b34-96a4-ce736c018eb3/customcss{?draft}","templated":true},"fetchCustomCssNames":{"href":"http://localhost:9087/bfd1d723-0656-4b34-96a4-ce736c018eb3/customcss/names"},"fetchViaContentDomain":{"href":"http://localhost:9087/?contentDomain=http%3A%2F%2Fhydraulikschneider.public.localhost"},"updateTenant":{"href":"http://localhost:9087/bfd1d723-0656-4b34-96a4-ce736c018eb3","type":"PUT"}}}

This is not readable, so here is it formatted:

{
  "id": "bfd1d723-0656-4b34-96a4-ce736c018eb3",
  "version": 14,
  "name": "Hydraulik Schneider Demo",
  "logoUrl": "fd70beb2-c64e-4551-8607-afb7adaf1a3e",
  "logoAssetId": null,
  "homepageUrl": "http://hydraulik-schneider.de",
  "publicContentDomain": "http://hydraulikschneider.public.localhost",
  "billingAddress": {
    "name": "Hydraulik Schneider AG",
    "additionalNameRow": null,
    "street": "Beispielstraße",
    "houseIdentifier": "42z",
    "zipCode": "54321",
    "city": "Beispielhausen",
    "state": null,
    "country": "Deutschland"
  },
  "presets": {
    "colors": {
      "primary": "#00589C"
    },
    "general": {},
    "blockDefaults": null,
    "activeCss": null,
    "draftCss": null,
    "cssClassNames": null,
    "imprint": null,
    "dataProtectionOfficer": null
  },
  "auditMetaData": {
    "createdBy": "demo",
    "createdDate": "2024-05-28T18:15:21.255939Z",
    "modifiedBy": "[email protected]",
    "modificationDate": "2024-07-03T11:02:09.989323Z"
  },
  "_links": {
    "self": {
      "href": "http://localhost:9087/bfd1d723-0656-4b34-96a4-ce736c018eb3"
    },
    "fetchCustomCss": {
      "href": "http://localhost:9087/bfd1d723-0656-4b34-96a4-ce736c018eb3/customcss{?draft}",
      "templated": true
    },
    "fetchCustomCssNames": {
      "href": "http://localhost:9087/bfd1d723-0656-4b34-96a4-ce736c018eb3/customcss/names"
    },
    "fetchViaContentDomain": {
      "href": "http://localhost:9087/?contentDomain=http%3A%2F%2Fhydraulikschneider.public.localhost"
    },
    "updateTenant": {
      "href": "http://localhost:9087/bfd1d723-0656-4b34-96a4-ce736c018eb3",
      "type": "PUT"
    }
  }
}

I can't see any problem here. And both RestClient and Webclient fail with the same exception.
Funny enough, when I let spring provide a ObjectMapper I can map to the class without problem. So there seems to be something fishy with the objectmapper that is used by default in webclient and restclient.

I am not providing my own objectmapper, as far as I can see to the context. And the problem is not present in 2023.0.1.

The error is triggered in the Jackson2HalModule while doing result.add(jp.readValueAs(Link.class).withRel(relation)); (Line 607) while reading the fetchCustomCss link. I have no clue what may trigger that.

@kschlesselmann
Copy link
Author

@pcornelissen Is there something I could (easily) test on our side to provide more information? Sadly I had no time to dig deeper but it's blocking our update as well :-(

@pcornelissen
Copy link

pcornelissen commented Jul 9, 2024

Well, I have exhausted the time I can invest for now. I have reverted to 2023.0.1 and the problem is gone.

As a workaround, you can try to fetch the result as string and use the objectmapper yourself, but I ran into the problem that the objectmapper was not configured for hateoas (although I used the builder) and this lead to the fact that the links of the entity were not parsed. But that may be a viable workaround with a proper object mapper.

It seems the problem is purely on the clientside. i have compared the input that came over the wire and it looked the same no matter if the sending service was on 2023.0.2 or .1

If you find a solution, please report here. I get anxious when I am too far behind with updates ;-)

@odrotbohm
Copy link
Member

I'm afraid that without a stack trace, some information about which Spring Cloud components are involved or anything that allows us to reproduce the issue, it's difficult to even remotely diagnose what the problem could be. Can anyone provide that information?

@pcornelissen
Copy link

pcornelissen commented Jul 9, 2024

Well, a stacktrace is easy:

2024-07-09T11:57:17.430+02:00 ERROR 34758 --- [Tenant] [mcat-handler-78] o.a.c.c.C.[.[.[/].[dispatcherServlet]    : Servlet.service() for servlet [dispatcherServlet] in context with path [] threw exception [Request processing failed: org.springframework.web.client.RestClientException: Error while extracting response for type [org.springframework.hateoas.EntityModel<MYPACKAGES_ANONYMIZED.BlanketContract>] and content type [application/hal+json]] with root cause

com.fasterxml.jackson.databind.exc.MismatchedInputException: Trailing token (of type FIELD_NAME) found after value (bound as `org.springframework.hateoas.Link`): not allowed as per `DeserializationFeature.FAIL_ON_TRAILING_TOKENS`
 at [Source: REDACTED (`StreamReadFeature.INCLUDE_SOURCE_IN_LOCATION` disabled); line: 1, column: 743] (through reference chain: org.springframework.hateoas.EntityModel["_links"])
	at com.fasterxml.jackson.databind.exc.MismatchedInputException.from(MismatchedInputException.java:63) ~[jackson-databind-2.17.1.jar:2.17.1]
	at com.fasterxml.jackson.databind.DeserializationContext.reportTrailingTokens(DeserializationContext.java:1840) ~[jackson-databind-2.17.1.jar:2.17.1]
	at com.fasterxml.jackson.databind.ObjectMapper._verifyNoTrailingTokens(ObjectMapper.java:5013) ~[jackson-databind-2.17.1.jar:2.17.1]
	at com.fasterxml.jackson.databind.ObjectMapper._readValue(ObjectMapper.java:4886) ~[jackson-databind-2.17.1.jar:2.17.1]
	at com.fasterxml.jackson.databind.ObjectMapper.readValue(ObjectMapper.java:3035) ~[jackson-databind-2.17.1.jar:2.17.1]
	at com.fasterxml.jackson.core.JsonParser.readValueAs(JsonParser.java:2453) ~[jackson-core-2.17.1.jar:2.17.1]
	at org.springframework.hateoas.mediatype.hal.Jackson2HalModule$HalLinkListDeserializer.deserialize(Jackson2HalModule.java:607) ~[spring-hateoas-2.3.0.jar:2.3.0]
	at org.springframework.hateoas.mediatype.hal.Jackson2HalModule$HalLinkListDeserializer.deserialize(Jackson2HalModule.java:554) ~[spring-hateoas-2.3.0.jar:2.3.0]
	at com.fasterxml.jackson.databind.deser.impl.FieldProperty.deserializeAndSet(FieldProperty.java:138) ~[jackson-databind-2.17.1.jar:2.17.1]
	at com.fasterxml.jackson.databind.deser.BeanDeserializer.deserializeWithUnwrapped(BeanDeserializer.java:740) ~[jackson-databind-2.17.1.jar:2.17.1]
	at com.fasterxml.jackson.databind.deser.BeanDeserializer.deserializeFromObject(BeanDeserializer.java:343) ~[jackson-databind-2.17.1.jar:2.17.1]
	at com.fasterxml.jackson.databind.deser.BeanDeserializer.deserialize(BeanDeserializer.java:185) ~[jackson-databind-2.17.1.jar:2.17.1]
	at com.fasterxml.jackson.databind.deser.DefaultDeserializationContext.readRootValue(DefaultDeserializationContext.java:342) ~[jackson-databind-2.17.1.jar:2.17.1]
	at com.fasterxml.jackson.databind.ObjectReader._bindAndClose(ObjectReader.java:2125) ~[jackson-databind-2.17.1.jar:2.17.1]
	at com.fasterxml.jackson.databind.ObjectReader.readValue(ObjectReader.java:1501) ~[jackson-databind-2.17.1.jar:2.17.1]
	at org.springframework.http.converter.json.AbstractJackson2HttpMessageConverter.readJavaType(AbstractJackson2HttpMessageConverter.java:395) ~[spring-web-6.1.8.jar:6.1.8]
	at org.springframework.http.converter.json.AbstractJackson2HttpMessageConverter.read(AbstractJackson2HttpMessageConverter.java:354) ~[spring-web-6.1.8.jar:6.1.8]
	at org.springframework.web.client.DefaultRestClient.readWithMessageConverters(DefaultRestClient.java:213) ~[spring-web-6.1.8.jar:6.1.8]
	at org.springframework.web.client.DefaultRestClient$DefaultResponseSpec.readBody(DefaultRestClient.java:688) ~[spring-web-6.1.8.jar:6.1.8]
	at org.springframework.web.client.DefaultRestClient$DefaultResponseSpec.body(DefaultRestClient.java:642) ~[spring-web-6.1.8.jar:6.1.8]
	at MYPACKAGES_ANONYMIZED.ContractsServiceRestRepository.fetchBlanketContract(ContractsServiceRestRepository.java:72) ~[classes/:na]
	at MYPACKAGES_ANONYMIZED.PublicInfoRestController.fetchPublicInfo(PublicInfoRestController.java:32) ~[classes/:na]
	at java.base/jdk.internal.reflect.DirectMethodHandleAccessor.invoke(DirectMethodHandleAccessor.java:103) ~[na:na]
	at java.base/java.lang.reflect.Method.invoke(Method.java:580) ~[na:na]
	at org.springframework.web.method.support.InvocableHandlerMethod.doInvoke(InvocableHandlerMethod.java:255) ~[spring-web-6.1.8.jar:6.1.8]
	at org.springframework.web.method.support.InvocableHandlerMethod.invokeForRequest(InvocableHandlerMethod.java:188) ~[spring-web-6.1.8.jar:6.1.8]
	at org.springframework.web.servlet.mvc.method.annotation.ServletInvocableHandlerMethod.invokeAndHandle(ServletInvocableHandlerMethod.java:118) ~[spring-webmvc-6.1.8.jar:6.1.8]
	at org.springframework.web.servlet.mvc.method.annotation.RequestMappingHandlerAdapter.invokeHandlerMethod(RequestMappingHandlerAdapter.java:926) ~[spring-webmvc-6.1.8.jar:6.1.8]
	at org.springframework.web.servlet.mvc.method.annotation.RequestMappingHandlerAdapter.handleInternal(RequestMappingHandlerAdapter.java:831) ~[spring-webmvc-6.1.8.jar:6.1.8]
	at org.springframework.web.servlet.mvc.method.AbstractHandlerMethodAdapter.handle(AbstractHandlerMethodAdapter.java:87) ~[spring-webmvc-6.1.8.jar:6.1.8]
	at org.springframework.web.servlet.DispatcherServlet.doDispatch(DispatcherServlet.java:1089) ~[spring-webmvc-6.1.8.jar:6.1.8]
	at org.springframework.web.servlet.DispatcherServlet.doService(DispatcherServlet.java:979) ~[spring-webmvc-6.1.8.jar:6.1.8]
	at org.springframework.web.servlet.FrameworkServlet.processRequest(FrameworkServlet.java:1014) ~[spring-webmvc-6.1.8.jar:6.1.8]
	at org.springframework.web.servlet.FrameworkServlet.doGet(FrameworkServlet.java:903) ~[spring-webmvc-6.1.8.jar:6.1.8]
	at jakarta.servlet.http.HttpServlet.service(HttpServlet.java:564) ~[tomcat-embed-core-10.1.24.jar:6.0]
	at org.springframework.web.servlet.FrameworkServlet.service(FrameworkServlet.java:885) ~[spring-webmvc-6.1.8.jar:6.1.8]
	at jakarta.servlet.http.HttpServlet.service(HttpServlet.java:658) ~[tomcat-embed-core-10.1.24.jar:6.0]
	at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:195) ~[tomcat-embed-core-10.1.24.jar:10.1.24]
	at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:140) ~[tomcat-embed-core-10.1.24.jar:10.1.24]
	at org.apache.tomcat.websocket.server.WsFilter.doFilter(WsFilter.java:51) ~[tomcat-embed-websocket-10.1.24.jar:10.1.24]
	at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:164) ~[tomcat-embed-core-10.1.24.jar:10.1.24]
	at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:140) ~[tomcat-embed-core-10.1.24.jar:10.1.24]
	at org.springframework.web.filter.ForwardedHeaderFilter.doFilterInternal(ForwardedHeaderFilter.java:173) ~[spring-web-6.1.8.jar:6.1.8]
	at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:116) ~[spring-web-6.1.8.jar:6.1.8]
	at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:164) ~[tomcat-embed-core-10.1.24.jar:10.1.24]
	at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:140) ~[tomcat-embed-core-10.1.24.jar:10.1.24]
	at org.springframework.web.filter.CompositeFilter$VirtualFilterChain.doFilter(CompositeFilter.java:108) ~[spring-web-6.1.8.jar:6.1.8]
	at org.springframework.security.web.FilterChainProxy.lambda$doFilterInternal$3(FilterChainProxy.java:231) ~[spring-security-web-6.3.0.jar:6.3.0]
	at org.springframework.security.web.ObservationFilterChainDecorator$FilterObservation$SimpleFilterObservation.lambda$wrap$1(ObservationFilterChainDecorator.java:479) ~[spring-security-web-6.3.0.jar:6.3.0]
	at org.springframework.security.web.ObservationFilterChainDecorator$AroundFilterObservation$SimpleAroundFilterObservation.lambda$wrap$1(ObservationFilterChainDecorator.java:340) ~[spring-security-web-6.3.0.jar:6.3.0]
	at org.springframework.security.web.ObservationFilterChainDecorator.lambda$wrapSecured$0(ObservationFilterChainDecorator.java:82) ~[spring-security-web-6.3.0.jar:6.3.0]
	at org.springframework.security.web.ObservationFilterChainDecorator$VirtualFilterChain.doFilter(ObservationFilterChainDecorator.java:128) ~[spring-security-web-6.3.0.jar:6.3.0]
	at org.springframework.security.web.access.intercept.AuthorizationFilter.doFilter(AuthorizationFilter.java:100) ~[spring-security-web-6.3.0.jar:6.3.0]
	at org.springframework.security.web.ObservationFilterChainDecorator$ObservationFilter.wrapFilter(ObservationFilterChainDecorator.java:240) ~[spring-security-web-6.3.0.jar:6.3.0]
	at org.springframework.security.web.ObservationFilterChainDecorator$ObservationFilter.doFilter(ObservationFilterChainDecorator.java:227) ~[spring-security-web-6.3.0.jar:6.3.0]
	at org.springframework.security.web.ObservationFilterChainDecorator$VirtualFilterChain.doFilter(ObservationFilterChainDecorator.java:137) ~[spring-security-web-6.3.0.jar:6.3.0]
	at org.springframework.security.web.access.ExceptionTranslationFilter.doFilter(ExceptionTranslationFilter.java:126) ~[spring-security-web-6.3.0.jar:6.3.0]
	at org.springframework.security.web.access.ExceptionTranslationFilter.doFilter(ExceptionTranslationFilter.java:120) ~[spring-security-web-6.3.0.jar:6.3.0]
	at org.springframework.security.web.ObservationFilterChainDecorator$ObservationFilter.wrapFilter(ObservationFilterChainDecorator.java:240) ~[spring-security-web-6.3.0.jar:6.3.0]
	at org.springframework.security.web.ObservationFilterChainDecorator$ObservationFilter.doFilter(ObservationFilterChainDecorator.java:227) ~[spring-security-web-6.3.0.jar:6.3.0]
	at org.springframework.security.web.ObservationFilterChainDecorator$VirtualFilterChain.doFilter(ObservationFilterChainDecorator.java:137) ~[spring-security-web-6.3.0.jar:6.3.0]
	at org.springframework.security.web.authentication.AnonymousAuthenticationFilter.doFilter(AnonymousAuthenticationFilter.java:100) ~[spring-security-web-6.3.0.jar:6.3.0]
	at org.springframework.security.web.ObservationFilterChainDecorator$ObservationFilter.wrapFilter(ObservationFilterChainDecorator.java:240) ~[spring-security-web-6.3.0.jar:6.3.0]
	at org.springframework.security.web.ObservationFilterChainDecorator$ObservationFilter.doFilter(ObservationFilterChainDecorator.java:227) ~[spring-security-web-6.3.0.jar:6.3.0]
	at org.springframework.security.web.ObservationFilterChainDecorator$VirtualFilterChain.doFilter(ObservationFilterChainDecorator.java:137) ~[spring-security-web-6.3.0.jar:6.3.0]
	at org.springframework.security.web.servletapi.SecurityContextHolderAwareRequestFilter.doFilter(SecurityContextHolderAwareRequestFilter.java:179) ~[spring-security-web-6.3.0.jar:6.3.0]
	at org.springframework.security.web.ObservationFilterChainDecorator$ObservationFilter.wrapFilter(ObservationFilterChainDecorator.java:240) ~[spring-security-web-6.3.0.jar:6.3.0]
	at org.springframework.security.web.ObservationFilterChainDecorator$ObservationFilter.doFilter(ObservationFilterChainDecorator.java:227) ~[spring-security-web-6.3.0.jar:6.3.0]
	at org.springframework.security.web.ObservationFilterChainDecorator$VirtualFilterChain.doFilter(ObservationFilterChainDecorator.java:137) ~[spring-security-web-6.3.0.jar:6.3.0]
	at org.springframework.security.web.savedrequest.RequestCacheAwareFilter.doFilter(RequestCacheAwareFilter.java:63) ~[spring-security-web-6.3.0.jar:6.3.0]
	at org.springframework.security.web.ObservationFilterChainDecorator$ObservationFilter.wrapFilter(ObservationFilterChainDecorator.java:240) ~[spring-security-web-6.3.0.jar:6.3.0]
	at org.springframework.security.web.ObservationFilterChainDecorator$ObservationFilter.doFilter(ObservationFilterChainDecorator.java:227) ~[spring-security-web-6.3.0.jar:6.3.0]
	at org.springframework.security.web.ObservationFilterChainDecorator$VirtualFilterChain.doFilter(ObservationFilterChainDecorator.java:137) ~[spring-security-web-6.3.0.jar:6.3.0]
	at org.springframework.security.oauth2.server.resource.web.authentication.BearerTokenAuthenticationFilter.doFilterInternal(BearerTokenAuthenticationFilter.java:145) ~[spring-security-oauth2-resource-server-6.3.0.jar:6.3.0]
	at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:116) ~[spring-web-6.1.8.jar:6.1.8]
	at org.springframework.security.web.ObservationFilterChainDecorator$ObservationFilter.wrapFilter(ObservationFilterChainDecorator.java:240) ~[spring-security-web-6.3.0.jar:6.3.0]
	at org.springframework.security.web.ObservationFilterChainDecorator$ObservationFilter.doFilter(ObservationFilterChainDecorator.java:227) ~[spring-security-web-6.3.0.jar:6.3.0]
	at org.springframework.security.web.ObservationFilterChainDecorator$VirtualFilterChain.doFilter(ObservationFilterChainDecorator.java:137) ~[spring-security-web-6.3.0.jar:6.3.0]
	at org.springframework.security.web.authentication.logout.LogoutFilter.doFilter(LogoutFilter.java:107) ~[spring-security-web-6.3.0.jar:6.3.0]
	at org.springframework.security.web.authentication.logout.LogoutFilter.doFilter(LogoutFilter.java:93) ~[spring-security-web-6.3.0.jar:6.3.0]
	at org.springframework.security.web.ObservationFilterChainDecorator$ObservationFilter.wrapFilter(ObservationFilterChainDecorator.java:240) ~[spring-security-web-6.3.0.jar:6.3.0]
	at org.springframework.security.web.ObservationFilterChainDecorator$ObservationFilter.doFilter(ObservationFilterChainDecorator.java:227) ~[spring-security-web-6.3.0.jar:6.3.0]
	at org.springframework.security.web.ObservationFilterChainDecorator$VirtualFilterChain.doFilter(ObservationFilterChainDecorator.java:137) ~[spring-security-web-6.3.0.jar:6.3.0]
	at org.springframework.web.filter.CorsFilter.doFilterInternal(CorsFilter.java:91) ~[spring-web-6.1.8.jar:6.1.8]
	at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:116) ~[spring-web-6.1.8.jar:6.1.8]
	at org.springframework.security.web.ObservationFilterChainDecorator$ObservationFilter.wrapFilter(ObservationFilterChainDecorator.java:240) ~[spring-security-web-6.3.0.jar:6.3.0]
	at org.springframework.security.web.ObservationFilterChainDecorator$ObservationFilter.doFilter(ObservationFilterChainDecorator.java:227) ~[spring-security-web-6.3.0.jar:6.3.0]
	at org.springframework.security.web.ObservationFilterChainDecorator$VirtualFilterChain.doFilter(ObservationFilterChainDecorator.java:137) ~[spring-security-web-6.3.0.jar:6.3.0]
	at org.springframework.security.web.header.HeaderWriterFilter.doHeadersAfter(HeaderWriterFilter.java:90) ~[spring-security-web-6.3.0.jar:6.3.0]
	at org.springframework.security.web.header.HeaderWriterFilter.doFilterInternal(HeaderWriterFilter.java:75) ~[spring-security-web-6.3.0.jar:6.3.0]
	at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:116) ~[spring-web-6.1.8.jar:6.1.8]
	at org.springframework.security.web.ObservationFilterChainDecorator$ObservationFilter.wrapFilter(ObservationFilterChainDecorator.java:240) ~[spring-security-web-6.3.0.jar:6.3.0]
	at org.springframework.security.web.ObservationFilterChainDecorator$ObservationFilter.doFilter(ObservationFilterChainDecorator.java:227) ~[spring-security-web-6.3.0.jar:6.3.0]
	at org.springframework.security.web.ObservationFilterChainDecorator$VirtualFilterChain.doFilter(ObservationFilterChainDecorator.java:137) ~[spring-security-web-6.3.0.jar:6.3.0]
	at org.springframework.security.web.context.SecurityContextHolderFilter.doFilter(SecurityContextHolderFilter.java:82) ~[spring-security-web-6.3.0.jar:6.3.0]
	at org.springframework.security.web.context.SecurityContextHolderFilter.doFilter(SecurityContextHolderFilter.java:69) ~[spring-security-web-6.3.0.jar:6.3.0]
	at org.springframework.security.web.ObservationFilterChainDecorator$ObservationFilter.wrapFilter(ObservationFilterChainDecorator.java:240) ~[spring-security-web-6.3.0.jar:6.3.0]
	at org.springframework.security.web.ObservationFilterChainDecorator$ObservationFilter.doFilter(ObservationFilterChainDecorator.java:227) ~[spring-security-web-6.3.0.jar:6.3.0]
	at org.springframework.security.web.ObservationFilterChainDecorator$VirtualFilterChain.doFilter(ObservationFilterChainDecorator.java:137) ~[spring-security-web-6.3.0.jar:6.3.0]
	at org.springframework.security.web.context.request.async.WebAsyncManagerIntegrationFilter.doFilterInternal(WebAsyncManagerIntegrationFilter.java:62) ~[spring-security-web-6.3.0.jar:6.3.0]
	at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:116) ~[spring-web-6.1.8.jar:6.1.8]
	at org.springframework.security.web.ObservationFilterChainDecorator$ObservationFilter.wrapFilter(ObservationFilterChainDecorator.java:240) ~[spring-security-web-6.3.0.jar:6.3.0]
	at org.springframework.security.web.ObservationFilterChainDecorator$ObservationFilter.doFilter(ObservationFilterChainDecorator.java:227) ~[spring-security-web-6.3.0.jar:6.3.0]
	at org.springframework.security.web.ObservationFilterChainDecorator$VirtualFilterChain.doFilter(ObservationFilterChainDecorator.java:137) ~[spring-security-web-6.3.0.jar:6.3.0]
	at org.springframework.security.web.session.DisableEncodeUrlFilter.doFilterInternal(DisableEncodeUrlFilter.java:42) ~[spring-security-web-6.3.0.jar:6.3.0]
	at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:116) ~[spring-web-6.1.8.jar:6.1.8]
	at org.springframework.security.web.ObservationFilterChainDecorator$ObservationFilter.wrapFilter(ObservationFilterChainDecorator.java:240) ~[spring-security-web-6.3.0.jar:6.3.0]
	at org.springframework.security.web.ObservationFilterChainDecorator$AroundFilterObservation$SimpleAroundFilterObservation.lambda$wrap$0(ObservationFilterChainDecorator.java:323) ~[spring-security-web-6.3.0.jar:6.3.0]
	at org.springframework.security.web.ObservationFilterChainDecorator$ObservationFilter.doFilter(ObservationFilterChainDecorator.java:224) ~[spring-security-web-6.3.0.jar:6.3.0]
	at org.springframework.security.web.ObservationFilterChainDecorator$VirtualFilterChain.doFilter(ObservationFilterChainDecorator.java:137) ~[spring-security-web-6.3.0.jar:6.3.0]
	at org.springframework.security.web.FilterChainProxy.doFilterInternal(FilterChainProxy.java:233) ~[spring-security-web-6.3.0.jar:6.3.0]
	at org.springframework.security.web.FilterChainProxy.doFilter(FilterChainProxy.java:191) ~[spring-security-web-6.3.0.jar:6.3.0]
	at org.springframework.web.filter.CompositeFilter$VirtualFilterChain.doFilter(CompositeFilter.java:113) ~[spring-web-6.1.8.jar:6.1.8]
	at org.springframework.web.servlet.handler.HandlerMappingIntrospector.lambda$createCacheFilter$3(HandlerMappingIntrospector.java:195) ~[spring-webmvc-6.1.8.jar:6.1.8]
	at org.springframework.web.filter.CompositeFilter$VirtualFilterChain.doFilter(CompositeFilter.java:113) ~[spring-web-6.1.8.jar:6.1.8]
	at org.springframework.web.filter.CompositeFilter.doFilter(CompositeFilter.java:74) ~[spring-web-6.1.8.jar:6.1.8]
	at org.springframework.security.config.annotation.web.configuration.WebMvcSecurityConfiguration$CompositeFilterChainProxy.doFilter(WebMvcSecurityConfiguration.java:230) ~[spring-security-config-6.3.0.jar:6.3.0]
	at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:352) ~[spring-web-6.1.8.jar:6.1.8]
	at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:268) ~[spring-web-6.1.8.jar:6.1.8]
	at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:164) ~[tomcat-embed-core-10.1.24.jar:10.1.24]
	at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:140) ~[tomcat-embed-core-10.1.24.jar:10.1.24]
	at org.springframework.web.filter.RequestContextFilter.doFilterInternal(RequestContextFilter.java:100) ~[spring-web-6.1.8.jar:6.1.8]
	at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:116) ~[spring-web-6.1.8.jar:6.1.8]
	at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:164) ~[tomcat-embed-core-10.1.24.jar:10.1.24]
	at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:140) ~[tomcat-embed-core-10.1.24.jar:10.1.24]
	at org.springframework.web.filter.FormContentFilter.doFilterInternal(FormContentFilter.java:93) ~[spring-web-6.1.8.jar:6.1.8]
	at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:116) ~[spring-web-6.1.8.jar:6.1.8]
	at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:164) ~[tomcat-embed-core-10.1.24.jar:10.1.24]
	at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:140) ~[tomcat-embed-core-10.1.24.jar:10.1.24]
	at org.springframework.web.filter.ServerHttpObservationFilter.doFilterInternal(ServerHttpObservationFilter.java:109) ~[spring-web-6.1.8.jar:6.1.8]
	at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:116) ~[spring-web-6.1.8.jar:6.1.8]
	at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:164) ~[tomcat-embed-core-10.1.24.jar:10.1.24]
	at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:140) ~[tomcat-embed-core-10.1.24.jar:10.1.24]
	at org.springframework.web.filter.CharacterEncodingFilter.doFilterInternal(CharacterEncodingFilter.java:201) ~[spring-web-6.1.8.jar:6.1.8]
	at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:116) ~[spring-web-6.1.8.jar:6.1.8]
	at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:164) ~[tomcat-embed-core-10.1.24.jar:10.1.24]
	at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:140) ~[tomcat-embed-core-10.1.24.jar:10.1.24]
	at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:167) ~[tomcat-embed-core-10.1.24.jar:10.1.24]
	at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:90) ~[tomcat-embed-core-10.1.24.jar:10.1.24]
	at org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:482) ~[tomcat-embed-core-10.1.24.jar:10.1.24]
	at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:115) ~[tomcat-embed-core-10.1.24.jar:10.1.24]
	at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:93) ~[tomcat-embed-core-10.1.24.jar:10.1.24]
	at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:74) ~[tomcat-embed-core-10.1.24.jar:10.1.24]
	at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:344) ~[tomcat-embed-core-10.1.24.jar:10.1.24]
	at org.apache.coyote.http11.Http11Processor.service(Http11Processor.java:389) ~[tomcat-embed-core-10.1.24.jar:10.1.24]
	at org.apache.coyote.AbstractProcessorLight.process(AbstractProcessorLight.java:63) ~[tomcat-embed-core-10.1.24.jar:10.1.24]
	at org.apache.coyote.AbstractProtocol$ConnectionHandler.process(AbstractProtocol.java:896) ~[tomcat-embed-core-10.1.24.jar:10.1.24]
	at org.apache.tomcat.util.net.NioEndpoint$SocketProcessor.doRun(NioEndpoint.java:1741) ~[tomcat-embed-core-10.1.24.jar:10.1.24]
	at org.apache.tomcat.util.net.SocketProcessorBase.run(SocketProcessorBase.java:52) ~[tomcat-embed-core-10.1.24.jar:10.1.24]
	at java.base/java.lang.VirtualThread.run(VirtualThread.java:309) ~[na:na]

The only thing I have to change to trigger this is switch from 2023.0.1 to 2023.0.2 in the service that triggers this error. No other change is necessary, the "source service" can remain on 2023.0.1 or be at 2023.0.2 as well, it doesn't matter.

@odrotbohm
Copy link
Member

What's weird is that you seem to be able to pinpoint the issue on the Spring Cloud update, but there's no Spring Cloud involved in the stacktrace. Can you elaborate what you mean with "source service"? On which side does the update cause an issue? Are you saying the system serving the response, alters the response as an effect of the upgrade so that the client breaks? Or is it the client that starts failing to read the response?

@dbernsau
Copy link

dbernsau commented Jul 9, 2024

Same issue here after the switch to 2023.0.2. We run integrations tests using TestRestTemplate.excahange(...).

Parsing the response into proper Java object fails now:

org.springframework.web.client.RestClientException: Error while extracting response for type [class com.unboundid.scim2.common.messages.ErrorResponse] and content type [application/json]
        at org.springframework.web.client.HttpMessageConverterExtractor.extractData(HttpMessageConverterExtractor.java:119)
        at org.springframework.web.client.RestTemplate$ResponseEntityResponseExtractor.extractData(RestTemplate.java:1159)
        at org.springframework.web.client.RestTemplate$ResponseEntityResponseExtractor.extractData(RestTemplate.java:1142)
        at org.springframework.web.client.RestTemplate.doExecute(RestTemplate.java:892)
        at org.springframework.web.client.RestTemplate.execute(RestTemplate.java:790)
        at org.springframework.web.client.RestTemplate.exchange(RestTemplate.java:672)
        at org.springframework.boot.test.web.client.TestRestTemplate.exchange(TestRestTemplate.java:711)
		...
Caused by: org.springframework.http.converter.HttpMessageNotReadableException: JSON parse error: Trailing token (of type FIELD_NAME) found after value (bound as `java.lang.Integer`): not allowed as per `DeserializationFeature.FAIL_ON_TRAILING_TOKENS`
        at org.springframework.http.converter.json.AbstractJackson2HttpMessageConverter.readJavaType(AbstractJackson2HttpMessageConverter.java:406)
        at org.springframework.http.converter.json.AbstractJackson2HttpMessageConverter.read(AbstractJackson2HttpMessageConverter.java:354)
        at org.springframework.web.client.HttpMessageConverterExtractor.extractData(HttpMessageConverterExtractor.java:104)
        ... 10 more
Caused by: com.fasterxml.jackson.databind.exc.MismatchedInputException: Trailing token (of type FIELD_NAME) found after value (bound as `java.lang.Integer`): not allowed as per `DeserializationFeature.FAIL_ON_TRAILING_TOKENS`
 at [Source: (org.springframework.util.StreamUtils$NonClosingInputStream); line: 1, column: 103] (through reference chain: com.unboundid.scim2.common.messages.ErrorResponse["status"])
        at com.fasterxml.jackson.databind.exc.MismatchedInputException.from(MismatchedInputException.java:63)
        at com.fasterxml.jackson.databind.DeserializationContext.reportTrailingTokens(DeserializationContext.java:1825)
        at com.fasterxml.jackson.databind.ObjectMapper._verifyNoTrailingTokens(ObjectMapper.java:4933)
        at com.fasterxml.jackson.databind.ObjectMapper._readValue(ObjectMapper.java:4806)
        at com.fasterxml.jackson.databind.ObjectMapper.readValue(ObjectMapper.java:2974)
        at com.fasterxml.jackson.core.JsonParser.readValueAs(JsonParser.java:2363)
        at com.unboundid.scim2.common.utils.StatusDeserializer.deserialize(StatusDeserializer.java:41)
        at com.unboundid.scim2.common.utils.StatusDeserializer.deserialize(StatusDeserializer.java:31)
        at com.fasterxml.jackson.databind.deser.SettableBeanProperty.deserialize(SettableBeanProperty.java:545)
        at com.fasterxml.jackson.databind.deser.BeanDeserializer._deserializeWithErrorWrapping(BeanDeserializer.java:570)
        at com.fasterxml.jackson.databind.deser.BeanDeserializer._deserializeUsingPropertyBased(BeanDeserializer.java:439)
        at com.fasterxml.jackson.databind.deser.BeanDeserializerBase.deserializeFromObjectUsingNonDefault(BeanDeserializerBase.java:1419)
        at com.fasterxml.jackson.databind.deser.BeanDeserializer.deserializeFromObject(BeanDeserializer.java:352)
        at com.fasterxml.jackson.databind.deser.BeanDeserializer.deserialize(BeanDeserializer.java:185)
        at com.fasterxml.jackson.databind.deser.DefaultDeserializationContext.readRootValue(DefaultDeserializationContext.java:323)
        at com.fasterxml.jackson.databind.ObjectReader._bindAndClose(ObjectReader.java:2105)
        at com.fasterxml.jackson.databind.ObjectReader.readValue(ObjectReader.java:1481)
        at org.springframework.http.converter.json.AbstractJackson2HttpMessageConverter.readJavaType(AbstractJackson2HttpMessageConverter.java:395)
        ... 12 more

I debugged into the whole thing and the key is within ObjectMapper._readValue(...) there is the part

        if (cfg.isEnabled(DeserializationFeature.FAIL_ON_TRAILING_TOKENS)) {
            this._verifyNoTrailingTokens(p, ctxt, valueType);
        }

and with the old version the config did not have FAIL_ON_TRAILING_TOKENS enabled. And since the JSON we parse has trailing tokens (which is OK and matches the target object structure) the whole thing fails now.

@pcornelissen
Copy link

Ah ok. Wenn I meant that this is communication between two microservices. The client is the one that throws the exception and the server spits out the same result, no matter which spring cloud version is in use. As far as I have seen it, the response from the server is identical with both versions.

The client on the other hand fails when reading the second link in the result with the latest spring cloud. (I stepped through and this is what it looked like. The bytestreams make it a bit hard to follow the exact position in the document)

@odrotbohm
Copy link
Member

Is there any chance you capture some of the payload and provide a minimal reproducer trying to statically read that? From what you describe the (reduced) content put into a file and an interaction with an ObjectMapper configured by a full app bootstrap should be able to reproduce the problem, shouldn't it?

@pcornelissen
Copy link

The payload from my project is in the comments before

@dbernsau
Copy link

dbernsau commented Jul 9, 2024

Here is my payload:

{"schemas":["urn:ietf:params:scim:api:messages:2.0:Error"],"id":null,"externalId":null,"status":"404","meta":null,"scimType":null,"detail":"User with uuid 03031ece-d670-4f23-adc8-48247aa078f2 doesn't exist"}

which will be deserialized into com.unboundid.scim2.common.messages.ErrorResponse which defines a specific deserializer (String => Integer) for the status field. And exactly that goes wrong.

 @JsonSerialize(
        using = StatusSerializer.class
    )
    @JsonDeserialize(
        using = StatusDeserializer.class
    )
    private final int status;

@dbernsau
Copy link

Spring Cloud 2023.0.3 fixed the issue for me

@Alien2150
Copy link

Spring Cloud 2023.0.3 fixed the issue for me

Confirmed. It was fixed by spring cloud: spring-cloud/spring-cloud-function#1148

@pcornelissen
Copy link

Confirmed from me as well, the problem is gone

@odrotbohm
Copy link
Member

Looks like this can be closed then. Thanks everyone for their input!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

5 participants