[BUG] DateTime Format without trailing zeros in CosmosDB Trigger

[noopliez]

Describe the bug
When you store a DateTime String with trailing zeros in milliseconds in CosmosDB and try to read it via CosmosDBTrigger, you'll get an JSON parsing exception, because apparently Cosmos destroys you specified DateFormat.

2023-03-31T14:12:27.8000000+02:00 will be cut down to 2023-03-31T14:12:27.8+02:00.

We're currently not able to fix it or to get a workaround, because it seems to be on the Cosmos side of things.

Exception or Stack Trace
Failed to deserialize java.time.OffsetDateTime: (java.time.format.DateTimeParseException) Text '2023-03-31T16:12:27.8+02:00' could not be parsed at index 20\n at [...]

To Reproduce

• Store a DateTime String in Cosmos with trailing zeros in the milliseconds representation.
• Let a CosmosDBTrigger run in a Function

Code Snippet

Expected behavior
We expect that we read what we write.

Screenshots

Setup (please complete the following information):

• OS: Windows 10
• IDE: IntelliJ
• Library/Libraries: azure-sdk-bom-1.2.14, azure-functions-java-library-3.0.0, azure-cosmos-4.46.0, azure-core-serializer-json-jackson-1.4.1
• Java version: 17
• App Server/Environment:Azure Function, Cosmos DB
• Frameworks:

If you suspect a dependency version mismatch (e.g. you see NoClassDefFoundError, NoSuchMethodError or similar), please check out Troubleshoot dependency version conflict article first. If it doesn't provide solution for the problem, please provide:

• verbose dependency tree (mvn dependency:tree -Dverbose)
• exception message, full stack trace, and any available logs

Additional context
Add any other context about the problem here.

Information Checklist
Kindly make sure that you have added all the following information above and checkoff the required fields otherwise we will treat the issuer as an incomplete report

• Bug Description Added
• Repro Steps Added
• Setup information Added

[kushagraThapar]

@noopliez - apologies for the delayed response, I will take a look at this issue soon.

[kushagraThapar]

@noopliez - I am investigating this, but unable to repro, can you provide the trigger definition that you are using?
Regarding DateTimeFormat, the default format for OffsetDateTime is without trailing zeros. Can you provide the DateTimeFormat you are using to store the date time?

[kushagraThapar]

FYI, on further research, this looks like an issue with the jackson module which is still open and has not been treated as a bug.
FasterXML/jackson-modules-java8#76

However, I still think it is worth debugging to see if there is something that Cosmos SDK can help in this case. Please provide the above information @noopliez

[github-actions]

Hi @noopliez. Thank you for opening this issue and giving us the opportunity to assist. To help our team better understand your issue and the details of your scenario please provide a response to the question asked above or the information requested above. This will help us more accurately address your issue.

[noopliez]

Hi @kushagraThapar, sorry for my late response. So, the trigger we're using is the CosmosDBTrigger for an Azure Function. We are storing the timestamp with the yyyy-MM-dd'T'HH:mm:ssSSSXXX DateFormat inside CosmosDB and we're deserializing our models with that format, too. That's when it crashes sometimes, because the trailing zeros are gone :)

Our current solution is to implement a custom deserializer.

To confirm your research, I also think it's an issue with jackson and maybe not solvable on your side. Maybe it would suffice to add this known issue to the docs, when it comes to storing dates and timestamps as Strings in Cosmos.

[kushagraThapar]

@noopliez - can you please double check the pattern you have provided, because I don't think this pattern even fits the date that is shared in the example:

Also, on further testing, I noticed, its not the Cosmos DB Java SDK or jackson which is skipping the trailing zeros, rather the way java DateTimeFormatter works.

I have put together an example here which you can run locally to see the following output - when writing to cosmos db and reading from it back: https://github.com/kushagraThapar/cosmos-java-sdk-testing/blob/master/src/main/java/com/example/common/CosmosTriggerExample.java

INFO com.example.common.CosmosTriggerExample - Test item is : TestItem{id='d7c9f445-dfc6-4b51-867f-8007f53188ca', mypk='13a8edeb-3fae-4fc3-98d6-a254d238d0e4', offsetDateTime=2023-03-31T14:12:27.800+02:00, offsetDateTimeString='2023-03-31T14:12:27.8+02:00', zonedDateTime=2023-11-08T12:28:20.185-08:00[America/Los_Angeles], zonedDateTimeString='2023-11-08T12:28:20.185-08:00[America/Los_Angeles]', dateTimeStringWithTrailingZeros='2023-03-31T14:12:27.8+02:00'}
[main] INFO com.example.common.CosmosTriggerExample - Read item is : TestItem{id='d7c9f445-dfc6-4b51-867f-8007f53188ca', mypk='13a8edeb-3fae-4fc3-98d6-a254d238d0e4', offsetDateTime=2023-03-31T12:12:27.800Z, offsetDateTimeString='2023-03-31T14:12:27.8+02:00', zonedDateTime=2023-11-08T20:28:20.185Z[UTC], zonedDateTimeString='2023-11-08T12:28:20.185-08:00[America/Los_Angeles]', dateTimeStringWithTrailingZeros='2023-03-31T14:12:27.8+02:00'}

If you notice the offsetDateTime object, before even writing to cosmos db, it is already skipping the trailing zeros from nanoseconds.

Please let me know if I am misunderstanding this issue, thanks!

[noopliez]

Hi @kushagraThapar, you're right about the format, but it doesn't really matter, because it's about the trailing zeros and that can come up with any format :)

Without testing your code locally, the output looks very interesting. If it's an issue with the DateTimeFormatter, I think our approach with the custom deserializer would be the way to go. Right now I don't have the code with me (not on my working machine), but I will paste an example here next week.

Thank you for your research @kushagraThapar !