TimeZoneRoundingTests#testRoundingRandom fails reproducibly

[s1monw]

 2> REPRODUCE WITH: gradle :core:test -Dtests.seed=17AC5B1BB751F423 -Dtests.class=org.elasticsearch.common.rounding.TimeZoneRoundingTests -Dtests.method="testRoundingRandom" -Dtests.security.manager=true -Dtests.locale=ko -Dtests.timezone=Indian/Comoro
FAILURE 0.69s | TimeZoneRoundingTests.testRoundingRandom <<< FAILURES!
   > Throwable #1: java.lang.AssertionError: dateBetween should round down to roundedDate
   > Expected: Expected: 2024-10-27T00:00:00.000Z [1729987200000] 
   >      but:  was "2024-10-27T00:00:00.000-01:00 [1729990800000]"
   >    at __randomizedtesting.SeedInfo.seed([17AC5B1BB751F423:B8E2E3098394017E]:0)
   >    at org.hamcrest.MatcherAssert.assertThat(MatcherAssert.java:20)
   >    at org.elasticsearch.common.rounding.TimeZoneRoundingTests.assertInterval(TimeZoneRoundingTests.java:541)
   >    at org.elasticsearch.common.rounding.TimeZoneRoundingTests.testRoundingRandom(TimeZoneRoundingTests.java:195)
   >    at java.lang.Thread.run(Thread.java:745)
  2> NOTE: leaving temporary files on disk at: /Users/simon/projects/elasticsearch/core/build/testrun/test/J0/temp/org.elasticsearch.common.rounding.TimeZoneRoundingTests_17AC5B1BB751F423-001
  2> NOTE: test params are: codec=Asserting(Lucene62): {}, docValues:{}, maxPointsInLeafNode=386, maxMBSortInHeap=5.970189533938357, sim=ClassicSimilarity, locale=ko, timezone=Indian/Comoro
  2> NOTE: Mac OS X 10.11.6 x86_64/Oracle Corporation 1.8.0_66 (64-bit)/cpus=4,threads=1,free=434107792,total=514850816
  2> NOTE: All tests run in this JVM: [TimeZoneRoundingTests]
Completed [1/1] in 2.26s, 1 test, 1 failure <<< FAILURES!

[s1monw]

@cbuescher can you look

[cbuescher]

I couldn't reproduce this on master with the given seed anymore, but going back to some earlier commit I was able get the random parameters that made the test trip in this case. Problem in that test occurs with:

DateTimeUnit.DAY_OF_MONTH
DateTimeZone.forID("Atlantic/Azores")
date = 1729989444634l;

Still need to do some digging what goes wrong here.

[tlrx]

@cbuescher Just a side note but I updated JodaTime to v2.9.5 recently and they fixed some bugs like JodaOrg/joda-time#373. Maybe related to this issue.

[cbuescher]

I found the root cause for this. The test error reveals an interesting edge case with DST overlaps (happening usually on DST end, turning back the clock one hour) in combination with rounding to day intervals.

In this particular case (tz: "Atlantic/Azores", dates around the DST transition on e.g. 2000-10-29T01:00:00.000Z) we currently get the overlapping part during the DST transition as a separate rounding interval as illustrated in this table:

date	round(date)	nextRoundingValue(date)
2000-10-28T22:00:00.000Z	2000-10-28T00:00:00.000Z	2000-10-29T00:00:00.000Z
2000-10-28T23:00:00.000Z	2000-10-28T00:00:00.000Z	2000-10-29T00:00:00.000Z
2000-10-29T00:00:00.000Z	2000-10-29T00:00:00.000Z	2000-10-30T00:00:00.000-01:00
2000-10-29T00:00:00.000-01:00	2000-10-29T00:00:00.000-01:00	2000-10-30T00:00:00.000-01:00
2000-10-29T01:00:00.000-01:00	2000-10-29T00:00:00.000-01:00	2000-10-30T00:00:00.000-01:00
2000-10-29T02:00:00.000-01:00	2000-10-29T00:00:00.000-01:00	2000-10-30T00:00:00.000-01:00

According to https://www.timeanddate.com/time/change/portugal/horta?year=2000 the DST change happenes on Oct-29th at 1am local time, turning back the clock one hour (to offset -01:00). Currently the dates between "2000-10-29T00:00:00.000Z" and "2000-10-29T01:00:00.000Z" all round down to "2000-10-29T00:00:00.000Z" (before the transition) and the dates after but before next midnight round down to "2000-10-29T00:00:00.000-01:00". For a day rounding we would prefer a 25h hour bucket for "2000-10-29" to a one hour + another 24h bucket I guess. I think I have a fix that detects this kind of situation and merges the two intervals.