BUG: Fix index.union failure at DST boundary #63017
Closed
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Description
Fixes #62915
When concatenating
DatetimeIndexobjects across DST transitions, the frequency preservation logic in_concat_same_typewas failing because it used naive addition that didn't account for timezone offset changes at DST boundaries.Problem
At line 2386 in
pandas/core/arrays/datetimelike.py, the code checked:When crossing a DST boundary (e.g., Europe/Helsinki on 2025-10-27):
pair[0][-1]=2025-10-26 00:00:00+03:00pair[0][-1] + Day()=2025-10-27 00:00:00+03:00(naive add, offset unchanged)pair[1][0]=2025-10-27 00:00:00+02:00(actual value after DST)These timestamps are not equal due to different UTC offsets, causing the assertion to fail even though they represent consecutive days.
Solution
For fixed (Tick) frequencies like Day, Hour, etc., the fix compares the underlying int64 values (UTC nanoseconds since epoch) instead of relying on timezone-aware arithmetic:
This correctly identifies consecutive timestamps regardless of DST transitions.
For non-fixed frequencies like MonthEnd or BusinessDay where
freq.nanosraises ValueError, the code falls back to the original comparison method:Testing
Added
test_union_dst_boundaryintest_setops.pythat reproduces the exact scenario from the issue report with Europe/Helsinki DST transition.Type of change