Skip to content

Commit cf17bfa

Browse files
Fix DateTime timezone support: namespace::autoclean stub + keyword autoquoting (#343)
* Fix DateTime timezone support: namespace::autoclean stub + keyword autoquoting Phase 11 completion: 1. namespace::autoclean stub (src/main/perl/lib/namespace/autoclean.pm) - Provides interface but skips cleanup entirely - Allows imported functions (Try::Tiny try/catch) to remain available - Fixes Undefined subroutine DateTime::TimeZone::catch error 2. Parser fix for keyword autoquoting (ListParser.java) - Extended AUTOQUOTABLE_KEYWORDS set to include: if, unless, while, until, for, foreach (in addition to and, or, xor, when) - Keywords followed by => are now treated as bareword hash keys - Fixes Expected token } but got until parser error Generated with [Devin](https://cli.devin.ai/docs) Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com> * Implement delete %hash{keys} and delete %array[indices] (kv-slice delete) Added missing key-value slice delete operations: 1. RuntimeHash.java: Added deleteKeyValueSlice() method - Returns alternating keys and values of deleted elements - Fixes delete %hash{key1, key2} returning keys and values 2. RuntimeArray.java: Added getKeyValueSlice() and deleteKeyValueSlice() - getKeyValueSlice() returns alternating indices and values - deleteKeyValueSlice() deletes and returns index/value pairs - Fixes delete %array[idx1, idx2] returning indices and values 3. Dereference.java: Extended handleHashElementOperator and handleArrayElementOperator - Added handling for % sigil with delete operation - Properly routes to the new *KeyValueSlice methods This improves op/delete.t from 13 to ~50 passing tests. Generated with [Devin](https://cli.devin.ai/docs) Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com> * Add slice delete operations to interpreter backend Implement three new opcodes for the bytecode interpreter: - ARRAY_SLICE_DELETE (390): delete @array[indices] - HASH_KV_SLICE_DELETE (391): delete %hash{keys} (returns key-value pairs) - ARRAY_KV_SLICE_DELETE (392): delete %array[indices] (returns index-value pairs) These operations were already supported in the JVM backend via RuntimeArray.deleteSlice() and RuntimeHash/RuntimeArray.deleteKeyValueSlice(), but the interpreter was failing with "Array exists/delete requires simple array variable" errors. Generated with [Devin](https://cli.devin.ai/docs) Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com> * Phase 12: DateTime Java XS fallback and fixes - Fix refaddr to return identity hash of underlying object, not wrapper This fixes Specio enum validation and DateTime truncate/today - Add POSIX math functions: floor, ceil, fmod, fabs, pow, trig, etc. DateTime uses POSIX::floor and POSIX::fmod - Update cpan_client.md with Phase 12 completion details DateTime now uses Java XS implementation ($DateTime::IsPurePerl = 0) with 98.5% of DateTime tests passing. Generated with [Devin](https://cli.devin.ai/docs) Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com> * Document DateTime test issues for Phase 13+ Added comprehensive list of remaining DateTime test failures (45/3292): - Overload stringification StackOverflowError (HIGH PRIORITY) - Leap second handling issues (19 failures) - End-of-month arithmetic bugs (21 failures) - Missing test dependencies - DateTime::Locale data file installation - IPC::Open3 read-only modification bug - Dist::CheckConflicts method resolution - Encode::PERLQQ undefined - Number::Overloaded integration Generated with [Devin](https://cli.devin.ai/docs) Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com> * Fix: single-variable string interpolation now properly stringifies overloaded objects When a double-quoted string contains only a single interpolated variable like "$obj", the parser was optimizing it to just return the variable directly. This broke overloaded objects because: 1. The 'eq' overload handler does: return "$a" eq "$b" 2. PerlOnJava was treating "$a" as just $a (no stringification) 3. This caused the eq overload to call itself infinitely (StackOverflowError) The fix ensures that single non-string segments in string interpolation are wrapped in a join() operation, which forces proper stringification. This does NOT apply in regex context (isRegex=true) because regex patterns should use the 'qr' overload, not stringify. Fixes DateTime test failures: - t/20infinite.t: stringified infinity comparisons now work - t/31formatter.t: formatter stringification now works Generated with [Devin](https://cli.devin.ai/docs) Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com> * Document Phase 13: overload stringification fix - t/20infinite.t: All 104 tests now pass - t/31formatter.t: All 11 tests now pass - DateTime test results improved: 3260/3302 (98.7%), down from 45 to 42 failures Generated with [Devin](https://cli.devin.ai/docs) Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com> * Add XSLoader VERSION and bootstrap_inherit for CPAN compatibility - Set $XSLoader::VERSION to 0.32 (compatible with CPAN version) - Add bootstrap_inherit() stub that delegates to load() This fixes 'XSLoader does not define $XSLoader::VERSION' errors when CPAN runs XSLoader's own test suite during dependency resolution. Generated with [Devin](https://cli.devin.ai/docs) Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com> --------- Co-authored-by: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>
1 parent 9363ed0 commit cf17bfa

15 files changed

Lines changed: 854 additions & 77 deletions

File tree

dev/design/cpan_client.md

Lines changed: 202 additions & 55 deletions
Original file line numberDiff line numberDiff line change
@@ -11,7 +11,7 @@ This document tracks CPAN client support for PerlOnJava. The `jcpan` command pro
1111
- `jcpan -f install Module::Name` - Force install (skip tests)
1212
- `jcpan -t Module::Name` - Test a module
1313
- Interactive CPAN shell via `jcpan`
14-
- **DateTime** - Core functionality working (new, datetime, add, subtract, formatting)
14+
- **DateTime** - Full functionality including timezone support
1515

1616
**Known Limitations:**
1717
- XS modules require manual porting (see `.cognition/skills/port-cpan-module/`)
@@ -44,6 +44,7 @@ This document tracks CPAN client support for PerlOnJava. The `jcpan` command pro
4444
| Safe | Stub using `eval` | CPAN metadata is trusted |
4545
| ExtUtils::MakeMaker | Custom | Installs directly, no `make` needed |
4646
| Module::Build::Base | Stub | Disables fork pipes |
47+
| namespace::autoclean | Stub (no-op) | Skips cleanup to allow imported functions |
4748

4849
### Not Implemented
4950

@@ -75,80 +76,226 @@ This document tracks CPAN client support for PerlOnJava. The `jcpan` command pro
7576
| 7 | Errno & Regex | `$!` dualvar, literal `{}` braces in regex |
7677
| 8 | User Experience | `jcpan` wrapper script |
7778
| 9 | Polish | YAML version update, Module::Build partial support |
79+
| 11 | DateTime Support | namespace::autoclean stub, keyword autoquoting parser fix |
80+
| 12 | DateTime Java XS | refaddr fix, POSIX math functions |
81+
| 13 | Overload Stringification | Single-variable interpolation now forces stringify |
7882

7983
---
8084

81-
## Phase 11: DateTime Support (Active)
85+
## Phase 11: DateTime Support (Completed 2026-03-20)
8286

8387
### Problem Statement
8488

85-
DateTime installation via jcpan completes but the module had issues loading due to its complex dependency chain involving namespace::autoclean.
89+
DateTime installation via jcpan completed but the module had issues loading due to its complex dependency chain involving namespace::autoclean.
8690

87-
### Current Status (2026-03-20)
91+
### Solution
8892

89-
**DateTime core functionality is working:**
90-
```bash
91-
./jperl -MDateTime -e '
92-
my $dt = DateTime->new(year => 2024, month => 3, day => 15, hour => 10);
93-
print $dt->datetime, "\n"; # 2024-03-15T10:00:00
94-
$dt->add(days => 5);
95-
print $dt->datetime, "\n"; # 2024-03-20T10:00:00
96-
'
97-
```
93+
Two fixes were required:
94+
95+
1. **namespace::autoclean stub** - Created `src/main/perl/lib/namespace/autoclean.pm` that provides the interface but skips cleanup. This allows imported functions (like Try::Tiny's `try`/`catch`) to remain available.
96+
97+
2. **Parser fix for keyword autoquoting** - Extended `ListParser.java` to handle keywords like `until`, `while`, `for`, `if`, `unless`, `foreach` as bareword hash keys when followed by `=>`. Previously these keywords would incorrectly terminate list parsing.
98+
99+
### DateTime Now Working
98100

99-
**Timezone support has remaining issues:**
100101
```bash
101-
# Fails because namespace::autoclean cleans imported Try::Tiny functions
102102
./jperl -MDateTime -e '
103-
my $dt = DateTime->new(year => 2024, time_zone => "America/New_York");
103+
my $dt = DateTime->new(
104+
year => 2024,
105+
month => 3,
106+
day => 15,
107+
hour => 14,
108+
minute => 30,
109+
time_zone => "America/New_York"
110+
);
111+
print $dt->datetime, "\n"; # 2024-03-15T14:30:00
112+
$dt->add(days => 5, hours => 2);
113+
print $dt->datetime, "\n"; # 2024-03-20T16:30:00
104114
'
105-
# Error: Undefined subroutine &DateTime::TimeZone::catch
106115
```
107116

108117
### Issues Fixed in Phase 11
109118

110-
| Issue | Fix | Commit |
111-
|-------|-----|--------|
119+
| Issue | Fix | File |
120+
|-------|-----|------|
112121
| `${ $stash{NAME} }` dereference | Fixed symbol table access | |
113122
| GLOBREFERENCE scalar dereference | `$$globref` now returns the glob itself | |
114-
| map/grep @_ access | Blocks now access outer subroutine's @_ | 67da75215 |
115-
| B::Hooks::EndOfScope NPE | Null check for fileName in beginFileLoad/endFileLoad | 5dc05ca6d |
123+
| map/grep @_ access | Blocks now access outer subroutine's @_ | |
124+
| B::Hooks::EndOfScope NPE | Null check for fileName | |
125+
| namespace::autoclean cleanup | Stub that skips cleanup | `src/main/perl/lib/namespace/autoclean.pm` |
126+
| Keywords as hash keys | Extended autoquoting to more keywords | `ListParser.java` |
116127

117-
### namespace::autoclean Issue (NOT YET FIXED)
128+
---
118129

119-
**Problem:** When DateTime::PP installs methods via glob assignment:
120-
```perl
121-
*{ 'DateTime::' . $sub } = __PACKAGE__->can($sub);
122-
```
123-
namespace::autoclean cleans them because `Sub::Util::subname` returns the *original* package name (`DateTime::PP::_ymd2rd`) instead of the *installed* name (`DateTime::_ymd2rd`).
124-
125-
**Why we can't just update packageName/subName on glob assignment:**
126-
- This breaks `next::method` which relies on subname to find the next method in MRO
127-
- Perl's behavior: glob assignment does NOT change subname (verified with system Perl)
128-
- The mro/next_edgecases.t tests specifically verify this behavior
129-
130-
**Possible solutions:**
131-
1. DateTime::PP should use `Sub::Name::subname()` to explicitly name the subs before installing
132-
2. Provide a namespace::autoclean stub that doesn't clean up methods
133-
3. Track installation location separately from intrinsic subname (complex)
134-
135-
### Remaining Issues
136-
137-
1. **namespace::autoclean cleans DateTime::PP methods** - Methods installed via glob assignment are cleaned
138-
- Root cause: subname returns original package, not installed location
139-
- This is correct Perl behavior; DateTime::PP should use Sub::Name
140-
141-
2. **namespace::autoclean + Exporter imports** - Imported functions are also cleaned
142-
- Affects: DateTime::TimeZone (Try::Tiny), and likely other modules
143-
144-
3. **XS fallback** - DateTime uses pure Perl mode; XS bridge not yet implemented
145-
- Low priority since PP mode works
146-
147-
### Next Steps
148-
149-
1. **Check if DateTime::PP uses Sub::Name** - May already be fixed in CPAN version
150-
2. **Consider namespace::autoclean stub** - Could skip cleanup entirely
151-
3. **Document workarounds** - Manual patches to use Sub::Name or exclude methods
130+
## Phase 12: DateTime with Java XS Fallback (Completed 2026-03-20)
131+
132+
### Objective
133+
134+
Test and verify DateTime uses the Java XS fallback mechanism instead of pure Perl fallback, providing better performance via native Java date/time operations.
135+
136+
### Status: COMPLETED
137+
138+
**DateTime now uses Java XS implementation** (`$DateTime::IsPurePerl = 0`)
139+
140+
### Fixes Applied
141+
142+
| Issue | Fix | File |
143+
|-------|-----|------|
144+
| Missing POSIX math functions | Added `floor`, `ceil`, `fmod`, `fabs`, `pow`, trig functions | `POSIX.pm` |
145+
| `refaddr` returning inconsistent values | Fixed to return identity hash of underlying referenced object | `ScalarUtil.java` |
146+
| Specio enum validation failing | Fixed by `refaddr` fix - env var names now stable | - |
147+
| DateTime truncate/today failing | Fixed by Specio fix | - |
148+
149+
### Technical Details
150+
151+
1. **Java XS Loading**: `XSLoader::load("DateTime")` successfully loads `DateTime.java` which provides:
152+
- `_rd2ymd` - Rata Die to year/month/day conversion using `java.time.JulianFields`
153+
- `_ymd2rd` - Year/month/day to Rata Die conversion
154+
- `_is_leap_year` - Using `java.time.Year.isLeap()`
155+
- `_time_as_seconds`, `_seconds_as_components` - Time arithmetic
156+
- `_normalize_tai_seconds`, `_normalize_leap_seconds` - TAI/UTC handling
157+
- `_day_length`, `_day_has_leap_second`, `_accumulated_leap_seconds` - Leap second support
158+
159+
2. **refaddr Bug**: The `Scalar::Util::refaddr` function was returning `System.identityHashCode(scalar)` where `scalar` is the RuntimeScalar wrapper, causing different values each time when called via a method. Fixed to return identity hash code of the underlying `scalar.value` for reference types.
160+
161+
3. **POSIX Math Functions**: Added complete set of POSIX math functions:
162+
- `floor`, `ceil` - Rounding functions
163+
- `fmod` - Floating-point modulo
164+
- `fabs`, `pow` - Absolute value and power
165+
- `asin`, `acos`, `atan`, `tan` - Trigonometric functions
166+
- `sinh`, `cosh`, `tanh` - Hyperbolic functions
167+
- `log10`, `ldexp`, `frexp`, `modf` - Logarithmic and mantissa functions
168+
169+
### Test Results
170+
171+
DateTime test suite: **3247/3292 subtests passed** (98.6%), **45 failures**
172+
173+
---
174+
175+
## Phase 13: Overload Stringification Fix (Completed 2026-03-20)
176+
177+
### Problem Statement
178+
179+
DateTime tests (t/20infinite.t, t/31formatter.t) were failing with `StackOverflowError` when comparing stringified DateTime objects using `eq`.
180+
181+
### Root Cause
182+
183+
When a double-quoted string contained only a single interpolated variable like `"$obj"`, the parser was optimizing it to just return the variable directly, without forcing stringification. This caused:
184+
185+
1. The `eq` overload handler does: `return "$a" eq "$b"`
186+
2. PerlOnJava was treating `"$a"` as just `$a` (no stringification)
187+
3. This caused the `eq` overload to call itself infinitely → StackOverflowError
188+
189+
### Solution
190+
191+
Fixed `StringDoubleQuoted.createJoinNode()` to ensure that single non-string segments in string interpolation are wrapped in a `join()` operation, which forces proper stringification.
192+
193+
The fix does NOT apply in regex context (`isRegex=true`) because regex patterns should use the `qr` overload, not stringify.
194+
195+
### Test Results After Fix
196+
197+
DateTime test suite: **3260/3302 subtests passed** (98.7%), **42 failures**
198+
199+
- **t/20infinite.t**: All 104 tests now pass (was failing on infinite stringification)
200+
- **t/31formatter.t**: All 11 tests now pass (was failing on formatter stringification)
201+
202+
### Files Changed
203+
204+
- `src/main/java/org/perlonjava/frontend/parser/StringDoubleQuoted.java` - Fixed single-variable string interpolation
205+
206+
---
207+
208+
### Known Issues To Be Fixed (Phase 14+)
209+
210+
The following issues remain from `./jcpan -t DateTime`:
211+
212+
#### 1. ~~Overload Stringification - StackOverflowError~~ **FIXED in Phase 13**
213+
214+
#### 2. Leap Second Handling (MEDIUM PRIORITY)
215+
216+
**Symptom**: DateTime fails to properly handle leap seconds (second = 60).
217+
218+
**Affected Tests**: t/19leap-second.t (12 failures), t/32leap-second2.t (7 failures)
219+
220+
**Examples**:
221+
- `Invalid second value (60)` - DateTime doesn't accept second=60
222+
- `delta_seconds` calculations off by 1 for leap second boundaries
223+
- `utc_rd_secs` should be 86400 for leap seconds, returns 0
224+
225+
**Root Cause**: Java XS `_seconds_as_components` and `_normalize_leap_seconds` may not fully match Perl's leap second semantics.
226+
227+
#### 3. End-of-Month Arithmetic (MEDIUM PRIORITY)
228+
229+
**Symptom**: Date arithmetic involving month ends produces incorrect results.
230+
231+
**Affected Tests**: t/06add.t (2), t/10subtract.t (4), t/11duration.t (4), t/27delta.t (4), t/38local-subtract.t (7)
232+
233+
**Examples**:
234+
- `2000-02-29 + 1 year` should give `2001-03-01`, got `2001-02-28`
235+
- `2003-12-31 - 1 month` should give `2003-11-30`, got `2003-12-01`
236+
- `delta_months` returns negative values incorrectly
237+
238+
**Root Cause**: The `end_of_month` handling mode ('preserve', 'limit') not fully implemented in Java XS or pure Perl fallback.
239+
240+
#### 4. Floating Time Comparison (LOW PRIORITY)
241+
242+
**Symptom**: Comparison with floating time zones returns 0 instead of -1.
243+
244+
**Affected Test**: t/07compare.t line 168
245+
246+
#### 5. Missing Test Dependencies
247+
248+
These cause test files to skip or fail to run:
249+
250+
| Module | Tests Affected |
251+
|--------|----------------|
252+
| `Test::Warnings` | t/29overload.t, t/46warnings.t |
253+
| `Test::Without::Module` | t/49-without-sub-util.t |
254+
| `Term::ANSIColor` | t/zzz-check-breaks.t |
255+
| `Storable` (locale data) | t/23storable.t |
256+
257+
#### 6. DateTime::Locale Data Files
258+
259+
**Symptom**: `Failed to find shared file 'de.pl' for dist 'DateTime-Locale'`
260+
261+
**Affected Tests**: t/13strftime.t, t/14locale.t, t/23storable.t, t/41cldr-format.t
262+
263+
**Root Cause**: DateTime::Locale locale data files (*.pl) not installed by jcpan. These are runtime data files, not Perl modules.
264+
265+
#### 7. IPC::Open3 Read-Only Modification
266+
267+
**Symptom**: `open3: Modification of a read-only value attempted`
268+
269+
**Affected Test**: Dist::CheckConflicts t/00-compile.t
270+
271+
**Root Cause**: Bug in IPCOpen3.java line 162 when handling read-only arguments.
272+
273+
#### 8. Dist::CheckConflicts Method Resolution
274+
275+
**Symptom**: `Can't locate object method "conflicts" via package`
276+
277+
**Affected Tests**: Multiple Dist::CheckConflicts tests
278+
279+
**Root Cause**: Dist::CheckConflicts uses complex method injection via `Sub::Exporter` that may not work correctly in PerlOnJava.
280+
281+
#### 9. Encode::PERLQQ Undefined
282+
283+
**Symptom**: `Undefined subroutine &Encode::PERLQQ called`
284+
285+
**Affected**: CPAN::Meta loading in t/00-report-prereqs.t
286+
287+
#### 10. Number::Overloaded Integration
288+
289+
**Symptom**: `Can't use string ("Number::Overloaded::(0+") as a symbol ref`
290+
291+
**Affected Test**: t/04epoch.t
292+
293+
**Root Cause**: overload.pm line 111 cannot resolve overloaded numification operator.
294+
295+
### Files Changed
296+
297+
- `src/main/perl/lib/POSIX.pm` - Added math functions
298+
- `src/main/java/org/perlonjava/runtime/perlmodule/ScalarUtil.java` - Fixed refaddr
152299

153300
---
154301

src/main/java/org/perlonjava/backend/bytecode/BytecodeInterpreter.java

Lines changed: 16 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1755,6 +1755,22 @@ public static RuntimeList execute(InterpretedCode code, RuntimeArray args, int c
17551755
registers[rd] = array.get(index);
17561756
}
17571757

1758+
// =================================================================
1759+
// KV-SLICE DELETE OPERATIONS (390-392)
1760+
// =================================================================
1761+
1762+
case Opcodes.ARRAY_SLICE_DELETE -> {
1763+
pc = SlowOpcodeHandler.executeArraySliceDelete(bytecode, pc, registers);
1764+
}
1765+
1766+
case Opcodes.HASH_KV_SLICE_DELETE -> {
1767+
pc = SlowOpcodeHandler.executeHashKVSliceDelete(bytecode, pc, registers);
1768+
}
1769+
1770+
case Opcodes.ARRAY_KV_SLICE_DELETE -> {
1771+
pc = SlowOpcodeHandler.executeArrayKVSliceDelete(bytecode, pc, registers);
1772+
}
1773+
17581774
default -> {
17591775
int opcodeInt = opcode;
17601776
throw new RuntimeException(

0 commit comments

Comments
 (0)