Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -89,3 +89,4 @@ Image-ExifTool-*
xxx/
*.jfr
report.txt
exiftool_results.json
9 changes: 8 additions & 1 deletion dev/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -13,6 +13,12 @@ This directory contains development resources, tools and experimental code for P
- Architecture decisions and planning documents
- Feature specifications and roadmaps

- **modules/** - CPAN module porting documentation
- Module compatibility guides (Moose, Moo, DateTime)
- XS fallback mechanisms and Java XS implementations
- jcpan client and MakeMaker documentation
- See [modules/README.md](modules/README.md) for module status

- **examples/** - Sample code and usage demonstrations

Mix of working examples and feature concepts for design exploration. Not all examples are guaranteed to run.
Expand Down Expand Up @@ -55,12 +61,13 @@ When adding new code or tools:
1. Place it in the appropriate category directory
2. Include clear documentation and usage examples
3. Add relevant tests where applicable
4. Reference related design documents from the `design/` directory
4. Reference related design documents from `design/` or `modules/`

## Development Guidelines

- Keep experimental code separate from production code
- Document design decisions in the `design/` directory
- Document module porting work in the `modules/` directory
- Use the `bench/` directory for performance testing
- Check `examples/` for implementation patterns
- Store reusable LLM prompts in `prompts/` directory
2 changes: 2 additions & 0 deletions dev/design/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -51,6 +51,8 @@ The design documents cover:
- **I/O & Signals:** File handles, signal handling, terminal control
- **Distribution:** Packaging, versioning, platform support

> **Note:** Module porting documentation (CPAN modules, XS fallback, Moose/Moo support) has been moved to [`dev/modules/`](../modules/README.md).

---

## Finding What You Need
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -307,5 +307,5 @@ mv.visitJumpInsn(Opcodes.GOTO, applyNoControlFlow);

## Related Documents

- `dev/design/cpan_client.md` - Main CPAN client documentation
- `dev/design/xsloader.md` - XSLoader implementation
- `cpan_client.md` - Main CPAN client documentation
- `xsloader.md` - XSLoader implementation
121 changes: 121 additions & 0 deletions dev/modules/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,121 @@
# Module Porting Documentation

This directory contains design documents and guides related to porting CPAN modules to PerlOnJava.

## Quick Reference

| Document | Description |
|----------|-------------|
| [moose_support.md](moose_support.md) | Path to Moose/Class::MOP support (blocked by B module) |
| [moo_support.md](moo_support.md) | Moo support status (96% working) |
| [xs_fallback.md](xs_fallback.md) | XS fallback mechanism for pure Perl modules |
| [xsloader.md](xsloader.md) | XSLoader architecture |
| [makemaker_perlonjava.md](makemaker_perlonjava.md) | ExtUtils::MakeMaker implementation |
| [cpan_client.md](cpan_client.md) | jcpan - CPAN client for PerlOnJava |

## Module Status Overview

### Working Modules

| Module | Status | Notes |
|--------|--------|-------|
| **Moo** | 96% tests pass | Recommended OO system |
| **DateTime** | Works | Java XS implementation |
| **JSON::PP** | Works | Built-in |
| **YAML** | Works | Pure Perl |
| **Try::Tiny** | Works | Pure Perl |
| **Test::More** | Works | Built-in |
| **DBI** | Works | Java implementation |

### Modules Requiring Environment Variables

| Module | Environment Variable | Notes |
|--------|---------------------|-------|
| Params::Util | `PERL_PARAMS_UTIL_PP=1` | Has PP fallback |
| Class::Load | Works after Params::Util | Has PP fallback |
| Package::Stash | Auto-fallback | Has PP fallback |

### Not Yet Working

| Module | Blocker | See |
|--------|---------|-----|
| Moose | B module subroutine names | [moose_support.md](moose_support.md) |
| Any XS-only module | No compiler | [xs_fallback.md](xs_fallback.md) |

## Key Concepts

### XS Fallback Pattern

Many CPAN modules with XS code also provide pure Perl fallbacks:

```perl
# Common pattern in modules
eval {
require XSLoader;
XSLoader::load(__PACKAGE__, $VERSION);
};
if ($@) {
require Module::PP; # Pure Perl fallback
}
```

PerlOnJava's XSLoader returns an error matching `/loadable object/` which these modules recognize.

### Java XS Implementations

For performance-critical modules, PerlOnJava can provide Java implementations:

- `DateTime` - Uses `java.time` APIs
- `JSON::XS` - Falls back to JSON::PP (or could use FASTJSON)
- `DBI` - Custom Java implementation with JDBC

See [xs_fallback.md](xs_fallback.md) for implementation details.

### Installing Modules

Use `jcpan` for module installation:

```bash
# Install a module
./jcpan install DateTime

# Test a module
./jcpan -t Moo

# With PP environment variables
PERL_PARAMS_UTIL_PP=1 ./jcpan -t Class::Load
```

## Adding Support for New Modules

1. **Check if module has XS**: Look for `.xs` files in the distribution
2. **Check for PP fallback**: Look for `*::PP` modules or fallback patterns
3. **Test installation**: `./jcpan -t ModuleName`
4. **Document blockers**: Create/update design doc if issues found
5. **Consider Java XS**: For critical modules, implement in Java

## Related Resources

- [AGENTS.md](../../AGENTS.md) - Project guidelines
- [docs/guides/module-porting.md](../../docs/guides/module-porting.md) - User-facing porting guide
- `.cognition/skills/port-cpan-module/` - AI skill for porting modules

## Document Index

### Core Infrastructure
- [xsloader.md](xsloader.md) - XSLoader implementation
- [dynaloader.md](dynaloader.md) - DynaLoader architecture
- [dynamic_loading.md](dynamic_loading.md) - Dynamic module loading
- [makemaker_perlonjava.md](makemaker_perlonjava.md) - MakeMaker for PerlOnJava
- [cpan_client.md](cpan_client.md) - jcpan CPAN client

### XS and Fallbacks
- [xs_fallback.md](xs_fallback.md) - XS fallback mechanism
- [pure-perl-exporter.md](pure-perl-exporter.md) - Pure Perl Exporter

### Specific Modules
- [moose_support.md](moose_support.md) - Moose support (in progress)
- [moo_support.md](moo_support.md) - Moo support (working)
- [JCPAN_DATETIME_FIXES.md](JCPAN_DATETIME_FIXES.md) - DateTime via jcpan
- [log4perl-compatibility.md](log4perl-compatibility.md) - Log::Log4perl
- [term_readkey.md](term_readkey.md) - Term::ReadKey
4 changes: 2 additions & 2 deletions dev/design/cpan_client.md → dev/modules/cpan_client.md
Original file line number Diff line number Diff line change
Expand Up @@ -409,7 +409,7 @@ The fix allows CPAN::Meta::YAML to properly parse MYMETA.yml files, enabling CPA

## Related Documents

- `dev/design/xsloader.md` - XSLoader/Java integration
- `dev/design/makemaker_perlonjava.md` - ExtUtils::MakeMaker implementation
- `xsloader.md` - XSLoader/Java integration
- `makemaker_perlonjava.md` - ExtUtils::MakeMaker implementation
- `.cognition/skills/port-cpan-module/` - Skill for porting CPAN modules
- `docs/guides/using-cpan-modules.md` - User documentation
File renamed without changes.
File renamed without changes.
Original file line number Diff line number Diff line change
Expand Up @@ -187,7 +187,7 @@ BEGIN failed--compilation aborted at -e line 1, near ""
- `src/main/java/org/perlonjava/backend/jvm/EmitSubroutine.java`
- `src/main/java/org/perlonjava/runtime/runtimetypes/ExceptionFormatter.java`

**Design Document:** `dev/design/caller_line_number_fix.md`
**Design Document:** `../design/caller_line_number_fix.md`

### Issue 2: Stack Trace Format (%T) - FIXED

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -335,6 +335,6 @@ jperl Makefile.PL

## Related Documents

- `dev/design/cpan_client.md` - CPAN client status
- `cpan_client.md` - CPAN client status
- `docs/guides/module-porting.md` - Module porting guide
- `.cognition/skills/port-cpan-module/` - Port CPAN module skill
16 changes: 8 additions & 8 deletions dev/design/moo_support.md → dev/modules/moo_support.md
Original file line number Diff line number Diff line change
Expand Up @@ -625,7 +625,7 @@ Moo tests run via `jcpan -t Moo`. Recent fixes (Phases 12-13) should improve pas
- Modified `setDebugInfoLineNumber()` to also call `saveSourceLocation()`
- This is called during emit when we have correct package context from the subroutine's symbol table
- The emit-time call overwrites parse-time entries with correct package
- See `dev/design/caller_package_context.md` for detailed analysis
- See `../design/caller_package_context.md` for detailed analysis
- No data structure changes, minimal 6-line fix

- [x] Phase 29: Fix caller() returning wrong line numbers (2026-03-17)
Expand Down Expand Up @@ -654,7 +654,7 @@ Moo tests run via `jcpan -t Moo`. Recent fixes (Phases 12-13) should improve pas
- **ExceptionFormatter.java fix**:
- Interpreter path now calls `ByteCodeSourceMapper.getPackageAtLocation()` for all frames
- Ensures consistent package reporting regardless of stack depth
- See `dev/design/unified_caller_stack.md` for full analysis
- See `../design/unified_caller_stack.md` for full analysis
- Result: Interpreter and JVM backends now report same package for same source location

- [x] Phase 37: Fix #line directive to update errorUtil.fileName during parsing (2026-03-17)
Expand Down Expand Up @@ -699,7 +699,7 @@ The remaining test failures require implementing core Perl features that are cur
#### Phase 31: DESTROY/Destructor Support (High Impact)
**Enables**: demolish tests (6 failures), proper object cleanup
**Status**: Analysis complete, implementation deferred
**Design doc**: `dev/design/object_lifecycle.md`
**Design doc**: `../design/object_lifecycle.md`

Perl's DESTROY relies on reference counting; Java uses GC. The challenge is detecting
when an object becomes unreachable while we can still access it to call DESTROY.
Expand All @@ -710,7 +710,7 @@ detailed analysis of implementation strategies, challenges, and test cases.
#### Phase 32: Weak Reference Emulation (High Impact)
**Enables**: accessor-weaken tests (20 failures), no-moo.t (5 failures)
**Status**: Analysis complete, implementation deferred
**Design doc**: `dev/design/object_lifecycle.md`
**Design doc**: `../design/object_lifecycle.md`

Perl's weak references are tied to reference counting, which Java doesn't have.

Expand Down Expand Up @@ -754,7 +754,7 @@ The interpreter path was using different package sources for inner vs outer fram
Fixed by adding `getPackageAtLocation()` to ByteCodeSourceMapper and using it in
ExceptionFormatter for all frames.

See `dev/design/unified_caller_stack.md` for detailed analysis.
See `../design/unified_caller_stack.md` for detailed analysis.

#### Phase 35: Mo strict.t - Make $^H Magical (Completed)
**Enables**: Mo t/strict.t (1 failure) → **FIXED**
Expand All @@ -768,7 +768,7 @@ that didn't communicate with the compiler's strict checking.
- On read: Returns current strict options from symbol table

**Future consideration**: Refactor to use `$^H` as single source of truth, eliminating
`strictOptionsStack`. See `dev/design/strict_hints_refactor.md` for analysis.
`strictOptionsStack`. See `../design/strict_hints_refactor.md` for analysis.

**Result**: Mo tests now 28/28 passing (was 27/28).

Expand Down Expand Up @@ -824,7 +824,7 @@ which are fundamentally limited by Java's GC model.

## Related Documents

- `dev/design/cpan_client.md` - jcpan implementation
- `dev/design/unified_caller_stack.md` - caller() package tracking analysis
- `cpan_client.md` - jcpan implementation
- `../design/unified_caller_stack.md` - caller() package tracking analysis
- `dev/import-perl5/README.md` - Module sync process
- `dev/import-perl5/config.yaml` - Module import configuration
Loading
Loading