Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 2 additions & 0 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -32,6 +32,8 @@ perl5/
# Ignore the t/ directory (Perl test suite copied from perl5/t/)
# These should be synced from perl5 repo, not committed here
t/
# But allow bundled module test directories
!src/test/resources/module/*/t/

# Ignore perl5_t/ directory (module tests from perl5/lib/*.t)
# These are synced from perl5 repo via sync.pl, not committed here
Expand Down
1 change: 1 addition & 0 deletions AGENTS.md
Original file line number Diff line number Diff line change
Expand Up @@ -82,6 +82,7 @@ PerlOnJava does **not** implement the following Perl features:
|---------|--------------|
| `make` | Build + run all unit tests (use before committing) |
| `make dev` | Build only, skip tests (for quick iteration during debugging) |
| `make test-bundled-modules` | Run bundled CPAN module tests (XML::Parser, etc.) |

- For interpreter changes, test with both backends:
```bash
Expand Down
11 changes: 10 additions & 1 deletion Makefile
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
.PHONY: all clean test test-unit test-interpreter test-exiftool test-all test-gradle test-gradle-unit test-gradle-all test-gradle-parallel test-maven-parallel build run wrapper check-java-gradle dev ci sbom sbom-java sbom-perl sbom-clean check-links
.PHONY: all clean test test-unit test-interpreter test-bundled-modules test-exiftool test-all test-gradle test-gradle-unit test-gradle-all test-gradle-parallel test-maven-parallel build run wrapper check-java-gradle dev ci sbom sbom-java sbom-perl sbom-clean check-links

all: build

Expand Down Expand Up @@ -64,6 +64,15 @@ test-interpreter:
@echo "Running unit tests with bytecode interpreter..."
JPERL_INTERPRETER=1 perl dev/tools/perl_test_runner.pl --jobs 8 --timeout 60 --output test_interpreter_results.json src/test/resources/unit

# Bundled CPAN module tests (XML::Parser, etc.)
# Tests live under src/test/resources/module/{ModuleName}/t/
test-bundled-modules: check-java-gradle
ifeq ($(OS),Windows_NT)
gradlew.bat testModule --rerun-tasks
else
./gradlew testModule --rerun-tasks
endif

# Image::ExifTool test suite (Image-ExifTool-13.44/t/ directory)
test-exiftool:
@echo "Running Image::ExifTool tests..."
Expand Down
2 changes: 1 addition & 1 deletion README.md
Original file line number Diff line number Diff line change
Expand Up @@ -11,7 +11,7 @@ PerlOnJava compiles Perl to JVM bytecode. One jar file runs on Linux, macOS, and

- **Single jar distribution** — no installation, no dependencies beyond Java
- **Full toolchain** — `jperl`, `jperldoc`, `jcpan`, `jprove`
- **150+ modules included** — [DBI](docs/guides/database-access.md), HTTP::Tiny, JSON, YAML, Text::CSV, and more
- **150+ modules included** — [DBI](docs/guides/database-access.md), HTTP::Tiny, JSON, XML::Parser, YAML, Text::CSV, and more
- **Install more with jcpan** — [pure-Perl CPAN modules](docs/guides/using-cpan-modules.md) work out of the box
- **JDBC database access** — [PostgreSQL, MySQL, SQLite, Oracle](docs/guides/database-access.md) via standard JDBC drivers
- **Embed in Java apps** — [JSR-223 ScriptEngine](docs/guides/java-integration.md) integration
Expand Down
16 changes: 16 additions & 0 deletions build.gradle
Original file line number Diff line number Diff line change
Expand Up @@ -264,6 +264,22 @@ tasks.register('testAll', Test) {
shouldRunAfter testUnit
}

// Bundled module tests (XML::Parser, etc.)
// Tests live under src/test/resources/module/{ModuleName}/t/
tasks.register('testModule', Test) {
description = 'Runs bundled CPAN module tests (e.g. XML::Parser)'
group = 'verification'

testClassesDirs = sourceSets.test.output.classesDirs
classpath = sourceSets.test.runtimeClasspath

useJUnitPlatform {
includeTags 'module'
}

shouldRunAfter testUnit
}

// Shadow JAR configuration for creating standalone executable
shadowJar {
archiveClassifier.set('')
Expand Down
48 changes: 48 additions & 0 deletions dev/design/xml_parser_xs.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,48 @@
# XML::Parser Java XS Implementation Plan

## Overview

XML::Parser is implemented as a Java XS module (`XMLParserExpat.java`) backed by JDK's built-in SAX parser (`javax.xml.parsers.SAXParser`). This replaces the native C/XS expat bindings with a pure-Java equivalent, dispatching SAX events to the same Perl callback interface.

## Architecture

- **Java XS**: `src/main/java/org/perlonjava/runtime/perlmodule/XMLParserExpat.java`
- **Perl shim**: `src/main/perl/lib/XML/Parser/Expat.pm` (modified from upstream)
- **Backend**: JDK SAX (Apache Xerces built into the JDK)

### Key Design Decisions

1. **SAX vs DOM**: SAX chosen for streaming event model that maps naturally to expat's callback API
2. **Namespace dualvars**: Namespace-qualified names use `DualVar(numericIndex, stringName)` matching expat's behavior where `int($name)` gives namespace index
3. **BYTE_STRING encoding**: ParseString uses `ISO_8859_1` for `BYTE_STRING` input to avoid double-encoding raw UTF-8 bytes
4. **SystemId un-resolution**: SAX resolves relative systemIds to absolute `file:///` URIs; `unresolveSysId()` strips the base to recover the original relative paths

## Test Status

**Current: 45/45 test files pass (100%), 434/434 subtests pass (100%)**

All XML::Parser 2.56 tests pass (excluding 2 `Devel::CheckLib` C compiler detection tests that are irrelevant to PerlOnJava).

### Running Tests

Tests are stored in `src/test/resources/module/XML-Parser/` and run via:

```bash
make test-bundled-modules
```

This uses JUnit 5 (`ModuleTestExecutionTest.java`) with `@Tag("module")` to discover and execute all `.t` files under `module/*/t/`. The test runner `chdir`s to the module directory so relative paths resolve correctly.

## Completed Phases

### Phase 5: Final fixes for 47/47 (2026-04-07)

**Tests fixed**: decl.t (44/46→46/46), foreign_dtd.t (0/5→5/5), checklib_findcc.t (2/3→3/3), checklib_tmpdir.t (1/3→3/3)

1. **NOTATION type format fix**: Off-by-one bug in `attributeDecl()` — `substring(8)` → `substring(9)` to strip the space SAX adds after `NOTATION`
2. **XMLDecl for text declarations**: Added `fireTextDeclHandler()` in `resolveEntity()` to fire the XMLDecl callback for text declarations in external parsed entities (with `version=undef`), before `convertEncoding()` rewrites the encoding
3. **UseForeignDTD**: When `UseForeignDTD => 1` and no DOCTYPE exists, calls ExternEnt handler with `(parser, base, undef, undef)`, reads DTD content, injects `<!DOCTYPE _fdt SYSTEM "__perlonjava_foreign_dtd__">` after the XML declaration, and resolves the synthetic system ID in `resolveEntity()`
4. **"undefined entity" error message**: SAX reports `"was referenced, but not declared"` for undefined entities; mapped to expat's `"undefined entity"` format in `formatError()`
5. **Devel::CheckLib**: Replaced 9-line stub with real upstream source from XML-Parser-2.56 tarball

Files: XMLParserExpat.java
1 change: 1 addition & 0 deletions dev/modules/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -124,3 +124,4 @@ PERL_PARAMS_UTIL_PP=1 ./jcpan -t Class::Load
- [dbix_class.md](dbix_class.md) - DBIx::Class support
- [log4perl-compatibility.md](log4perl-compatibility.md) - Log::Log4perl
- [term_readkey.md](term_readkey.md) - Term::ReadKey
- [xml_parser.md](xml_parser.md) - XML::Parser (Java XS via JDK SAX)
Loading
Loading