diff --git a/.gitignore b/.gitignore index 9cd0fb8bb..b2c7dfd1b 100644 --- a/.gitignore +++ b/.gitignore @@ -67,6 +67,8 @@ dev/prompts/CONTROL_FLOW_FIX_INSTRUCTIONS.md !LICENSE.md !MILESTONES.md !README.md +!QUICKSTART.md +!CONTRIBUTING.md test_*.log test_*.json diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md new file mode 100644 index 000000000..198df294c --- /dev/null +++ b/CONTRIBUTING.md @@ -0,0 +1,225 @@ +# Contributing to PerlOnJava + +We welcome contributions! This guide will help you get started. + +## Ways to Contribute + +- **Report bugs** - Open issues on GitHub +- **Fix bugs** - Submit pull requests +- **Add features** - Implement missing Perl features +- **Improve docs** - Fix typos, add examples, clarify explanations +- **Port modules** - Help port CPAN modules to PerlOnJava +- **Write tests** - Add test coverage + +## Quick Start for Contributors + +### 1. Set Up Development Environment + +```bash +# Clone repository +git clone https://github.com/fglock/PerlOnJava.git +cd PerlOnJava + +# Build and run tests +make # Build + fast unit tests +make dev # Force clean rebuild + +# Run comprehensive tests +make test-all +``` + +### 2. Development Workflow + +**Before making changes:** +1. Create a feature branch: `git checkout -b feature-name` +2. Read the [Architecture Guide](docs/reference/architecture.md) +3. Check [Roadmap](docs/about/roadmap.md) for planned features + +**While coding:** +1. Follow existing code style +2. Run `make` frequently to ensure tests pass +3. Add tests for new features in `src/test/resources/unit/` + +**Before committing:** +1. Ensure `make` passes (fast unit tests) +2. Run `make test-all` for comprehensive testing +3. Write clear commit messages + +### 3. Submit Pull Request + +1. Push your branch: `git push origin feature-name` +2. Open a PR on GitHub +3. Describe what your PR does and why +4. Link related issues + +**The #1 priority is keeping `make` working.** All unit tests must pass. + +## Code Organization + +### Source Code Structure + +``` +src/main/java/org/perlonjava/ +├── astnode/ # AST node representations +├── parser/ # Parser implementation +├── lexer/ # Tokenization +├── codegen/ # Bytecode generation and class creation +├── astvisitor/ # AST traversal (EmitterVisitor, PrinterVisitor) +├── runtime/ # Runtime implementations (RuntimeScalar, RuntimeArray, etc.) +├── operators/ # Operator implementations +├── regex/ # Regex engine integration +├── perlmodule/ # Java implementations of Perl modules +└── scriptengine/ # JSR 223 scripting API + +src/main/perl/lib/ # Perl module implementations +src/test/resources/ # Test files +``` + +### Key Components + +- **Lexer** (`lexer/Lexer.java`) - Tokenizes Perl source +- **Parser** (`parser/Parser.java`) - Builds AST from tokens +- **EmitterVisitor** (`astvisitor/`) - Traverses AST and coordinates bytecode generation +- **Emit classes** (`codegen/`) - Generate JVM bytecode using ASM library +- **Runtime** (`runtime/`) - RuntimeScalar, RuntimeArray, RuntimeHash, RuntimeCode +- **Operators** (`operators/`) - Perl operator implementations + +### Test Organization + +``` +src/test/resources/ +└── unit/ # Fast unit tests (run in seconds) + +perl5_t/ (at project root) +├── t/ # Perl 5 core test suite +└── [Module]/ # Perl 5 module tests +``` + +## Development Guidelines + +### Building and Testing + +```bash +# Build with incremental compilation +make build + +# Force clean rebuild (during active development) +make dev + +# Run fast unit tests (default) +make test + +# Run all tests including Perl 5 core tests +make test-all + +# Run single test +./jperl path/to/test.t +``` + +### Adding Tests + +1. Create `.t` file in `src/test/resources/unit/` +2. Use TAP format (Test::More compatible) +3. Test should complete in < 1 second +4. Run with `./jperl path/to/test.t` + +Example test: +```perl +print "1..3\n"; # Plan +print "ok 1 - basic test\n"; +my $x = 1 + 1; +print $x == 2 ? "ok 2\n" : "not ok 2\n"; +print "ok 3 - done\n"; +``` + +### Debugging + +```bash +# Show debug information +./jperl --debug -E 'code' + +# Check syntax only +./jperl -c script.pl + +# Show disassembled bytecode +./jperl --disassemble script.pl + +# Run lexer only +./jperl --tokenize -E 'code' + +# Run parser only +./jperl --parse -E 'code' +``` + +## Documentation + +### User Documentation + +User-facing documentation is in `docs/`: +- **getting-started/** - Installation, quick start, Docker +- **guides/** - How-to guides (database, Java integration, modules) +- **reference/** - Technical reference (features, testing, architecture) +- **about/** - Project information (why, roadmap, support) + +### Developer Documentation + +Developer documentation is in `dev/`: +- **architecture/** - Internal architecture docs +- **implementation/** - Implementation notes (regex, tie, overload) +- **maintenance/** - Maintenance procedures + +## Feature Development + +### Before Adding a Feature + +1. Check if it's in [Feature Matrix](docs/reference/feature-matrix.md) +2. Review [Roadmap](docs/about/roadmap.md) +3. Open an issue to discuss the approach +4. Read relevant design docs in `dev/` + +### High-Risk Areas + +When modifying these, test extensively: +- **String parsing** (`parser/StringDoubleQuoted.java`) +- **Lexer** (`lexer/Lexer.java`) +- **Unicode handling** - Surrogate pairs, identifier parsing +- **Control flow** - die/eval, loop control +- **Bytecode generation** (`codegen/`, `astvisitor/EmitterVisitor.java`) + +### Adding Operators + +1. Implement in `operators/` package +2. Add parser support in `parser/` +3. Add tests in `src/test/resources/unit/` +4. Update [Feature Matrix](docs/reference/feature-matrix.md) + +### Porting Modules + +See **[Module Porting Guide](docs/guides/module-porting.md)** for: +- How to port pure-Perl modules +- Implementing XS modules in Java +- Using XSLoader mechanism + +## Community + +- **GitHub Issues** - Bug reports and feature requests +- **Pull Requests** - Code contributions +- **Discussions** - Questions and design discussions + +## Code of Conduct + +- Be respectful and constructive +- Focus on technical merit +- Help newcomers +- Keep discussions on-topic + +## Getting Help + +- Read the [Architecture Guide](docs/reference/architecture.md) +- Check existing [Issues](https://github.com/fglock/PerlOnJava/issues) +- Ask in [Discussions](https://github.com/fglock/PerlOnJava/discussions) +- See [Support](docs/about/support.md) for more resources + +## License + +By contributing, you agree that your contributions will be licensed under the Artistic License 2.0. diff --git a/MILESTONES.md b/MILESTONES.md index 1f04332af..5e97eb61a 100644 --- a/MILESTONES.md +++ b/MILESTONES.md @@ -1,523 +1,21 @@ # PerlOnJava Milestones -# Table of Contents +**Note:** This file has been split for better organization. -- [Completed Milestones](#completed-milestones) -- [Work in progress](#work-in-progress) -- [Upcoming Milestones](#upcoming-milestones) -- [Future Development Areas](#future-development-areas) +## Current Development -## Completed Milestones +See **[Roadmap](docs/about/roadmap.md)** for: +- Work in progress +- Upcoming milestones +- Future development areas -- **v5.42.2**: 250k Tests, Class Features, System V IPC, Sockets, and More +## Release History - - Add Perl 5.38+ Class Features - - Class keyword with block syntax fully working - - Method declarations with automatic $self injection - - Field declarations supporting all sigils ($, @, %) - - Constructor parameter fields with :param attribute - - Reader method generation with :reader attribute - - Automatic constructor generation with named parameters - - Default values for fields fully functional - - ADJUST blocks with field transformation working - - Field transformation to $self->{field} in methods - - Lexical method calls using $self->&priv syntax - - Class inheritance with :isa attribute working - - Version checking in :isa(Parent version) implemented - - Parent class field inheritance fully functional - - Object stringification shows OBJECT not HASH - - ClassRegistry tracks Perl 5.38+ class instances - - Context-aware reader methods for arrays/hashes - - Field transformation in string interpolation works - - __CLASS__ keyword with compile-time evaluation - - Add System V IPC operators: `msgctl`, `msgget`, `msgrcv`, `msgsnd`, `semctl`, `semget`, `semop`, `shmctl`, `shmget`, `shmread`, `shmwrite`. - - Add network enumeration operators: `endhostent`, `endnetent`, `endprotoent`, `endservent`, `gethostent`, `getnetbyaddr`, `getnetbyname`, `getnetent`, `getprotoent`, `getservent`, `sethostent`, `setnetent`, `setprotoent`, `setservent`. - - Add socket operators: `socket`, `bind`, `listen`, `accept`, `connect`, `send`, `recv`, `shutdown`, `setsockopt`, `getsockopt`, `getsockname`, `getpeername`, `socketpair`. - - Add Socket.pm module with socket constants and functions. - - Add `alarm` operator with `$SIG{ALRM}` signal handling. - - Fix `truncate` operator. - - Add `pipe` operator. - - Add `do \&subroutine`. - - Add `formline` operator and `$^A` accumulator variable - - Add file descriptor duplication support in `open` (`<&`, `>&`, `<&=`, `>&=`). - - Add statement: `format`, and `write` operator - - Add special variables: `@{^CAPTURE}`, `${^LAST_SUCCESSFUL_PATTERN}`. - - Add pack format `x`. - - Add `do filehandle`. - - Add module `Storable`, `experimental`, `Unicode::UCD`. - - Add single-quote as package separator. - - Dereferencing using `$$var{...}` and `$$var[...]` works. - - Add declared references: `my \$x`, `my(\@arr)`, `my(\%hash)`. - - Add subroutines declared `my`, `state`, or `our`. - - Bugfix in regex `/r`. - - Bugfix in transliterate with octal values. - - Bugfix in nested heredocs. - +See **[Changelog](docs/about/changelog.md)** for: +- Completed milestones +- Release notes +- Historical changes -- **v5.42.1**: 150k Tests, Extended Operators, and More Perl 5 Features - - - Add operators: `getlogin`, `getpwnam`, `getpwuid`, `getgrnam`, `getgrgid`, `getpwent`, `getgrent`, `setpwent`, `setgrent`, `endpwent`, `endgrent`, `gethostbyname`, `gethostbyaddr`, `getservbyname`, `getservbyport`, `getprotobyname`, `getprotobynumber`, `reset`. - - Add overload operators: `<=>`, `cmp`, `<`, `<=`, `>`, `>=`, `==`, `!=`, `lt`, `le`, `gt`, `ge`, `eq`, `ne`, `qr`. - - Add command line switches: `-s`, `-f`. - - Add `__CLASS__` keyword. - - Add modules: `mro`, `version`, `List::Util`. - - Add more `sprintf` formatters. - - Add readline modes depending on `$/` special variable. - - Add `PERL5OPT` environment variable. - - Add regex extended character classes `(?[...])` - - Bugfix: fixed vstring with codepoints above 65535. - - -- **v5.42.0**: 100k Tests Passed, Tie Support, and Total Compatibility - - Add `tie`, `tied`, `untie` operators. - - Add all `tie` types: scalar, array, hash, and handle. - - Add operators: `sysread`, `syswrite`, `kill`, `utime`, `chown`, `waitpid`, `umask`, `readlink`, `link`, `symlink`, `rename`. - - Add modules: `XSLoader`, `Encode`,`Config`, `Errno`, `Tie::Scalar`, `Tie::Array`, `Tie::Hash`, `Tie::Handle`, `Perl::OSType`, `Env`, `MIME::Base64`, `MIME::QuotedPrint`, `Digest::SHA`, `Digest::MD5`, `Digest`. - - Add key-value slices: `%c{"1", "3"}`. - - Add special variable: `$^X`. - - Add `W`, `H`, `F`, `h`, `c`, `u`, `C0`, `U0` formats to `pack`, `unpack`. - - Add dualvar. - - Add `DATA` file handle. - - Add Indirect method call. - - Add regex variables: `${^PREMATCH}`, `${^MATCH}`, `${^POSTMATCH}`. - - Add regex operators: `\N` not-newline, `\b{gcb}`, `\B{gcb}` boundary assertions. - - Add regex properties supported by Perl but missing in Java regex. - - Add command line switches: `-w`, `-W`, `-X`. - - Process `\L`, `\U`, `\l`, `\u` in regex. - - `Test::More` `skip` works. - - UTF-16 is accepted in source code. - - Add support for `pmc` files. - - Bugfix: methods can be called in all blessed reference types. - - Bugfix: more robust `sprintf` formatting. - - Bugfix: string constants can be larger than 64k. - - Bugfix: fixed foreach loops with global variables. - - -- **v3.1.0**: Tracks Perl 5.42.0 - - Update Perl version to `5.42.0`. - - Added features: `keyword_all`, `keyword_any` - - - Accept input program in several ways: - 1. **Piped input**: `echo 'print "Hello\n"' | ./jperl` - reads from pipe and executes immediately - 2. **Interactive input**: `./jperl` - shows a prompt and waits for you to type code, then press Ctrl+D (on Unix/Linux/Mac) or Ctrl+Z (on Windows) to signal end of input - 3. **File redirection**: `./jperl < script.pl` - reads from the file - 4. **With arguments**: `./jperl -e 'print "Hello\n"'` or `./jperl script.pl` - - - Added overload operators: `!`, `+`, `-`, `*`, `/`, `%`, `int`, `neg`, `log`, `sqrt`, `cos`, `sin`, `exp`, `abs`, `atan2`, `**`, `@{}`, `%{}`. `${}`, `&{}`, `*{}`. - - Subroutine prototypes are fully implemented. Added or fixed: `+`, `;`, `*`, `\@`, `\%`, `\$`, `\[@%]`. - - Added double quoted string escapes: `\U`, `\L`, `\u`, `\l`. - - Added star count (`C*`) in `pack`, `unpack`. - - Added operators: `read`, `tell`, `seek`, `system`, `exec`, `sysopen`, `chmod`. - - Added operator: `select(undef,undef,undef,$time)`. - - Added operator: `^^=`. - - Added operator: `delete`, `exists` for array indexes. - - Added `open` option: in-memory files. - - Syntax: identifiers starting with `::` are in `main` package. - - Added I/O layers support to `open`, `binmode`: `:raw`, `:bytes`, `:crlf`, `:utf8`, `:unix`, `:encoding()`. - - Add `open` support for pipe `-|`, `|-`, `ls|`, `|sort`. - - Added `# line` preprocessor directive. - - `Test::More` module: added `subtest`, `use_ok`, `require_ok`. - - `CORE::` operators have the same prototypes as in Perl. - - Added modules: `Fcntl`, `Test`, `Text::CSV`. - - Operator `$#` returns an lvalue. - - Improved autovivification handling: distinguish between contexts where undefined references should automatically create data structures versus where they should throw errors. - - Bugfix: fix a problem with Windows newlines and qw(). Also fixed `mkdir` in Windows. - - Bugfix: `-E` switch was setting strict mode. - - BugFix: fix calling context in operators that return list. - - BugFix: fix rules for overriding operators. - - Added Makefile. - - Debian package can be created with `make deb`. - - -- **v3.0.0**: Performance Boost, New Modules, and Streamlined Configuration - - Added `--upgrade` option to `Configure.pl` to upgrade dependencies. - - Added `Dockerfile` configuration. - - Added `Time::HiRes`, `Benchmark` modules. - - Added `/ee` regex modifier. - - Added no strict `vars`, `subs`. - - Execute the code generation on demand, for faster module loading. - - Use `int` instead of `enum` to reduce the memory overhead of scalar variables. - - -- **v2.3.0**: Modern Perl Features, Expanded Modules, and Developer Tools - - Project description updated in `README.md` to "A Perl Distribution for the JVM" - - Added module porting guide at `docs/PORTING_MODULES.md` - - Added wrapper scripts (`jperl`/`jperl.bat`) for easier command-line usage - - Added `YAML` and `YAML::PP` modules. - - Added `Text::Balanced` module. - - Added `Unicode::Normalize` module. - - Added subroutine signatures and `signature` feature. - - Added chained operators. - - Added stacked file test operators. - - Added `module_true` feature. - - Added `<<` and `<<~` Here documents. - - Added `/p`, `/c`, `/n` regex modifiers. - - Added regex `(?^` clear embedded pattern-match modifier. - - Added regex `(?'name'...)` named capture groups. - - Added regex `\k` and `\g{name}` backreferences to named groups. - - Added regex `\p{...}` and `\P{...}` for Unicode properties. - - Added regex `\g{-n}` for relative backreferences. - - Added regex `*+`, `++`, `?+`, `{n,m}+` possessive quantifiers. - - Added regex `(?>...)` for atomic groups. - - Added overload: `""`, `0+`, `bool`, `fallback`, `nomethod`. - - Added `class` feature and `class` keyword. - - Library upgrades. - Maven: `mvn versions:use-latest-versions`. - Gradle: `./gradlew useLatestVersions`. - -- **v2.2.0**: Core modules - - Perl version is now v5.40.0 - - `for` loop can iterate over multiple values at the same time. - - `for` loop variables are aliased. - - Added `DBI` module with JDBC support. - - Added `URI::Escape` module. - - Added `builtin` methods: `inf` `nan` `weaken` `unweaken` `is_weak` `blessed` `refaddr` `reftype` `created_as_string` `created_as_number` `stringify` `ceil` `floor` `indexed` `trim` `is_tainted`. - - Added command line switches: `-S`. - - Added low-precedence xor `^^` operator. - - Added [Configure.pl](Configure.pl) to set compiler options and add JDBC drivers. - - Added Links to Perl on JVM resources in README - https://github.com/fglock/PerlOnJava/tree/master#additional-information-and-resources - - Added [SUPPORT.md](docs/SUPPORT.md) - -- **v2.1.0**: Core modules and optimization - - Added `Getopt::Long`, `JSON` modules. - - Optimized `print` to `STDOUT`/`STDERR` performance by running in a separate thread. - - Added `subs` pragma. - - Added regex `$+` variable. - - Added command line switches: `-v`, `-V` . - - Added file test operators: `-R`, `-W`, `-X`, `-O`, `-t`. - - Added feature flags: `evalbytes`. - - Added `CORE::GLOBAL` and core function overrides. - - Added hexadecimal floating point numbers. - -- **v2.0.0**: Towards a Complete Perl Port on the JVM - - Added unmodified core Perl modules `File::Basename`, `File::Find`, `Data::Dumper`, `Term::ANSIColor`, `Time::Local`, `HTTP::Date`, `HTTP::CookieJar`. - - Added `Cwd`, `File::Spec`, `File::Spec::Functions`, `HTTP::Tiny` modules. - - "use feature" implemented: `fc`, `say`, `current_sub`, `isa`, `state`, `try`, `bitwise`, `postderef`. - - Stash can be accessed as a hash like `$namespace::{entry}`. - - Added stash constants: `$constant::{_CAN_PCS} = \$const`; - - Added `exists &sub`, `defined &sub`. - - Added `builtin` pragma: `true`, `false`, `is_bool`. - - Added `re` pragma: `is_regexp`. - - Added `vars` pragma. - - Added `SUPER::method` method resolution. - - Added `AUTOLOAD` default subroutine. - - Added `stat`, `lstat` operators. Some fields are not available in JVM and return `undef`. - - Added directory operators. - - Added regex patterns: `[[:ascii:]]`, `[[:print:]]`, `(?#comment)`, and the `/xx` modifier. - -- **v1.11.0**: Compile-time Features - - Added `BEGIN`, `CHECK`, `UNITCHECK`, `INIT`, `END` blocks. - - Added subroutine hoisting: Invoking subroutines before their actual declaration in the code. - - Improved Exporter.pm, glob assignment. - - Added modules: `constant`, `if`, `lib`, `Internals` (`SvREADONLY`), `Carp`. - - Added `goto &name`; not a tail-call. - - Added `state` variables. - - Added `$SIG{ALRM}`, `${^GLOBAL_PHASE}`. - - Added operators: `fileno`, `getc`, `prototype`. - - Added `\N{U+hex}` operator in double quoted strings and regex. - -- **v1.10.0**: Operators and Special Variables - - Error messages mimic those in Perl for consistency. - - Added `$.`, `$]`, `$^V`, `${^LAST_FH}`, `$SIG{__DIE__}`, `$SIG{__WARN__}` special variables. - - Added command line switches `-E`, `-p`, `-n`, `-i`, `-0`, `-a`, `-F`, `-m`, `-M`, `-g`, `-l`, `-x`, `-?`. - - Added `select(filehandle)` operator, `ARGVOUT` filehandle. - - Added `~.`, `&.`, `|.`, `^.` operators. - - Added `try catch` statement. - - Added Scalar::Util: `blessed`, `reftype`. - - Added UNIVERSAL: `VERSION`. - - Added v-strings. - - Added Infinity, -Infinity, NaN. - - Added `\N{name}` operator for named characters in double quoted strings and in regex. - - Added lvalue subroutines. - - CI/CD runs in Ubuntu and Windows - -- **v1.9.0**: Operators and Special Variables - - Added bitwise string operators. - - Added lvalue `substr`, lvalue `vec` - - Fix `%b` specifier in `sprintf` - - Emulate Perl behaviour with unsigned integers in bitwise operators. - - Regex `m?pat?` match-once and the `reset()` operator are implemented. - - Regex `\G` and the `pos` operator are implemented. - - Regex `@-`, `@+`, `%+`, `%-` special variables are implemented. - - Regex `` $` ``, `$&`, `$'` special variables are implemented. - - Regex performance comparable to Perl; optimized regex variables. - - Regex matching plain strings: `$var =~ "Test"`. - - Added `__SUB__` keyword; `readpipe`. - - Added `&$sub` call syntax. - - Added `local` dynamic variables. - - Tests in `src/test/resources` are executed automatically. - -- **v1.8.0**: Operators - - Added `continue` blocks and loop operators `next`, `last`, `redo`; a bare-block is a loop - - Added bitwise operators `vec`, `pack`, `unpack` - - Added `srand`, `crypt`, `exit`, ellipsis statement (`...`) - - Added `readdir`, `opendir`, `closedir`, `telldir`, `seekdir`, `rewinddir`, `mkdir`, `rmdir` - - Added file test operators like `-d`, `-f` - - Added the variants of diamond operator `<>` and special cases of `while` - - Completed `chomp` operator; fixed `qw//` operator, `defined-or` and `x=` - - Added modules: `parent`, `Test::More` - -- **v1.7.0**: Performance Improvements - - Focus on optimizing the execution engine for better performance. - - Improve error handling and debugging tools to make development easier. More detailed debugging symbols added to the bytecode. Added `Carp` module. - - Moved Perl standard library modules into the jar file. - - More tests and various bug fixes - -- **v1.6.0**: Module System and Standard Library Enhancements - - Module system for improved code organization and reuse - - Core Perl module operators: `do FILE`, `require`, `caller`, `use`, `no` - - Module special subroutines: `import`, `unimport` - - Environment and special variables: `PERL5LIB`, `@INC`, `%INC`, `@ARGV`, `%ENV`, `$0`, `$` - - Additional operators: `die`, `warn`, `time`, `times`, `localtime`, `gmtime`, `index`, `rindex` - - Standard library ported modules: `Data::Dumper`, `Symbol`, `strict` - - Expanded documentation and usage examples - -- **v1.5.0**: Regex operators - - Added Regular expressions and pattern matching: m//, pos, qr//, quotemeta, s///, split - - More complete set of operations on strings, numbers, arrays, hashes, lists - - More special variables - - More tests and various bug fixes - -- **v1.4.0**: I/O operators - - File i/o operators, STDOUT, STDERR, STDIN - - TAP (Perl standard) tests - -- **v1.3.0**: Added Objects. - - Objects and object operators, UNIVERSAL class - - Array and List related operators - - More tests and various bug fixes - -- **v1.2.0**: Added Namespaces and named subroutines. - - Added typeglobs - - Added more operators - -- **v1.1.0**: Established architecture and added key features. The system now supports benchmarks and tests. - - JSR 223 integration - - Support for closures - - Eval-string functionality - - Enhanced statements, data types, and call context - -- **v1.0.0**: Initial proof of concept for the parser and execution engine. - - -## Work in Progress - -The following areas are currently under active development to enhance the functionality and performance of PerlOnJava: - -- **Compiler Subsystem** - - Implementing the `lexical_subs` feature. - - Enhancing version control with `use VERSION` and `require VERSION`. - - Introducing lexical warnings and strictness for improved code safety. - - Supporting lexical UTF-8 source code. - - Ensure that stack traces are well-formed. - - Developing subroutine prototypes. - - Handling `goto` special cases for `Text::Balanced`. - - Addressing indirect object special cases for `GetOpt::Long`. - - Localizing regex variables. - - Fix handling of global variable aliasing in `for`. - -- **Regex Subsystem** - - Ongoing improvements and feature additions. - -- **Class Subsystem** - - Enhancements and new feature implementations. - -- **DBI Subsystem** - - Adding additional methods for database interaction. - -- **Overload Subsystem** - - Expanding with additional methods. - -- **I/O Subsystem** - - Implementing in-memory I/O operations. - - Developing `socket` and related operators. - - Enhancing `truncate`, `seek`, and `read` operators. - - Introducing IO layers and experimenting with memory-mapped I/O. - -- **Threads Subsystem** - - Documenting preliminary features. - - Planning for fork emulation and interpreter cloning. - - Developing `multiplicity` support. - - Adjusting `exit` semantics to exit the current thread only. - - Adding `ThreadLocal` to global variables, special variables, and caches, with tear down hooks for cleanup. - -- **CPAN Support** - - Porting `cpan` and `prove`. - - Adding module testing for PerlOnJava core modules. - -- **Optimization** - - Inlining `map` and related blocks. - - Inlining constant subroutines. - - Prefetch named subroutines to lexical (`our`). - -- **Compilation with GraalVM** - - Documenting preliminary results in [docs/GRAALVM.md](docs/GRAALVM.md). - - -## Upcoming Milestones - -- **v5.42.3**: Next minor version - - - Non-local control flow: `last`/`next`/`redo`/`goto LABEL` - - Tail call with trampoline for `goto &NAME` and `goto __SUB__` - - Add modules: `TOML`. - - Bugfix: operator override in Time::Hires now works. - - Bugfix: internal temp variables are now pre-initialized. - - Optimization: faster list assignment. - - Optimization: faster type resolution in Perl scalars. - - Optimization: `make` now runs tests in parallel. - - Optimization: A workaround is implemented to Java 64k bytes segment limit. - - Planned release date: 2026-02-10. - -- Work in Progress - - PerlIO - - `get_layers` - - Term::ReadLine - - Term::ReadKey - - FileHandle - - File::Temp - - File::Path - - File::Copy - - IO::File - - IO::Handle - - `ungetc` - - Auto-bless filehandle into IO::Handle subclass - - IO::Seekable - - Filter::Simple - - Math::BigInt - - Text::ParseWords - - Text::Tabs - - Locale::Maketext::Simple - - Params::Check - - SelectSaver - - locale pragma - - utf8 pragma - - bytes pragma - - threads pragma - - warnings pragma - - vmsish pragma - - Constant folding - in ConstantFoldingVisitor.java - - `method` keyword - - Overload operators: `++`, `--`. - - String interpolation fixes. - - Command line option `-C` - -### v4.0.0 Milestone (Planned Release Date: 2026-05-10) - -**Objective:** Enhance core functionality and improve developer experience with a focus on integration and performance. - -1. **Concurrency and Security Enhancements** - - **Specific:** Implement basic concurrency support with threads and async/await capabilities. - - **Measurable:** Ensure at least 50% of core modules support concurrent execution. - - **Achievable:** Utilize existing JVM concurrency features. - - **Relevant:** Addresses modern application requirements for parallel processing. - - **Time-bound:** Complete by Q1 2026. - -2. **External Integration** - - **Specific:** Integrate with popular external libraries for HTTP requests and database access. - - **Measurable:** Provide at least two integration examples in documentation. - - **Achievable:** Leverage existing Java libraries for integration. - - **Relevant:** Enhances PerlOnJava's utility in web and data applications. - - **Time-bound:** Complete by Q1 2026. - -3. **Development Tools** - - **Specific:** Develop an interactive debugger with breakpoints and variable inspection. - - **Measurable:** Debugger should support at least 80% of core language features. - - **Achievable:** Build on existing JVM debugging capabilities. - - **Relevant:** Improves developer productivity and code quality. - - **Time-bound:** Complete by Q1 2026. - -4. **Distribution and Packaging** - - **Specific:** Provide Docker containers and Kubernetes configurations for easy deployment. - - **Measurable:** Ensure compatibility with at least two major cloud platforms. - - **Achievable:** Use standard containerization practices. - - **Relevant:** Facilitates deployment in modern cloud environments. - - **Time-bound:** Complete by Q1 2026. - -### v5.0.0 Milestone (Planned Release Date: 2027-04-10) - -**Objective:** Expand platform capabilities and improve performance with advanced features and optimizations. - -1. **GraalVM Integration** - - **Specific:** Implement native image compilation for instant startup and reduced memory footprint. - - **Measurable:** Achieve at least a 30% reduction in startup time and memory usage. - - **Achievable:** Collaborate with GraalVM community for support. - - **Relevant:** Enhances performance for cloud and serverless applications. - - **Time-bound:** Complete by Q1 2027. - -2. **Mobile Development** - - **Specific:** Develop an Android app with native UI for Perl scripting. - - **Measurable:** Ensure app compatibility with at least 80% of Android devices. - - **Achievable:** Utilize existing Android development frameworks. - - **Relevant:** Expands PerlOnJava's reach to mobile developers. - - **Time-bound:** Complete by Q1 2027. - -3. **Advanced Data Manipulation** - - **Specific:** Add XML parsing and data transformation features. - - **Measurable:** Provide at least three example scripts demonstrating these capabilities. - - **Achievable:** Use existing Java libraries for data manipulation. - - **Relevant:** Increases PerlOnJava's applicability in data-driven applications. - - **Time-bound:** Complete by Q1 2027. - -4. **Enterprise Features** - - **Specific:** Integrate with logging frameworks and provide monitoring with Prometheus/Grafana. - - **Measurable:** Ensure logging and monitoring support for at least 70% of core modules. - - **Achievable:** Leverage existing enterprise tools and frameworks. - - **Relevant:** Meets enterprise requirements for observability and security. - - **Time-bound:** Complete by Q1 2027. - - -## Future Development Areas - -- **Concurrency and Security Features** - - Add support for concurrency and parallelism, such as threads and async/await. - - Enhance security features, including sandboxing and input validation. - - Increase test coverage. - -- **External Integration and Advanced Data Manipulation** - - Integrate with external libraries and APIs for tasks like HTTP requests and database access. - - Add advanced data manipulation features, such as JSON/XML parsing and data transformation. - - Allow users to define their own operators and macros for greater flexibility. - -- **Mobile Development** - - Android app with native UI for Perl scripting - - Integration with Android's file system and permissions - - Mobile-optimized performance settings - -- **Distribution and Packaging** - - Native installers for Windows/Linux/MacOS - - Docker containers and Kubernetes configurations - - Maven Central Repository publishing - - Integration with package managers (apt, yum, chocolatey, homebrew) - -- **Development Tools** - - IDE plugins for IntelliJ IDEA, Eclipse, VSCode - - Interactive debugger with breakpoints and variable inspection - - Code formatting and static analysis tools - - Performance profiling tools - -- **Enterprise Features** - - Monitoring and metrics with Prometheus/Grafana - - Integration with logging frameworks (Log4j, SLF4J) - - Security hardening and vulnerability scanning - - Cloud deployment templates (AWS, Azure, GCP) - -- **GraalVM Integration** - - Native image compilation for instant startup - - Polyglot integration with JavaScript, Python, Ruby - - Ahead-of-time compilation optimizations - - Reduced memory footprint for cloud deployments - - SubstrateVM support for containerized environments - -- **Native System Integration** - - JNI/XS bridge to enable native code modules - - Integration with existing Perl XS ecosystem - - Core system operations - - Process control (fork, exec) - - Direct memory access (mmap) - - Native sockets and file descriptors - - Platform-specific features - - Unix: ptrace, signals, IPC - - Windows: Registry, COM objects - - MacOS: FSEvents, CoreFoundation, Security framework +--- +For the complete documentation structure, see [README.md](README.md). diff --git a/Makefile b/Makefile index d96a25f0c..244afa93e 100644 --- a/Makefile +++ b/Makefile @@ -33,10 +33,13 @@ endif test: test-unit # Fast unit tests only (from src/test/resources/unit/ directory) -# Uses perl_test_runner.pl with TAP output and parallel execution -test-unit: - @echo "Running fast unit tests..." - perl dev/tools/perl_test_runner.pl --jobs 8 --timeout 10 src/test/resources/unit +# Uses Gradle's testUnitParallel (same as default make build) +test-unit: wrapper +ifeq ($(OS),Windows_NT) + gradlew.bat testUnitParallel --parallel +else + ./gradlew testUnitParallel --parallel +endif # Perl 5 core test suite (perl5_t/t/ directory) # Run 'perl dev/import-perl5/sync.pl' first to populate perl5_t/ diff --git a/QUICKSTART.md b/QUICKSTART.md new file mode 100644 index 000000000..4146ebbb7 --- /dev/null +++ b/QUICKSTART.md @@ -0,0 +1,191 @@ +# Quick Start Guide + +Get PerlOnJava running in 5 minutes. + +## Prerequisites + +- **Java Development Kit (JDK) 21 or later** +- **Git** for cloning the repository +- **Make** (optional - can use Gradle directly) + +### Verify You Have JDK Installed + +Check your Java version: +```bash +java -version +``` +Should show version 21 or higher. + +**Important:** Check you have the **JDK** (not just JRE): +```bash +javac -version +``` +Should show the same version. If `javac: command not found`, you need to install a JDK. + +**Installing JDK:** +- Use your system's package manager, or +- Download from a JDK provider (Adoptium, Oracle, Azul, Amazon Corretto, etc.) +- Common package manager commands: + - **macOS**: `brew install openjdk@21` + - **Ubuntu/Debian**: `sudo apt install openjdk-21-jdk` + - **Windows**: Use package manager like [Chocolatey](https://chocolatey.org/) or [Scoop](https://scoop.sh/) + +## Installation + +### 1. Clone and Build + +```bash +git clone https://github.com/fglock/PerlOnJava.git +cd PerlOnJava +make +``` + +The `make` command compiles the project and runs the fast unit tests. The complete build with tests typically completes in ~30 seconds. + +**Build troubleshooting:** See [Installation Guide](docs/getting-started/installation.md) + +**Debian/Ubuntu users:** You can also build and install a `.deb` package: +```bash +make deb +sudo dpkg -i build/distributions/perlonjava_*.deb +``` +This installs `jperl` systemwide. See [Installation Guide](docs/getting-started/installation.md#debian-package) for details. + +### 2. Verify Installation + +
+Linux/Mac + +```bash +./jperl -E 'say "Hello from PerlOnJava!"' +./jperl -v # Show version +``` +
+ +
+Windows + +```bash +jperl -E "say 'Hello from PerlOnJava!'" +jperl -v # Show version +``` +
+ +## Basic Usage + +### Run a Perl Script + +```bash +# One-liner +./jperl -E 'for (1..5) { say "Count: $_" }' + +# Script file +echo 'use strict; use warnings; say "It works!";' > test.pl +./jperl test.pl +``` + +### Use Core Modules + +```bash +./jperl -MJSON -E 'say encode_json({hello => "world"})' +./jperl -MYAML::PP -E 'say Dump({foo => "bar"})' +./jperl -MData::Dumper -E 'print Dumper [1,2,3]' +``` + +### More Examples + +See **[One-liners Guide](docs/getting-started/oneliners.md)** for practical examples. + +## Database Access with DBI + +PerlOnJava includes the DBI module with JDBC support. + +### Quick Example + +**1. Download a JDBC driver** (H2 database for testing): +```bash +wget https://repo1.maven.org/maven2/com/h2database/h2/2.2.224/h2-2.2.224.jar +``` + +**2. Set CLASSPATH and run:** +```bash +export CLASSPATH=/path/to/h2-2.2.224.jar +./jperl your_script.pl +``` + +**3. Use DBI in your script:** +```perl +use DBI; + +my $dbh = DBI->connect("jdbc:h2:mem:testdb;DB_CLOSE_DELAY=-1") + or die $DBI::errstr; + +$dbh->do("CREATE TABLE users (id INT, name VARCHAR(50))"); +$dbh->do("INSERT INTO users VALUES (1, 'Alice')"); + +my $sth = $dbh->prepare("SELECT * FROM users"); +$sth->execute(); + +while (my $row = $sth->fetchrow_hashref) { + say "$row->{id}: $row->{name}"; +} +``` + +**→ For PostgreSQL, MySQL, and other databases:** See **[Database Access Guide](docs/guides/database-access.md)** + +## Using Perl from Java + +PerlOnJava implements JSR-223 (Java Scripting API): + +```java +import javax.script.*; + +public class TestPerl { + public static void main(String[] args) throws Exception { + ScriptEngineManager manager = new ScriptEngineManager(); + ScriptEngine engine = manager.getEngineByName("perl"); + + // Execute Perl code + engine.eval("print 'Hello from Java!\\n'"); + + // Pass variables + engine.put("name", "World"); + engine.eval("say \"Hello, $name!\""); + + // Get results + Object result = engine.eval("2 + 2"); + System.out.println("Result: " + result); + } +} +``` + +**Full guide:** **[Java Integration Guide](docs/guides/java-integration.md)** + +## Running in Docker + +Quick start with Docker: + +```bash +# Build image +docker build -t perlonjava . + +# Run container +docker run -it perlonjava ./jperl -E 'say "Hello from Docker!"' +``` + +**Full guide:** **[Docker Guide](docs/getting-started/docker.md)** + +## Next Steps + +### Learn More +- **[Feature Matrix](docs/reference/feature-matrix.md)** - See what Perl features are supported +- **[One-liners](docs/getting-started/oneliners.md)** - Practical Perl examples +- **[Module Porting](docs/guides/module-porting.md)** - Port your favorite Perl modules + +### Get Help +- **[Support](docs/about/support.md)** - Community resources +- **[GitHub Issues](https://github.com/fglock/PerlOnJava/issues)** - Report bugs + +### Contribute +- **[CONTRIBUTING.md](CONTRIBUTING.md)** - How to contribute +- **[Roadmap](docs/about/roadmap.md)** - Future plans diff --git a/README.md b/README.md index d461c85ac..8766b8545 100644 --- a/README.md +++ b/README.md @@ -1,255 +1,73 @@ -# PerlOnJava: A Perl Distribution for the JVM +# PerlOnJava -PerlOnJava provides a Perl distribution designed to run natively on the Java Virtual Machine (JVM). -It allows Perl scripts to integrate seamlessly with Java-based ecosystems while offering familiar tools and modules for Perl development. +> Perl running natively on the JVM -The JAR package features a variety of Perl modules, such as `DBI` (with JDBC support), `HTTP::Tiny`, `Text::CSV`, `JSON`, `YAML`, `TOML`, `File::Find`, and `Data::Dumper`. -Users can also add their own database JDBC drivers, making it a flexible solution for cross-platform Perl applications. +[![Build Status](https://github.com/fglock/PerlOnJava/workflows/CI/badge.svg)](https://github.com/fglock/PerlOnJava/actions) +[![License](https://img.shields.io/badge/license-Artistic_2.0-blue.svg)](LICENSE.md) -## Table of Contents +## About Perl -1. [Introduction](#introduction) -2. [Why PerlOnJava](docs/WHY_PERLONJAVA.md) -3. [Target Audience](#target-audience) -4. [Quick Start](#quick-start) -5. [Features and Limitations](docs/FEATURE_MATRIX.md) -6. [Build Instructions](docs/BUILD.md) -7. [Testing](docs/TESTING.md) -8. [Running with Docker](docs/DOCKER.md) -9. [Running the JAR File](#running-the-jar-file) -10. [Debugging Tools](#debugging-tools) -11. [Architecture](docs/ARCHITECTURE.md) -12. [Porting Modules](docs/PORTING_MODULES.md) -13. [Milestones](MILESTONES.md) -14. [Community and Support](docs/SUPPORT.md) -15. [License](#license) -16. [Additional Resources](docs/RESOURCES.md) +[Perl](https://www.perl.org/) is a high-level, general-purpose programming language known for text processing, system administration, web development, and database integration. It combines features from C, shell scripting, and other languages with powerful regular expressions and flexible syntax. -## Introduction +**Learn more:** [www.perl.org](https://www.perl.org/) -PerlOnJava bridges the gap between Perl and Java by providing a platform that compiles Perl scripts into Java bytecode, making them executable on the JVM. -By leveraging this distribution, developers can run familiar Perl code while accessing Java's ecosystem of libraries and tools. -This project aims to bring the strengths of Perl to modern JVM environments while supporting a growing list of core modules and pragmas. - -Need help? Check out our [Community and Support](docs/SUPPORT.md) section. - -### What This Project Is - -- **A JVM-Native Perl Implementation**: Runs Perl code directly on the Java Virtual Machine -- **A Bridge to Java Ecosystems**: Enables Perl scripts to interact with Java libraries and frameworks -- **A Cross-Platform Solution**: Provides consistent Perl behavior across different operating systems via JVM -- **A Modern Integration Tool**: Allows Perl codebases to participate in Java-based enterprise environments - -## Target Audience - -- **Java Developers with Perl Knowledge**: Provides a method for integrating Perl scripts into Java applications. -- **Compiler and Language Enthusiasts**: Offers insights into translating high-level languages into JVM bytecode. -- **Experimenters and Tinkerers**: A tool for experimenting with language interoperability. +## What is PerlOnJava? +A Perl compiler and runtime for the JVM that: +- Compiles Perl scripts to Java bytecode +- Integrates with Java ecosystems (JDBC, Maven, Spring) +- Supports most Perl 5.42 features +- Includes 150+ core Perl modules (DBI, HTTP::Tiny, JSON, YAML, Text::CSV) ## Quick Start -Get started quickly with PerlOnJava. For a complete list of capabilities, see our [Feature Matrix](docs/FEATURE_MATRIX.md). - -1. Build the project ([detailed instructions](docs/BUILD.md)): - ```bash - make - ``` - -2. Run a simple Perl script: - -
-Linux/Mac - -```bash -./jperl -E 'say "Hello World"' -``` -
- -
-Windows - -```bash -jperl -E "say 'Hello World'" -``` -
- -3. Use Perl in your Java application: - ```java - import javax.script.*; - - public class TestPerl { - public static void main(String[] args) throws Exception { - ScriptEngineManager manager = new ScriptEngineManager(); - ScriptEngine engine = manager.getEngineByName("perl"); - engine.eval("print 'Hello from Java-integrated Perl!\n'"); - } - } - ``` - -4. Connect to a database: - ```perl - use DBI; - my $dbh = DBI->connect("jdbc:h2:mem:testdb;DB_CLOSE_DELAY=-1"); - $dbh->do("CREATE TABLE test (id INT, name VARCHAR(50))"); - $dbh->do("INSERT INTO test VALUES (1, 'Hello World')"); - ``` - -## Testing - -PerlOnJava provides a two-level testing strategy for development and validation: - -### Fast Unit Tests (Recommended for Development) - -```bash -make test # Fast unit tests (default) -make test-unit # Same as above -``` - -Runs tests from `src/test/resources/unit/` in seconds with parallel execution. - -### Comprehensive Test Suite - -```bash -make test-all # All tests including module tests -``` - -Runs all tests including Benchmark.pm and other Perl core modules. Takes longer but provides comprehensive validation with detailed JSON reports. - -See [Testing Guide](docs/TESTING.md) for detailed information about: -- Test organization and categories -- Using `perl_test_runner.pl` (like `prove`) -- JUnit/Gradle integration for CI/CD -- Performance tips and debugging - -## Running the JAR File - -1. **Show Instructions**: -
-Linux/Mac - ```bash -./jperl --help -``` -
- -
-Windows - -```bash -jperl --help -``` -
- -2. **Execute Something**: -
-Linux/Mac - -```bash -./jperl -E 'print 123' -``` -
+# Build +make -
-Windows - -```bash -jperl -E "print 123" -``` -
- -## Debugging Tools - -1. **Execute Emitting Debug Information**: -
-Linux/Mac - -```bash -./jperl --debug -E 'print 123' -``` -
- -
-Windows - -```bash -jperl --debug -E "print 123" -``` -
- -2. **Compile Only; Can Be Combined with --debug**: -
-Linux/Mac - -```bash -./jperl -c -E 'print 123' -./jperl --debug -c -E 'print 123' -``` -
- -
-Windows - -```bash -jperl -c -E "print 123" -jperl --debug -c -E "print 123" +# Run Perl +./jperl -E 'say "Hello World"' ``` -
-3. **Execute and Emit Disassembled ASM Code**: -
-Linux/Mac +**→ [Full Quick Start Guide](QUICKSTART.md)** - Installation, examples, and database setup -```bash -./jperl --disassemble -E 'print 123' -``` -
+**→ [Database Access Guide](docs/guides/database-access.md)** - DBI with JDBC drivers -
-Windows - -```bash -jperl --disassemble -E "print 123" -``` -
- -4. **Run the Lexer Only**: -
-Linux/Mac - -```bash -./jperl --tokenize -E 'print 123' -``` -
+## Documentation -
-Windows +### Getting Started +- **[Installation](docs/getting-started/installation.md)** - Build and setup +- **[Quick Start](QUICKSTART.md)** - Get running in 5 minutes +- **[Docker](docs/getting-started/docker.md)** - Run in containers +- **[One-liners](docs/getting-started/oneliners.md)** - Quick examples -```bash -jperl --tokenize -E "print 123" -``` -
+### Guides +- **[Database Access](docs/guides/database-access.md)** - Using DBI with JDBC drivers +- **[Java Integration](docs/guides/java-integration.md)** - Call Perl from Java (JSR-223) +- **[Module Porting](docs/guides/module-porting.md)** - Port Perl modules -5. **Run the Parser Only**: -
-Linux/Mac +### Reference +- **[Feature Matrix](docs/reference/feature-matrix.md)** - What's implemented +- **[Testing](docs/reference/testing.md)** - Test suite information +- **[Architecture](docs/reference/architecture.md)** - How it works +- **[CLI Options](docs/reference/cli-options.md)** - Command-line reference +- **[Configure.pl](docs/reference/configure.md)** - Configuration and dependency management -```bash -./jperl --parse -E 'print 123' -``` -
+### About +- **[Why PerlOnJava?](docs/about/why-perlonjava.md)** - Project goals and use cases +- **[Roadmap](docs/about/roadmap.md)** - Future plans +- **[Changelog](docs/about/changelog.md)** - Release history +- **[Support](docs/about/support.md)** - Get help and contribute +- **[Resources](docs/about/resources.md)** - External links -
-Windows +## Contributing -```bash -jperl --parse -E "print 123" -``` -
+We welcome contributions! See **[CONTRIBUTING.md](CONTRIBUTING.md)** for: +- How to build and test +- Code organization +- Submitting pull requests +- Developer documentation ## License -This project is licensed under the Perl Artistic License 2.0 - see the [LICENSE](LICENSE.md) file for details. - -![Java CI with Maven](https://github.com/fglock/PerlOnJava/workflows/Java%20CI%20with%20Maven/badge.svg) - +[Artistic License 2.0](LICENSE.md) - Copyright (c) Flavio Glock diff --git a/dev/design/BLOCK_DISPATCHER_OPTIMIZATION.md b/dev/architecture/block-dispatcher-optimization.md similarity index 100% rename from dev/design/BLOCK_DISPATCHER_OPTIMIZATION.md rename to dev/architecture/block-dispatcher-optimization.md diff --git a/dev/design/CONTROL_FLOW_IMPLEMENTATION.md b/dev/architecture/control-flow.md similarity index 100% rename from dev/design/CONTROL_FLOW_IMPLEMENTATION.md rename to dev/architecture/control-flow.md diff --git a/dev/design/INLINE_CACHE_IMPLEMENTATION.md b/dev/architecture/inline-cache.md similarity index 100% rename from dev/design/INLINE_CACHE_IMPLEMENTATION.md rename to dev/architecture/inline-cache.md diff --git a/dev/design/LARGECODE_REFACTORING.md b/dev/architecture/large-code-refactoring.md similarity index 100% rename from dev/design/LARGECODE_REFACTORING.md rename to dev/architecture/large-code-refactoring.md diff --git a/dev/design/METHOD_CALL_OPTIMIZATION_PLAN.md b/dev/architecture/method-call-optimization.md similarity index 100% rename from dev/design/METHOD_CALL_OPTIMIZATION_PLAN.md rename to dev/architecture/method-call-optimization.md diff --git a/docs/GRAALVM.md b/dev/design/graalvm.md similarity index 100% rename from docs/GRAALVM.md rename to dev/design/graalvm.md diff --git a/dev/design/alarm.md b/dev/implementation/alarm.md similarity index 100% rename from dev/design/alarm.md rename to dev/implementation/alarm.md diff --git a/dev/design/overload.md b/dev/implementation/overload.md similarity index 100% rename from dev/design/overload.md rename to dev/implementation/overload.md diff --git a/docs/PACK_UNPACK_ARCHITECTURE.md b/dev/implementation/pack-unpack.md similarity index 100% rename from docs/PACK_UNPACK_ARCHITECTURE.md rename to dev/implementation/pack-unpack.md diff --git a/dev/design/regex.md b/dev/implementation/regex.md similarity index 100% rename from dev/design/regex.md rename to dev/implementation/regex.md diff --git a/dev/design/signal_handling.md b/dev/implementation/signal-handling.md similarity index 100% rename from dev/design/signal_handling.md rename to dev/implementation/signal-handling.md diff --git a/dev/design/tie.md b/dev/implementation/tie.md similarity index 100% rename from dev/design/tie.md rename to dev/implementation/tie.md diff --git a/dev/design/DEPENDENCY_UPDATES.md b/dev/maintenance/dependency-updates.md similarity index 100% rename from dev/design/DEPENDENCY_UPDATES.md rename to dev/maintenance/dependency-updates.md diff --git a/docs/BUILD.md b/docs/BUILD.md deleted file mode 100644 index c71be89d2..000000000 --- a/docs/BUILD.md +++ /dev/null @@ -1,130 +0,0 @@ -# Build and Execution Guide - -## Table of Contents -1. [Prerequisites](#prerequisites) -2. [Build Options](#build-options) - - [Using Maven](#using-maven) - - [Using Gradle](#using-gradle) -3. [Dependencies](#dependencies) -4. [Running PerlOnJava](#running-perlonjava) - - [Platform-Specific Instructions](#platform-specific-instructions) - - [Common Options](#common-options) -5. [Database Integration](#database-integration) - - [Adding JDBC Drivers](#adding-jdbc-drivers) - - [Database Connection Example](#database-connection-example) -6. [Build Notes](#build-notes) -7. [Java Library Upgrades](#java-library-upgrades) -8. [Using Configure.pl](#using-configurepl) - - [Upgrade Dependencies](#upgrade-dependencies) - -## Prerequisites -- Java 21 or higher -- Maven or Gradle -- Optional: JDBC drivers for database connectivity - -## Build Options - -### Using Make -The project includes a Makefile that wraps Gradle commands for a familiar build experience: - -```bash -make # same as 'make build' -make build # builds the project -make test # runs tests -make clean # cleans build artifacts -make deb # creates a Debian package -``` - -### Using Maven -```bash -mvn clean package -``` - -### Using Gradle -```bash -./gradlew clean build -``` - -## Dependencies -- JUnit: For testing -- ASM: For bytecode manipulation -- ICU4J: For Unicode support -- FASTJSON v2: For JSON support -- SnakeYAML Engine: for YAML support - -## Running PerlOnJava - -### Platform-Specific Instructions - -**Unix/Linux/Mac:** -```bash -./jperl -E 'print "Hello World"' -./jperl myscript.pl -CLASSPATH="jdbc-drivers/h2-2.2.224.jar" ./jperl myscript.pl -``` - -**Windows:** -```bash -jperl -E "print 'Hello World'" -jperl myscript.pl -set CLASSPATH=jdbc-drivers\h2-2.2.224.jar -jperl myscript.pl -``` - -### Common Options -- `-I lib`: Add library path -- `--debug`: Enable debug output -- `--help`: Show all options - -## Database Integration - -### Adding JDBC Drivers - -1. Using Configure.pl: -```bash -./Configure.pl --search mysql-connector-java -``` - -2. Using Java classpath (shown in platform-specific examples above) - -### Database Connection Example -```perl -use DBI; -my $dbh = DBI->connect("jdbc:h2:mem:testdb;DB_CLOSE_DELAY=-1"); -$dbh->do("CREATE TABLE test (id INT, name VARCHAR(50))"); -``` - -See [JDBC Database Guide](JDBC_GUIDE.md) for detailed connection examples and supported databases. - -## Build Notes -- Maven builds use `maven-shade-plugin` for creating the shaded JAR -- Gradle builds use `com.github.johnrengelman.shadow` plugin -- Both configurations target Java 21 - -## Java Library Upgrades - -**Maven:** - -`mvn versions:use-latest-versions`. - -**Gradle:** - -`./gradlew useLatestVersions`. - -## Using Configure.pl - -### Upgrade Dependencies - -The `Configure.pl` script provides an option to upgrade your project's dependencies to their latest versions. This can be useful for ensuring that your project uses the most up-to-date libraries. - -To upgrade dependencies, use the `--upgrade` option: - -```bash -perl ./Configure.pl --upgrade -``` - -This command will: -- Update Maven dependencies to their latest versions if a `pom.xml` file is present. -- Update Gradle dependencies to their latest versions if a `build.gradle` file is present. - -Make sure that Maven and Gradle are installed and accessible in your environment. diff --git a/docs/DOCKER.md b/docs/DOCKER.md deleted file mode 100644 index 3e6d277cc..000000000 --- a/docs/DOCKER.md +++ /dev/null @@ -1,49 +0,0 @@ -## Running the Application with Docker - -This project includes a `Dockerfile` that allows you to build and run the application in a Docker container. Follow the steps below to get started: - -### Prerequisites - -- Ensure you have [Docker](https://www.docker.com/products/docker-desktop) installed on your machine. - -### Building the Docker Image - -To build the Docker image, navigate to the root directory of the project where the `Dockerfile` is located and run the following command: - -```bash -docker build -t perlonjava:latest . -``` - -This command will create a Docker image named `perlonjava` with the `latest` tag. - -### Running the Docker Container - -Once the image is built, you can run the application in a Docker container using the following command: - -```bash -docker run --name perlonjava_app -p 8080:8080 perlonjava:latest -E ' say "hello, World!" ' -``` - -- `--name perlonjava_app`: Assigns a name to the running container. -- `-p 8080:8080`: Maps port 8080 of the host to port 8080 of the container. Adjust the ports as needed for your application. - -### Stopping and Removing the Container - -To stop the running container, use the following command: - -```bash -docker stop perlonjava_app -``` - -To remove the container after stopping it, run: - -```bash -docker rm perlonjava_app -``` - -### Additional Notes - -- Ensure that any required environment variables or configuration files are properly set up before running the container. -- You can modify the `Dockerfile` to include additional dependencies or configurations as needed for your specific use case. - - diff --git a/docs/README.md b/docs/README.md new file mode 100644 index 000000000..95be56de1 --- /dev/null +++ b/docs/README.md @@ -0,0 +1,58 @@ +# PerlOnJava Documentation + +User-facing documentation for PerlOnJava. + +## Getting Started + +New to PerlOnJava? Start here: + +- **[Quick Start Guide](../QUICKSTART.md)** - Get running in 5 minutes +- **[Installation](getting-started/installation.md)** - Build and setup instructions +- **[Docker](getting-started/docker.md)** - Run PerlOnJava in containers +- **[One-liners](getting-started/oneliners.md)** - Practical Perl examples + +## Guides + +How-to guides for common tasks: + +- **[Database Access](guides/database-access.md)** - Using DBI with JDBC drivers +- **[Java Integration](guides/java-integration.md)** - Call Perl from Java (JSR-223) +- **[Module Porting](guides/module-porting.md)** - Port Perl modules to PerlOnJava + +## Reference + +Technical reference documentation: + +- **[Feature Matrix](reference/feature-matrix.md)** - What Perl features are implemented +- **[CLI Options](reference/cli-options.md)** - Command-line reference +- **[Testing](reference/testing.md)** - Test suite information +- **[Architecture](reference/architecture.md)** - How PerlOnJava works + +## About + +Project information: + +- **[Why PerlOnJava?](about/why-perlonjava.md)** - Project goals and use cases +- **[Roadmap](about/roadmap.md)** - Future plans and upcoming features +- **[Changelog](about/changelog.md)** - Release history +- **[Support](about/support.md)** - Get help and contribute +- **[Resources](about/resources.md)** - External links and references + +## For Contributors + +Looking to contribute? See: + +- **[CONTRIBUTING.md](../CONTRIBUTING.md)** - Contribution guidelines +- **[dev/](../dev/)** - Developer documentation and internal architecture + +## Finding What You Need + +**"How do I...?"** → Start with [Guides](guides/) + +**"What features are supported?"** → See [Feature Matrix](reference/feature-matrix.md) + +**"How does it work?"** → Read [Architecture](reference/architecture.md) + +**"Where's the roadmap?"** → Check [Roadmap](about/roadmap.md) + +**"I want to contribute"** → Read [CONTRIBUTING.md](../CONTRIBUTING.md) diff --git a/docs/about/changelog.md b/docs/about/changelog.md new file mode 100644 index 000000000..d2c0c5ceb --- /dev/null +++ b/docs/about/changelog.md @@ -0,0 +1,303 @@ +# Changelog + +Release history of PerlOnJava. See [Roadmap](roadmap.md) for future plans. + + +- [Completed Milestones](#completed-milestones) +- [Work in progress](#work-in-progress) +- [Upcoming Milestones](#upcoming-milestones) +- [Future Development Areas](#future-development-areas) + +## Completed Milestones + +- **v5.42.2**: 250k Tests, Class Features, System V IPC, Sockets, and More + + - Add Perl 5.38+ Class Features + - Class keyword with block syntax fully working + - Method declarations with automatic $self injection + - Field declarations supporting all sigils ($, @, %) + - Constructor parameter fields with :param attribute + - Reader method generation with :reader attribute + - Automatic constructor generation with named parameters + - Default values for fields fully functional + - ADJUST blocks with field transformation working + - Field transformation to $self->{field} in methods + - Lexical method calls using $self->&priv syntax + - Class inheritance with :isa attribute working + - Version checking in :isa(Parent version) implemented + - Parent class field inheritance fully functional + - Object stringification shows OBJECT not HASH + - ClassRegistry tracks Perl 5.38+ class instances + - Context-aware reader methods for arrays/hashes + - Field transformation in string interpolation works + - __CLASS__ keyword with compile-time evaluation + - Add System V IPC operators: `msgctl`, `msgget`, `msgrcv`, `msgsnd`, `semctl`, `semget`, `semop`, `shmctl`, `shmget`, `shmread`, `shmwrite`. + - Add network enumeration operators: `endhostent`, `endnetent`, `endprotoent`, `endservent`, `gethostent`, `getnetbyaddr`, `getnetbyname`, `getnetent`, `getprotoent`, `getservent`, `sethostent`, `setnetent`, `setprotoent`, `setservent`. + - Add socket operators: `socket`, `bind`, `listen`, `accept`, `connect`, `send`, `recv`, `shutdown`, `setsockopt`, `getsockopt`, `getsockname`, `getpeername`, `socketpair`. + - Add Socket.pm module with socket constants and functions. + - Add `alarm` operator with `$SIG{ALRM}` signal handling. + - Fix `truncate` operator. + - Add `pipe` operator. + - Add `do \&subroutine`. + - Add `formline` operator and `$^A` accumulator variable + - Add file descriptor duplication support in `open` (`<&`, `>&`, `<&=`, `>&=`). + - Add statement: `format`, and `write` operator + - Add special variables: `@{^CAPTURE}`, `${^LAST_SUCCESSFUL_PATTERN}`. + - Add pack format `x`. + - Add `do filehandle`. + - Add module `Storable`, `experimental`, `Unicode::UCD`. + - Add single-quote as package separator. + - Dereferencing using `$$var{...}` and `$$var[...]` works. + - Add declared references: `my \$x`, `my(\@arr)`, `my(\%hash)`. + - Add subroutines declared `my`, `state`, or `our`. + - Bugfix in regex `/r`. + - Bugfix in transliterate with octal values. + - Bugfix in nested heredocs. + + +- **v5.42.1**: 150k Tests, Extended Operators, and More Perl 5 Features + + - Add operators: `getlogin`, `getpwnam`, `getpwuid`, `getgrnam`, `getgrgid`, `getpwent`, `getgrent`, `setpwent`, `setgrent`, `endpwent`, `endgrent`, `gethostbyname`, `gethostbyaddr`, `getservbyname`, `getservbyport`, `getprotobyname`, `getprotobynumber`, `reset`. + - Add overload operators: `<=>`, `cmp`, `<`, `<=`, `>`, `>=`, `==`, `!=`, `lt`, `le`, `gt`, `ge`, `eq`, `ne`, `qr`. + - Add command line switches: `-s`, `-f`. + - Add `__CLASS__` keyword. + - Add modules: `mro`, `version`, `List::Util`. + - Add more `sprintf` formatters. + - Add readline modes depending on `$/` special variable. + - Add `PERL5OPT` environment variable. + - Add regex extended character classes `(?[...])` + - Bugfix: fixed vstring with codepoints above 65535. + + +- **v5.42.0**: 100k Tests Passed, Tie Support, and Total Compatibility + - Add `tie`, `tied`, `untie` operators. + - Add all `tie` types: scalar, array, hash, and handle. + - Add operators: `sysread`, `syswrite`, `kill`, `utime`, `chown`, `waitpid`, `umask`, `readlink`, `link`, `symlink`, `rename`. + - Add modules: `XSLoader`, `Encode`,`Config`, `Errno`, `Tie::Scalar`, `Tie::Array`, `Tie::Hash`, `Tie::Handle`, `Perl::OSType`, `Env`, `MIME::Base64`, `MIME::QuotedPrint`, `Digest::SHA`, `Digest::MD5`, `Digest`. + - Add key-value slices: `%c{"1", "3"}`. + - Add special variable: `$^X`. + - Add `W`, `H`, `F`, `h`, `c`, `u`, `C0`, `U0` formats to `pack`, `unpack`. + - Add dualvar. + - Add `DATA` file handle. + - Add Indirect method call. + - Add regex variables: `${^PREMATCH}`, `${^MATCH}`, `${^POSTMATCH}`. + - Add regex operators: `\N` not-newline, `\b{gcb}`, `\B{gcb}` boundary assertions. + - Add regex properties supported by Perl but missing in Java regex. + - Add command line switches: `-w`, `-W`, `-X`. + - Process `\L`, `\U`, `\l`, `\u` in regex. + - `Test::More` `skip` works. + - UTF-16 is accepted in source code. + - Add support for `pmc` files. + - Bugfix: methods can be called in all blessed reference types. + - Bugfix: more robust `sprintf` formatting. + - Bugfix: string constants can be larger than 64k. + - Bugfix: fixed foreach loops with global variables. + + +- **v3.1.0**: Tracks Perl 5.42.0 + - Update Perl version to `5.42.0`. + - Added features: `keyword_all`, `keyword_any` + + - Accept input program in several ways: + 1. **Piped input**: `echo 'print "Hello\n"' | ./jperl` - reads from pipe and executes immediately + 2. **Interactive input**: `./jperl` - shows a prompt and waits for you to type code, then press Ctrl+D (on Unix/Linux/Mac) or Ctrl+Z (on Windows) to signal end of input + 3. **File redirection**: `./jperl < script.pl` - reads from the file + 4. **With arguments**: `./jperl -e 'print "Hello\n"'` or `./jperl script.pl` + + - Added overload operators: `!`, `+`, `-`, `*`, `/`, `%`, `int`, `neg`, `log`, `sqrt`, `cos`, `sin`, `exp`, `abs`, `atan2`, `**`, `@{}`, `%{}`. `${}`, `&{}`, `*{}`. + - Subroutine prototypes are fully implemented. Added or fixed: `+`, `;`, `*`, `\@`, `\%`, `\$`, `\[@%]`. + - Added double quoted string escapes: `\U`, `\L`, `\u`, `\l`. + - Added star count (`C*`) in `pack`, `unpack`. + - Added operators: `read`, `tell`, `seek`, `system`, `exec`, `sysopen`, `chmod`. + - Added operator: `select(undef,undef,undef,$time)`. + - Added operator: `^^=`. + - Added operator: `delete`, `exists` for array indexes. + - Added `open` option: in-memory files. + - Syntax: identifiers starting with `::` are in `main` package. + - Added I/O layers support to `open`, `binmode`: `:raw`, `:bytes`, `:crlf`, `:utf8`, `:unix`, `:encoding()`. + - Add `open` support for pipe `-|`, `|-`, `ls|`, `|sort`. + - Added `# line` preprocessor directive. + - `Test::More` module: added `subtest`, `use_ok`, `require_ok`. + - `CORE::` operators have the same prototypes as in Perl. + - Added modules: `Fcntl`, `Test`, `Text::CSV`. + - Operator `$#` returns an lvalue. + - Improved autovivification handling: distinguish between contexts where undefined references should automatically create data structures versus where they should throw errors. + - Bugfix: fix a problem with Windows newlines and qw(). Also fixed `mkdir` in Windows. + - Bugfix: `-E` switch was setting strict mode. + - BugFix: fix calling context in operators that return list. + - BugFix: fix rules for overriding operators. + - Added Makefile. + - Debian package can be created with `make deb`. + + +- **v3.0.0**: Performance Boost, New Modules, and Streamlined Configuration + - Added `--upgrade` option to `Configure.pl` to upgrade dependencies. + - Added `Dockerfile` configuration. + - Added `Time::HiRes`, `Benchmark` modules. + - Added `/ee` regex modifier. + - Added no strict `vars`, `subs`. + - Execute the code generation on demand, for faster module loading. + - Use `int` instead of `enum` to reduce the memory overhead of scalar variables. + + +- **v2.3.0**: Modern Perl Features, Expanded Modules, and Developer Tools + - Project description updated in `README.md` to "A Perl Distribution for the JVM" + - Added module porting guide at `docs/PORTING_MODULES.md` + - Added wrapper scripts (`jperl`/`jperl.bat`) for easier command-line usage + - Added `YAML` and `YAML::PP` modules. + - Added `Text::Balanced` module. + - Added `Unicode::Normalize` module. + - Added subroutine signatures and `signature` feature. + - Added chained operators. + - Added stacked file test operators. + - Added `module_true` feature. + - Added `<<` and `<<~` Here documents. + - Added `/p`, `/c`, `/n` regex modifiers. + - Added regex `(?^` clear embedded pattern-match modifier. + - Added regex `(?'name'...)` named capture groups. + - Added regex `\k` and `\g{name}` backreferences to named groups. + - Added regex `\p{...}` and `\P{...}` for Unicode properties. + - Added regex `\g{-n}` for relative backreferences. + - Added regex `*+`, `++`, `?+`, `{n,m}+` possessive quantifiers. + - Added regex `(?>...)` for atomic groups. + - Added overload: `""`, `0+`, `bool`, `fallback`, `nomethod`. + - Added `class` feature and `class` keyword. + - Library upgrades. + Maven: `mvn versions:use-latest-versions`. + Gradle: `./gradlew useLatestVersions`. + +- **v2.2.0**: Core modules + - Perl version is now v5.40.0 + - `for` loop can iterate over multiple values at the same time. + - `for` loop variables are aliased. + - Added `DBI` module with JDBC support. + - Added `URI::Escape` module. + - Added `builtin` methods: `inf` `nan` `weaken` `unweaken` `is_weak` `blessed` `refaddr` `reftype` `created_as_string` `created_as_number` `stringify` `ceil` `floor` `indexed` `trim` `is_tainted`. + - Added command line switches: `-S`. + - Added low-precedence xor `^^` operator. + - Added [Configure.pl](Configure.pl) to set compiler options and add JDBC drivers. + - Added Links to Perl on JVM resources in README - https://github.com/fglock/PerlOnJava/tree/master#additional-information-and-resources + - Added [SUPPORT.md](docs/SUPPORT.md) + +- **v2.1.0**: Core modules and optimization + - Added `Getopt::Long`, `JSON` modules. + - Optimized `print` to `STDOUT`/`STDERR` performance by running in a separate thread. + - Added `subs` pragma. + - Added regex `$+` variable. + - Added command line switches: `-v`, `-V` . + - Added file test operators: `-R`, `-W`, `-X`, `-O`, `-t`. + - Added feature flags: `evalbytes`. + - Added `CORE::GLOBAL` and core function overrides. + - Added hexadecimal floating point numbers. + +- **v2.0.0**: Towards a Complete Perl Port on the JVM + - Added unmodified core Perl modules `File::Basename`, `File::Find`, `Data::Dumper`, `Term::ANSIColor`, `Time::Local`, `HTTP::Date`, `HTTP::CookieJar`. + - Added `Cwd`, `File::Spec`, `File::Spec::Functions`, `HTTP::Tiny` modules. + - "use feature" implemented: `fc`, `say`, `current_sub`, `isa`, `state`, `try`, `bitwise`, `postderef`. + - Stash can be accessed as a hash like `$namespace::{entry}`. + - Added stash constants: `$constant::{_CAN_PCS} = \$const`; + - Added `exists &sub`, `defined &sub`. + - Added `builtin` pragma: `true`, `false`, `is_bool`. + - Added `re` pragma: `is_regexp`. + - Added `vars` pragma. + - Added `SUPER::method` method resolution. + - Added `AUTOLOAD` default subroutine. + - Added `stat`, `lstat` operators. Some fields are not available in JVM and return `undef`. + - Added directory operators. + - Added regex patterns: `[[:ascii:]]`, `[[:print:]]`, `(?#comment)`, and the `/xx` modifier. + +- **v1.11.0**: Compile-time Features + - Added `BEGIN`, `CHECK`, `UNITCHECK`, `INIT`, `END` blocks. + - Added subroutine hoisting: Invoking subroutines before their actual declaration in the code. + - Improved Exporter.pm, glob assignment. + - Added modules: `constant`, `if`, `lib`, `Internals` (`SvREADONLY`), `Carp`. + - Added `goto &name`; not a tail-call. + - Added `state` variables. + - Added `$SIG{ALRM}`, `${^GLOBAL_PHASE}`. + - Added operators: `fileno`, `getc`, `prototype`. + - Added `\N{U+hex}` operator in double quoted strings and regex. + +- **v1.10.0**: Operators and Special Variables + - Error messages mimic those in Perl for consistency. + - Added `$.`, `$]`, `$^V`, `${^LAST_FH}`, `$SIG{__DIE__}`, `$SIG{__WARN__}` special variables. + - Added command line switches `-E`, `-p`, `-n`, `-i`, `-0`, `-a`, `-F`, `-m`, `-M`, `-g`, `-l`, `-x`, `-?`. + - Added `select(filehandle)` operator, `ARGVOUT` filehandle. + - Added `~.`, `&.`, `|.`, `^.` operators. + - Added `try catch` statement. + - Added Scalar::Util: `blessed`, `reftype`. + - Added UNIVERSAL: `VERSION`. + - Added v-strings. + - Added Infinity, -Infinity, NaN. + - Added `\N{name}` operator for named characters in double quoted strings and in regex. + - Added lvalue subroutines. + - CI/CD runs in Ubuntu and Windows + +- **v1.9.0**: Operators and Special Variables + - Added bitwise string operators. + - Added lvalue `substr`, lvalue `vec` + - Fix `%b` specifier in `sprintf` + - Emulate Perl behaviour with unsigned integers in bitwise operators. + - Regex `m?pat?` match-once and the `reset()` operator are implemented. + - Regex `\G` and the `pos` operator are implemented. + - Regex `@-`, `@+`, `%+`, `%-` special variables are implemented. + - Regex `` $` ``, `$&`, `$'` special variables are implemented. + - Regex performance comparable to Perl; optimized regex variables. + - Regex matching plain strings: `$var =~ "Test"`. + - Added `__SUB__` keyword; `readpipe`. + - Added `&$sub` call syntax. + - Added `local` dynamic variables. + - Tests in `src/test/resources` are executed automatically. + +- **v1.8.0**: Operators + - Added `continue` blocks and loop operators `next`, `last`, `redo`; a bare-block is a loop + - Added bitwise operators `vec`, `pack`, `unpack` + - Added `srand`, `crypt`, `exit`, ellipsis statement (`...`) + - Added `readdir`, `opendir`, `closedir`, `telldir`, `seekdir`, `rewinddir`, `mkdir`, `rmdir` + - Added file test operators like `-d`, `-f` + - Added the variants of diamond operator `<>` and special cases of `while` + - Completed `chomp` operator; fixed `qw//` operator, `defined-or` and `x=` + - Added modules: `parent`, `Test::More` + +- **v1.7.0**: Performance Improvements + - Focus on optimizing the execution engine for better performance. + - Improve error handling and debugging tools to make development easier. More detailed debugging symbols added to the bytecode. Added `Carp` module. + - Moved Perl standard library modules into the jar file. + - More tests and various bug fixes + +- **v1.6.0**: Module System and Standard Library Enhancements + - Module system for improved code organization and reuse + - Core Perl module operators: `do FILE`, `require`, `caller`, `use`, `no` + - Module special subroutines: `import`, `unimport` + - Environment and special variables: `PERL5LIB`, `@INC`, `%INC`, `@ARGV`, `%ENV`, `$0`, `$` + - Additional operators: `die`, `warn`, `time`, `times`, `localtime`, `gmtime`, `index`, `rindex` + - Standard library ported modules: `Data::Dumper`, `Symbol`, `strict` + - Expanded documentation and usage examples + +- **v1.5.0**: Regex operators + - Added Regular expressions and pattern matching: m//, pos, qr//, quotemeta, s///, split + - More complete set of operations on strings, numbers, arrays, hashes, lists + - More special variables + - More tests and various bug fixes + +- **v1.4.0**: I/O operators + - File i/o operators, STDOUT, STDERR, STDIN + - TAP (Perl standard) tests + +- **v1.3.0**: Added Objects. + - Objects and object operators, UNIVERSAL class + - Array and List related operators + - More tests and various bug fixes + +- **v1.2.0**: Added Namespaces and named subroutines. + - Added typeglobs + - Added more operators + +- **v1.1.0**: Established architecture and added key features. The system now supports benchmarks and tests. + - JSR 223 integration + - Support for closures + - Eval-string functionality + - Enhanced statements, data types, and call context + +- **v1.0.0**: Initial proof of concept for the parser and execution engine. + + diff --git a/docs/RELATION_WITH_PERLITO_COMPILER.md b/docs/about/relation-perlito.md similarity index 100% rename from docs/RELATION_WITH_PERLITO_COMPILER.md rename to docs/about/relation-perlito.md diff --git a/docs/RESOURCES.md b/docs/about/resources.md similarity index 80% rename from docs/RESOURCES.md rename to docs/about/resources.md index a3241e954..7bd182d3b 100644 --- a/docs/RESOURCES.md +++ b/docs/about/resources.md @@ -22,13 +22,12 @@ ### Related Projects - [Perlito5 Compiler](https://github.com/fglock/Perlito): Perl to Java and JavaScript compiler -- See [RELATION_WITH_PERLITO_COMPILER.md](RELATION_WITH_PERLITO_COMPILER.md) for details on how Perlito5 influenced and complements PerlOnJava ### Advanced Topics -- [GraalVM Native Image Support Investigation](GRAALVM.md): Technical exploration of GraalVM integration -- [Feature Matrix](FEATURE_MATRIX.md): Detailed implementation status -- [Architecture Overview](ARCHITECTURE.md): System design and components -- [Porting Guide](PORTING_MODULES.md): Guidelines for porting Perl modules +- [GraalVM Native Image Support Investigation](../../dev/design/graalvm.md): Technical exploration of GraalVM integration +- [Feature Matrix](../reference/feature-matrix.md): Detailed implementation status +- [Architecture Overview](../reference/architecture.md): System design and components +- [Porting Guide](../guides/module-porting.md): Guidelines for porting Perl modules ## Community Resources - [GitHub Issues](https://github.com/fglock/PerlOnJava/issues) diff --git a/docs/about/roadmap.md b/docs/about/roadmap.md new file mode 100644 index 000000000..ad75d0184 --- /dev/null +++ b/docs/about/roadmap.md @@ -0,0 +1,225 @@ +# Roadmap + +Future plans for PerlOnJava. See [Changelog](changelog.md) for release history. + +## Work in Progress + +The following areas are currently under active development to enhance the functionality and performance of PerlOnJava: + +- **Compiler Subsystem** + - Implementing the `lexical_subs` feature. + - Enhancing version control with `use VERSION` and `require VERSION`. + - Introducing lexical warnings and strictness for improved code safety. + - Supporting lexical UTF-8 source code. + - Ensure that stack traces are well-formed. + - Developing subroutine prototypes. + - Handling `goto` special cases for `Text::Balanced`. + - Addressing indirect object special cases for `GetOpt::Long`. + - Localizing regex variables. + - Fix handling of global variable aliasing in `for`. + +- **Regex Subsystem** + - Ongoing improvements and feature additions. + +- **Class Subsystem** + - Enhancements and new feature implementations. + +- **DBI Subsystem** + - Adding additional methods for database interaction. + +- **Overload Subsystem** + - Expanding with additional methods. + +- **I/O Subsystem** + - Implementing in-memory I/O operations. + - Developing `socket` and related operators. + - Enhancing `truncate`, `seek`, and `read` operators. + - Introducing IO layers and experimenting with memory-mapped I/O. + +- **Threads Subsystem** + - Documenting preliminary features. + - Planning for fork emulation and interpreter cloning. + - Developing `multiplicity` support. + - Adjusting `exit` semantics to exit the current thread only. + - Adding `ThreadLocal` to global variables, special variables, and caches, with tear down hooks for cleanup. + +- **CPAN Support** + - Porting `cpan` and `prove`. + - Adding module testing for PerlOnJava core modules. + +- **Optimization** + - Inlining `map` and related blocks. + - Inlining constant subroutines. + - Prefetch named subroutines to lexical (`our`). + +- **Compilation with GraalVM** + - Documenting preliminary results in [docs/GRAALVM.md](docs/GRAALVM.md). + + +## Upcoming Milestones + +- **v5.42.3**: Next minor version + + - Non-local control flow: `last`/`next`/`redo`/`goto LABEL` + - Tail call with trampoline for `goto &NAME` and `goto __SUB__` + - Add modules: `TOML`. + - Bugfix: operator override in Time::Hires now works. + - Bugfix: internal temp variables are now pre-initialized. + - Optimization: faster list assignment. + - Optimization: faster type resolution in Perl scalars. + - Optimization: `make` now runs tests in parallel. + - Optimization: A workaround is implemented to Java 64k bytes segment limit. + - Planned release date: 2026-02-10. + +- Work in Progress + - PerlIO + - `get_layers` + - Term::ReadLine + - Term::ReadKey + - FileHandle + - File::Temp + - File::Path + - File::Copy + - IO::File + - IO::Handle + - `ungetc` + - Auto-bless filehandle into IO::Handle subclass + - IO::Seekable + - Filter::Simple + - Math::BigInt + - Text::ParseWords + - Text::Tabs + - Locale::Maketext::Simple + - Params::Check + - SelectSaver + - locale pragma + - utf8 pragma + - bytes pragma + - threads pragma + - warnings pragma + - vmsish pragma + - Constant folding - in ConstantFoldingVisitor.java + - `method` keyword + - Overload operators: `++`, `--`. + - String interpolation fixes. + - Command line option `-C` + +### v4.0.0 Milestone (Planned Release Date: 2026-05-10) + +**Objective:** Enhance core functionality and improve developer experience with a focus on integration and performance. + +1. **Concurrency and Security Enhancements** + - **Specific:** Implement basic concurrency support with threads and async/await capabilities. + - **Measurable:** Ensure at least 50% of core modules support concurrent execution. + - **Achievable:** Utilize existing JVM concurrency features. + - **Relevant:** Addresses modern application requirements for parallel processing. + - **Time-bound:** Complete by Q1 2026. + +2. **External Integration** + - **Specific:** Integrate with popular external libraries for HTTP requests and database access. + - **Measurable:** Provide at least two integration examples in documentation. + - **Achievable:** Leverage existing Java libraries for integration. + - **Relevant:** Enhances PerlOnJava's utility in web and data applications. + - **Time-bound:** Complete by Q1 2026. + +3. **Development Tools** + - **Specific:** Develop an interactive debugger with breakpoints and variable inspection. + - **Measurable:** Debugger should support at least 80% of core language features. + - **Achievable:** Build on existing JVM debugging capabilities. + - **Relevant:** Improves developer productivity and code quality. + - **Time-bound:** Complete by Q1 2026. + +4. **Distribution and Packaging** + - **Specific:** Provide Docker containers and Kubernetes configurations for easy deployment. + - **Measurable:** Ensure compatibility with at least two major cloud platforms. + - **Achievable:** Use standard containerization practices. + - **Relevant:** Facilitates deployment in modern cloud environments. + - **Time-bound:** Complete by Q1 2026. + +### v5.0.0 Milestone (Planned Release Date: 2027-04-10) + +**Objective:** Expand platform capabilities and improve performance with advanced features and optimizations. + +1. **GraalVM Integration** + - **Specific:** Implement native image compilation for instant startup and reduced memory footprint. + - **Measurable:** Achieve at least a 30% reduction in startup time and memory usage. + - **Achievable:** Collaborate with GraalVM community for support. + - **Relevant:** Enhances performance for cloud and serverless applications. + - **Time-bound:** Complete by Q1 2027. + +2. **Mobile Development** + - **Specific:** Develop an Android app with native UI for Perl scripting. + - **Measurable:** Ensure app compatibility with at least 80% of Android devices. + - **Achievable:** Utilize existing Android development frameworks. + - **Relevant:** Expands PerlOnJava's reach to mobile developers. + - **Time-bound:** Complete by Q1 2027. + +3. **Advanced Data Manipulation** + - **Specific:** Add XML parsing and data transformation features. + - **Measurable:** Provide at least three example scripts demonstrating these capabilities. + - **Achievable:** Use existing Java libraries for data manipulation. + - **Relevant:** Increases PerlOnJava's applicability in data-driven applications. + - **Time-bound:** Complete by Q1 2027. + +4. **Enterprise Features** + - **Specific:** Integrate with logging frameworks and provide monitoring with Prometheus/Grafana. + - **Measurable:** Ensure logging and monitoring support for at least 70% of core modules. + - **Achievable:** Leverage existing enterprise tools and frameworks. + - **Relevant:** Meets enterprise requirements for observability and security. + - **Time-bound:** Complete by Q1 2027. + + +## Future Development Areas + +- **Concurrency and Security Features** + - Add support for concurrency and parallelism, such as threads and async/await. + - Enhance security features, including sandboxing and input validation. + - Increase test coverage. + +- **External Integration and Advanced Data Manipulation** + - Integrate with external libraries and APIs for tasks like HTTP requests and database access. + - Add advanced data manipulation features, such as JSON/XML parsing and data transformation. + - Allow users to define their own operators and macros for greater flexibility. + +- **Mobile Development** + - Android app with native UI for Perl scripting + - Integration with Android's file system and permissions + - Mobile-optimized performance settings + +- **Distribution and Packaging** + - Native installers for Windows/Linux/MacOS + - Docker containers and Kubernetes configurations + - Maven Central Repository publishing + - Integration with package managers (apt, yum, chocolatey, homebrew) + +- **Development Tools** + - IDE plugins for IntelliJ IDEA, Eclipse, VSCode + - Interactive debugger with breakpoints and variable inspection + - Code formatting and static analysis tools + - Performance profiling tools + +- **Enterprise Features** + - Monitoring and metrics with Prometheus/Grafana + - Integration with logging frameworks (Log4j, SLF4J) + - Security hardening and vulnerability scanning + - Cloud deployment templates (AWS, Azure, GCP) + +- **GraalVM Integration** + - Native image compilation for instant startup + - Polyglot integration with JavaScript, Python, Ruby + - Ahead-of-time compilation optimizations + - Reduced memory footprint for cloud deployments + - SubstrateVM support for containerized environments + +- **Native System Integration** + - JNI/XS bridge to enable native code modules + - Integration with existing Perl XS ecosystem + - Core system operations + - Process control (fork, exec) + - Direct memory access (mmap) + - Native sockets and file descriptors + - Platform-specific features + - Unix: ptrace, signals, IPC + - Windows: Registry, COM objects + - MacOS: FSEvents, CoreFoundation, Security framework + diff --git a/docs/SUPPORT.md b/docs/about/support.md similarity index 91% rename from docs/SUPPORT.md rename to docs/about/support.md index f4129a5e8..a72554c1f 100644 --- a/docs/SUPPORT.md +++ b/docs/about/support.md @@ -3,9 +3,10 @@ ## Getting Help ### Documentation -- [README.md](../README.md) - Project overview and setup instructions -- [FEATURE_MATRIX.md](FEATURE_MATRIX.md) - Detailed feature compatibility list -- [MILESTONES.md](../MILESTONES.md) - Project roadmap and version history +- [README.md](../../README.md) - Project overview and setup instructions +- [Feature Matrix](../reference/feature-matrix.md) - Detailed feature compatibility list +- [Roadmap](roadmap.md) - Future plans +- [Changelog](changelog.md) - Release history ### Community Resources - GitHub Issues: Technical questions and bug reports @@ -55,7 +56,7 @@ PerlOnJava follows semantic versioning (MAJOR.MINOR.PATCH): ### Pull Request Process 1. Update documentation 2. Add/update tests -3. Update FEATURE_MATRIX.md if needed +3. Update Feature Matrix if needed 4. Request review from maintainers ### Issue Reporting diff --git a/docs/WHY_PERLONJAVA.md b/docs/about/why-perlonjava.md similarity index 93% rename from docs/WHY_PERLONJAVA.md rename to docs/about/why-perlonjava.md index 8f4f97579..8ec0cfd66 100644 --- a/docs/WHY_PERLONJAVA.md +++ b/docs/about/why-perlonjava.md @@ -14,8 +14,8 @@ PerlOnJava offers a unique solution for JVM-based environments or use cases wher 3. **Modular Architecture**: PerlOnJava’s architecture facilitates integration with other JVM-based languages (e.g., Kotlin, Scala) and provides tools like JDBC for database interaction, making it ideal for enterprise environments. -4. **Advanced Performance Features**: - The project uses modern Java and ASM techniques to optimize execution. Features like multithreaded I/O can offer performance benefits in specific use cases compared to traditional Perl. +4. **JVM Performance Optimizations**: + The project uses modern Java and ASM techniques to optimize execution. The JVM's just-in-time (JIT) compilation and garbage collection can provide performance benefits for compute-intensive tasks. 5. **Educational and Experimental Value**: PerlOnJava is a compelling resource for understanding compiler design, language interoperability, and the translation of high-level languages to bytecode. diff --git a/docs/getting-started/docker.md b/docs/getting-started/docker.md new file mode 100644 index 000000000..b9bb1eae8 --- /dev/null +++ b/docs/getting-started/docker.md @@ -0,0 +1,100 @@ +# Docker Guide + +Run PerlOnJava in a Docker container for isolated and portable execution. + +## Prerequisites + +- [Docker](https://www.docker.com/products/docker-desktop) installed on your machine + +## Quick Start + +### Build the Docker Image + +```bash +docker build -t perlonjava:latest . +``` + +This creates a Docker image named `perlonjava` with the `latest` tag. + +### Run Perl Code + +```bash +# Simple one-liner +docker run --rm perlonjava:latest -E 'say "Hello from Docker!"' + +# Run a script file (mount current directory) +docker run --rm -v "$(pwd)":/scripts perlonjava:latest /scripts/myscript.pl + +# Interactive shell +docker run --rm -it perlonjava:latest -de0 +``` + +### Common Options + +**`--rm`**: Automatically remove the container when it exits + +**`-v /path/to/scripts:/scripts`**: Mount local directory for script access + +**`-e VAR=value`**: Set environment variables + +**`--name myname`**: Assign a name to the container for easier management + +## Examples + +### Running a Test File + +```bash +# Copy test file to local directory +cp src/test/resources/unit/array.t . + +# Run test in Docker +docker run --rm -v "$(pwd)":/test perlonjava:latest /test/array.t +``` + +### With JDBC Drivers + +To use JDBC drivers, add them to the Dockerfile or mount them at runtime: + +```bash +# Mount driver and use it +docker run --rm \ + -v /path/to/mysql-connector.jar:/drivers/mysql.jar \ + -e CLASSPATH=/drivers/mysql.jar \ + perlonjava:latest your_script.pl +``` + +### Persistent Containers + +If you need to keep a container running: + +```bash +# Start named container +docker run --name perlonjava_app -d perlonjava:latest -E 'sleep infinity' + +# Execute commands in running container +docker exec perlonjava_app jperl -E 'say "Hello"' + +# Stop and remove +docker stop perlonjava_app +docker rm perlonjava_app +``` + +## Customizing the Dockerfile + +Modify the `Dockerfile` to include additional dependencies: + +```dockerfile +# Add JDBC driver +FROM eclipse-temurin:21-jdk +COPY --from=build /app/target/perlonjava-3.0.0.jar /app/perlonjava.jar +COPY path/to/driver.jar /app/drivers/ +ENV CLASSPATH=/app/drivers/driver.jar +ENTRYPOINT ["java", "-jar", "/app/perlonjava.jar"] +``` + +## See Also + +- [Installation Guide](installation.md) - Build without Docker +- [Database Access](../guides/database-access.md) - Using JDBC drivers + + diff --git a/docs/getting-started/installation.md b/docs/getting-started/installation.md new file mode 100644 index 000000000..1b0edb4e5 --- /dev/null +++ b/docs/getting-started/installation.md @@ -0,0 +1,213 @@ +# Build and Execution Guide + +## Table of Contents +1. [Prerequisites](#prerequisites) +2. [Build Options](#build-options) + - [Using Make](#using-make) + - [Using Maven](#using-maven) + - [Using Gradle](#using-gradle) +3. [Package Installation](#package-installation) + - [Debian Package](#debian-package) +4. [Dependencies](#dependencies) +5. [Running PerlOnJava](#running-perlonjava) + - [Platform-Specific Instructions](#platform-specific-instructions) + - [Common Options](#common-options) +6. [Database Integration](#database-integration) + - [Adding JDBC Drivers](#adding-jdbc-drivers) + - [Database Connection Example](#database-connection-example) +7. [Build Notes](#build-notes) +8. [Java Library Upgrades](#java-library-upgrades) +9. [Using Configure.pl](#using-configurepl) + - [Common Tasks](#common-tasks) + - [Available Options](#available-options) + - [Important Notes](#important-notes) + +## Prerequisites +- Java 21 or higher +- Maven or Gradle +- Optional: JDBC drivers for database connectivity + +## Build Options + +### Using Make +The project includes a Makefile that wraps Gradle commands for a familiar build experience: + +```bash +make # same as 'make build' +make build # builds the project and runs unit tests +make dev # force clean rebuild (use during active development) +make test # runs fast unit tests +make clean # cleans build artifacts +make deb # creates a Debian package (Linux only) +``` + +### Using Maven +```bash +mvn clean package +``` + +### Using Gradle +```bash +./gradlew clean build +``` + +## Package Installation + +### Debian Package + +For Debian-based systems (Ubuntu, Debian, Mint, etc.), you can create and install a `.deb` package: + +**Build the package:** +```bash +make deb +``` + +This creates a Debian package in `build/distributions/` with: +- PerlOnJava JAR installed to `/usr/share/perlonjava/` +- `jperl` command installed to `/usr/bin/` +- All dependencies bundled +- Systemwide availability + +**Install the package:** +```bash +sudo dpkg -i build/distributions/perlonjava_*.deb +``` + +**Usage after installation:** +```bash +# jperl is now available systemwide +jperl -E 'say "Hello World"' +jperl myscript.pl + +# No need for ./jperl - it's in your PATH +``` + +**Uninstall:** +```bash +sudo dpkg -r perlonjava +``` + +**Benefits of Debian package:** +- Clean installation and removal +- Systemwide availability (no need for `./jperl`) +- Automatic dependency tracking +- Integrates with system package manager +- Can be distributed to other Debian-based systems + +## Dependencies +- JUnit: For testing +- ASM: For bytecode manipulation +- ICU4J: For Unicode support +- FASTJSON v2: For JSON support +- SnakeYAML Engine: for YAML support + +## Running PerlOnJava + +### Platform-Specific Instructions + +**Unix/Linux/Mac:** +```bash +./jperl -E 'print "Hello World"' +./jperl myscript.pl +CLASSPATH="jdbc-drivers/h2-2.2.224.jar" ./jperl myscript.pl +``` + +**Windows:** +```bash +jperl -E "print 'Hello World'" +jperl myscript.pl +set CLASSPATH=jdbc-drivers\h2-2.2.224.jar +jperl myscript.pl +``` + +### Common Options +- `-I lib`: Add library path +- `--debug`: Enable debug output +- `--help`: Show all options + +## Database Integration + +### Adding JDBC Drivers + +1. Using Configure.pl: +```bash +./Configure.pl --search mysql-connector-java +``` + +2. Using Java classpath (shown in platform-specific examples above) + +### Database Connection Example +```perl +use DBI; +my $dbh = DBI->connect("jdbc:h2:mem:testdb;DB_CLOSE_DELAY=-1"); +$dbh->do("CREATE TABLE test (id INT, name VARCHAR(50))"); +``` + +See [Database Access Guide](../guides/database-access.md) for detailed connection examples and supported databases. + +## Build Notes +- Maven builds use `maven-shade-plugin` for creating the shaded JAR +- Gradle builds use `com.github.johnrengelman.shadow` plugin +- Both configurations target Java 21 + +## Java Library Upgrades + +**Maven:** + +`mvn versions:use-latest-versions`. + +**Gradle:** + +`./gradlew useLatestVersions`. + +## Using Configure.pl + +The `Configure.pl` script manages configuration settings and dependencies for PerlOnJava. + +### Common Tasks + +**View current configuration:** +```bash +./Configure.pl +``` + +**Add JDBC driver (search):** +```bash +./Configure.pl --search mysql +make # Rebuild to include driver +``` + +**Add JDBC driver (direct):** +```bash +./Configure.pl --direct com.h2database:h2:2.2.224 +make # Rebuild to include driver +``` + +**Update configuration:** +```bash +./Configure.pl -D perlVersion=v5.42.0 +``` + +**Upgrade all dependencies:** +```bash +./Configure.pl --upgrade +``` + +### Available Options + +- **`-h, --help`** - Show help message +- **`-D key=value`** - Set configuration value +- **`--search keyword`** - Search Maven Central for artifacts +- **`--direct group:artifact:version`** - Add dependency with Maven coordinates +- **`--verbose`** - Enable verbose output +- **`--upgrade`** - Upgrade dependencies to latest versions + +### Important Notes + +1. **Rebuild required**: After adding dependencies with `--search` or `--direct`, you must run `make` to download and bundle them +2. **Alternative approach**: Instead of bundling drivers, you can use CLASSPATH: + ```bash + CLASSPATH=/path/to/driver.jar ./jperl script.pl + ``` + +**→ See [Configure.pl Reference](../reference/configure.md) for complete documentation** diff --git a/docs/ONELINERS.md b/docs/getting-started/oneliners.md similarity index 80% rename from docs/ONELINERS.md rename to docs/getting-started/oneliners.md index a78f401b8..49d778350 100644 --- a/docs/ONELINERS.md +++ b/docs/getting-started/oneliners.md @@ -33,7 +33,12 @@ Write and read YAML file: Write and read JSON file: ```bash -./jperl -MJSON -e 'my $json = JSON->new; DumpFile("test.json", { hello => "world" }); print encode_json(from_json(LoadFile("test.json")))' +./jperl -MJSON -e 'use JSON; my $data = { hello => "world" }; open my $fh, ">", "test.json"; print $fh encode_json($data); close $fh; open my $in, "<", "test.json"; my $json = do { local $/; <$in> }; print encode_json(decode_json($json))' +``` + +Or more simply: +```bash +./jperl -MJSON -E 'my $data = { hello => "world" }; say encode_json($data)' ``` ## Benchmark diff --git a/docs/JDBC_GUIDE.md b/docs/guides/database-access.md similarity index 98% rename from docs/JDBC_GUIDE.md rename to docs/guides/database-access.md index 3a37c8891..d2df23c6b 100644 --- a/docs/JDBC_GUIDE.md +++ b/docs/guides/database-access.md @@ -260,5 +260,5 @@ for my $id (@ids) { ## See Also -- [FEATURE_MATRIX.md](FEATURE_MATRIX.md) for complete feature list +- [Feature Matrix](../reference/feature-matrix.md) for complete feature list - [DBI documentation](https://metacpan.org/pod/DBI) for standard DBI interface diff --git a/docs/guides/java-integration.md b/docs/guides/java-integration.md new file mode 100644 index 000000000..881774580 --- /dev/null +++ b/docs/guides/java-integration.md @@ -0,0 +1,193 @@ +# Java Integration Guide + +How to use PerlOnJava from Java applications. + +## JSR-223 Scripting API + +PerlOnJava implements the Java Scripting API (JSR-223), allowing you to execute Perl code from Java. + +### Basic Usage + +```java +import javax.script.*; + +public class PerlExample { + public static void main(String[] args) throws Exception { + // Get Perl engine + ScriptEngineManager manager = new ScriptEngineManager(); + ScriptEngine engine = manager.getEngineByName("perl"); + + // Execute Perl code + engine.eval("print 'Hello from Perl!\\n'"); + } +} +``` + +### Passing Variables + +```java +ScriptEngine engine = manager.getEngineByName("perl"); + +// Set variables +engine.put("name", "World"); +engine.put("count", 42); + +// Access in Perl +engine.eval("say \"Hello, $name! Count: $count\""); +``` + +### Getting Results + +```java +// Evaluate and get result +Object result = engine.eval("2 + 2"); +System.out.println("Result: " + result); // Output: Result: 4 + +// Use variables +engine.put("x", 10); +engine.put("y", 20); +Object sum = engine.eval("$x + $y"); +System.out.println("Sum: " + sum); // Output: Sum: 30 +``` + +### Using Perl Subroutines + +```java +ScriptEngine engine = manager.getEngineByName("perl"); + +// Define subroutine +engine.eval("sub multiply { my ($a, $b) = @_; return $a * $b; }"); + +// Call it +engine.put("x", 5); +engine.put("y", 7); +Object result = engine.eval("multiply($x, $y)"); +System.out.println("Result: " + result); // Output: Result: 35 +``` + +### Error Handling + +```java +ScriptEngine engine = manager.getEngineByName("perl"); + +try { + engine.eval("die 'Something went wrong';"); +} catch (ScriptException e) { + System.err.println("Perl error: " + e.getMessage()); +} +``` + +## Using PerlOnJava Directly + +For more control, you can use PerlOnJava's internal API directly. + +### Compiling Perl Code + +```java +import org.perlonjava.runtime.PerlCompiler; + +public class DirectExample { + public static void main(String[] args) { + PerlCompiler compiler = new PerlCompiler(); + compiler.compile("say 'Hello World';"); + compiler.run(); + } +} +``` + +## Building with PerlOnJava + +### Maven + +Add PerlOnJava as a dependency: + +```xml + + org.perlonjava + perlonjava + 5.42.2 + +``` + +### Gradle + +```gradle +dependencies { + implementation 'org.perlonjava:perlonjava:5.42.2' +} +``` + +### Manual JAR + +1. Build PerlOnJava: + ```bash + make + ``` + +2. Find JAR in `build/libs/perlonjava-*-all.jar` + +3. Add to your classpath: + ```bash + javac -cp perlonjava-5.42.2-all.jar YourApp.java + java -cp .:perlonjava-5.42.2-all.jar YourApp + ``` + +## Use Cases + +### Configuration Scripts + +Use Perl for flexible configuration: + +```java +ScriptEngine engine = manager.getEngineByName("perl"); +engine.eval(Files.readString(Path.of("config.pl"))); +Object config = engine.get("config"); +``` + +### Data Processing + +```java +ScriptEngine engine = manager.getEngineByName("perl"); + +// Process CSV with Perl +engine.eval(""" + use Text::CSV; + my $csv = Text::CSV->new(); + # ... process CSV data + """); +``` + +### Legacy Perl Integration + +Run existing Perl scripts from Java: + +```java +ScriptEngine engine = manager.getEngineByName("perl"); +String perlScript = Files.readString(Path.of("legacy.pl")); +engine.eval(perlScript); +``` + +## Examples + +See also: +- [Quick Start](../../QUICKSTART.md) - Basic examples +- [Architecture](../reference/architecture.md) - How it works internally + +## Troubleshooting + +### Engine Not Found + +If `getEngineByName("perl")` returns null: +1. Ensure `perlonjava-*.jar` is in classpath +2. Check `META-INF/services/javax.script.ScriptEngineFactory` exists in JAR +3. Verify Java 21 or later is being used + +### ClassNotFoundException + +Make sure the PerlOnJava JAR is in your classpath when compiling and running. + +### Performance + +- First execution may be slow (JIT compilation) +- Subsequent executions are faster +- Consider caching compiled scripts diff --git a/docs/PORTING_MODULES.md b/docs/guides/module-porting.md similarity index 50% rename from docs/PORTING_MODULES.md rename to docs/guides/module-porting.md index 6c1a13427..53825c3da 100644 --- a/docs/PORTING_MODULES.md +++ b/docs/guides/module-porting.md @@ -4,11 +4,11 @@ PerlOnJava supports three types of Perl modules: -1. Pure Perl modules (.pm files) -2. Java-implemented modules (replacing XS/C modules) -3. Hybrid modules (combining Perl and Java implementations) +1. **Pure Perl modules** (.pm files) - No Java code needed +2. **Java-implemented modules** (via XSLoader) - Perl modules that load Java implementations, replacing XS/C modules +3. **Built-in modules** (in GlobalContext) - Internal PerlOnJava modules available at startup (e.g., UNIVERSAL) -A hybrid module typically consists of a .pm file containing Perl code and a corresponding Java class that implements performance-critical or system-level functionality. This approach lets you leverage the best of both languages - Perl's expressiveness and Java's performance. +**Most CPAN module ports should use type #2 (XSLoader).** Type #3 is only for internal PerlOnJava functionality. ## Directory Structure @@ -27,59 +27,107 @@ sub import { shift; unshift @_, 1; goto &work } sub unimport { shift; unshift @_, 0; goto &work } ``` -## Java-Implemented Modules +## Java-Implemented Modules (via XSLoader) -Java implementations replace Perl XS modules. They extend `PerlModuleBase` and implement the module's functionality in Java. +Java implementations replace Perl XS modules. They extend `PerlModuleBase` and are loaded via `XSLoader::load()`. -Example from DBI module: +### Naming Convention + +XSLoader maps Perl module names to Java class names: + +- **Perl module**: `DBI` → **Java class**: `org.perlonjava.perlmodule.Dbi` +- **Perl module**: `Text::CSV` → **Java class**: `org.perlonjava.perlmodule.Text_CSV` +- **Perl module**: `My::Module` → **Java class**: `org.perlonjava.perlmodule.My_Module` + +Rules: +- Package: Always `org.perlonjava.perlmodule` +- Class name: Perl module name with `::` replaced by `_` +- First letter capitalized (Java convention) + +### Basic Structure ```java +package org.perlonjava.perlmodule; + public class Dbi extends PerlModuleBase { public Dbi() { super("DBI", false); } + // Called by XSLoader::load('DBI') public static void initialize() { Dbi dbi = new Dbi(); dbi.registerMethod("connect", null); dbi.registerMethod("prepare", null); // Register other methods... } + + // Implement methods + public static RuntimeList connect(RuntimeArray args, int ctx) { + // Implementation... + } } ``` -## Module Loading API +## Using Java-Implemented Modules + +### From Perl Code (XSLoader) + +In your Perl module, load the Java implementation: -### Pure Perl Module Loading ```perl -use ModuleName; -require "ModuleName.pm"; -``` +package My::Module; +use strict; +use warnings; -### Java Module Registration +our $VERSION = '1.00'; -1. Extend PerlModuleBase: -```java -public class MyModule extends PerlModuleBase { - public MyModule() { - super("My::Module"); - } +# Load Java implementation +require XSLoader; +XSLoader::load('My::Module', $VERSION); + +# Pure Perl methods can call Java methods +sub helper_method { + my ($self, @args) = @_; + return $self->java_implemented_method(@args); } + +1; ``` -2. Register methods: +### From User Code + +Users just use the module normally: + +```perl +use My::Module; + +my $obj = My::Module->new(); +$obj->method(); +``` + +The XSLoader mechanism is completely transparent to end users. + +## Implementing Java Module Methods + +### Method Registration + +In your Java class's `initialize()` method, register all methods: + ```java -protected void initialize() { - registerMethod("method_name", null); - registerMethod("perl_name", "java_name", null); +public static void initialize() { + MyModule module = new MyModule(); + module.registerMethod("method_name", null); + module.registerMethod("perl_name", "java_method_name", null); } ``` -3. Define exports: +### Defining Exports + ```java -defineExport("EXPORT", "function1", "function2"); -defineExport("EXPORT_OK", "optional_function"); -defineExportTag("group", "function1", "function2"); +module.defineExport("EXPORT", "function1", "function2"); +module.defineExport("EXPORT_OK", "optional_function"); +module.defineExportTag("group", "function1", "function2"); ``` ## Calling Conventions @@ -103,34 +151,75 @@ public static RuntimeList method_name(RuntimeArray args, int ctx) { - For list context: return multi-element list - For void context: return empty list -## Module Registration in GlobalContext +## Module Registration + +There are two ways to register Java-implemented modules: -After implementing a module, register it in `GlobalContext.java`: +### 1. Built-in/Internal Modules (GlobalContext) + +**Only for internal PerlOnJava modules** that need to be available immediately at startup (e.g., UNIVERSAL, CORE functions). + +Register in `GlobalContext.java`: ```java // Initialize built-in Perl classes DiamondIO.initialize(compilerOptions); Universal.initialize(); -MyNewModule.initialize(); // Add your module here ``` -This step ensures your module is initialized during PerlOnJava startup alongside other core modules. +**Do not use this approach for regular CPAN-style modules.** -The initialization sequence handles: -- Method registration -- Export definitions -- Global variable setup -- Module state initialization +### 2. Regular Modules (XSLoader) + +**This is the standard approach for porting modules.** Use XSLoader in your Perl module: + +```perl +package DBI; +use strict; +use warnings; + +our $VERSION = '1.643'; + +# Load Java implementation +require XSLoader; +XSLoader::load('DBI', $VERSION); + +# Pure Perl methods +sub do { + my ($dbh, $statement, $attr, @params) = @_; + my $sth = $dbh->prepare($statement, $attr) or return undef; + $sth->execute(@params) or return undef; + my $rows = $sth->rows; + ($rows == 0) ? "0E0" : $rows; +} + +1; +``` + +When `XSLoader::load('DBI')` is called: +1. XSLoader looks for the Java class `org.perlonjava.perlmodule.Dbi` +2. Calls the static `initialize()` method +3. Registers all methods defined in the Java class + +This is transparent to users - they just `use DBI` and it works. ## Real-World Example: DBI Module -The DBI module demonstrates a complete port: +The DBI module demonstrates a complete port using XSLoader: -1. Pure Perl portion (`DBI.pm`): +1. **Perl module** (`DBI.pm`): ```perl package DBI; use strict; +use warnings; + +our $VERSION = '1.643'; +# Load Java implementation +require XSLoader; +XSLoader::load('DBI', $VERSION); + +# Pure Perl helper method sub do { my ($dbh, $statement, $attr, @params) = @_; my $sth = $dbh->prepare($statement, $attr) or return undef; @@ -138,21 +227,43 @@ sub do { my $rows = $sth->rows; ($rows == 0) ? "0E0" : $rows; } + +1; ``` -2. Java implementation (`Dbi.java`): +2. **Java implementation** (`org/perlonjava/perlmodule/Dbi.java`): ```java public class Dbi extends PerlModuleBase { + public Dbi() { + super("DBI", false); + } + + // Called by XSLoader + public static void initialize() { + Dbi dbi = new Dbi(); + dbi.registerMethod("connect", null); + dbi.registerMethod("prepare", null); + dbi.registerMethod("execute", null); + // ... register other methods + } + + // Implementation of connect method public static RuntimeList connect(RuntimeArray args, int ctx) { RuntimeHash dbh = new RuntimeHash(); String jdbcUrl = args.get(1).toString(); dbh.put("Username", new RuntimeScalar(args.get(2).toString())); - // Implementation... + // ... JDBC connection logic return dbh.createReference().getList(); } } ``` +**Key points:** +- DBI.pm calls `XSLoader::load('DBI')` to load the Java implementation +- Java class is in `org.perlonjava.perlmodule.Dbi` (naming convention) +- `initialize()` method registers all Java-implemented methods +- Pure Perl methods (like `do()`) can call Java methods (like `prepare()`, `execute()`) + ## Best Practices 1. Keep pure Perl code for simple functionality diff --git a/docs/ARCHITECTURE.md b/docs/reference/architecture.md similarity index 87% rename from docs/ARCHITECTURE.md rename to docs/reference/architecture.md index 411e5ebaf..3c9b61fe9 100644 --- a/docs/ARCHITECTURE.md +++ b/docs/reference/architecture.md @@ -1,5 +1,17 @@ # PerlOnJava Architecture +## Overview + +PerlOnJava compiles Perl source code into JVM bytecode, enabling Perl programs to run natively on the Java Virtual Machine. The compilation process transforms Perl scripts into Java classes at runtime, providing seamless integration with Java libraries and frameworks while maintaining Perl semantics. + +**Key features:** +- Compile-time transformation from Perl to JVM bytecode +- Direct access to Java libraries via JDBC, Maven, and other JVM tools +- Implements most Perl 5.42 features including references, closures, and regular expressions +- Includes 150+ core Perl modules + +This document describes the internal architecture and compilation pipeline. + ## Internal Modules ### Project Structure diff --git a/docs/reference/cli-options.md b/docs/reference/cli-options.md new file mode 100644 index 000000000..1fcfb142a --- /dev/null +++ b/docs/reference/cli-options.md @@ -0,0 +1,217 @@ +# CLI Options Reference + +Command-line options for the `jperl` command. + +## Synopsis + +```bash +jperl [options] [program | -e 'command'] [arguments] +``` + +## Common Options + +### Basic Usage + +- **`-e 'command'`** - Execute command + ```bash + ./jperl -e 'print "Hello\n"' + ``` + +- **`-E 'command'`** - Execute command with all features enabled + ```bash + ./jperl -E 'say "Hello"' # say requires -E + ``` + +- **`-c`** - Check syntax only (no execution) + ```bash + ./jperl -c script.pl + ``` + +- **`-v`** - Print version + ```bash + ./jperl -v + ``` + +- **`-h`** or **`-?`** - Show help + ```bash + ./jperl -h + ``` + +### Module and Library + +- **`-I dir`** - Add directory to `@INC` (module search path) + ```bash + ./jperl -I/path/to/lib script.pl + ``` + +- **`-M module`** - Load module (like `use module`) + ```bash + ./jperl -MJSON -E 'say encode_json({a => 1})' + ``` + +- **`-m module`** - Load module (like `use module ()`) + ```bash + ./jperl -mJSON script.pl + ``` + +### Input Processing + +- **`-n`** - Process input line-by-line (implicit loop, no print) + ```bash + echo -e "foo\nbar" | ./jperl -ne 'print if /foo/' + ``` + +- **`-p`** - Like `-n` but print `$_` after each iteration + ```bash + echo -e "foo\nbar" | ./jperl -pe 's/foo/baz/' + ``` + +- **`-a`** - Autosplit mode (splits `$_` into `@F`) + ```bash + echo "a b c" | ./jperl -ane 'print "$F[1]\n"' # Prints: b + ``` + +- **`-F pattern`** - Split pattern for `-a` (default: whitespace) + ```bash + echo "a:b:c" | ./jperl -F: -ane 'print "$F[1]\n"' # Prints: b + ``` + +- **`-0[octal]`** - Input record separator (default: newline) + ```bash + # Read null-terminated records + ./jperl -0 -ne 'print' file.txt + + # Read entire file as one record + ./jperl -0777 -ne 'print length' file.txt + ``` + +- **`-l[octal]`** - Enable line ending processing + ```bash + # Auto-chomp and add newline to print + ./jperl -lne 'print uc' file.txt + ``` + +- **`-i[extension]`** - Edit files in-place + ```bash + # In-place edit with backup + ./jperl -i.bak -pe 's/foo/bar/g' file.txt + + # In-place edit without backup + ./jperl -i -pe 's/foo/bar/g' file.txt + ``` + +### Script Arguments + +- **`-s`** - Enable switch parsing for script + ```bash + # script.pl can access -foo as $foo + ./jperl -s script.pl -foo=bar + ``` + +### Warnings and Strictness + +- **`-w`** - Enable warnings + ```bash + ./jperl -w script.pl + ``` + +- **`-W`** - Enable all warnings + ```bash + ./jperl -W script.pl + ``` + +- **`-X`** - Disable all warnings + ```bash + ./jperl -X script.pl + ``` + +### Other Options + +- **`-x[dir]`** - Extract script from message + ```bash + ./jperl -x script.txt + ``` + +- **`-S`** - Search for script in PATH + ```bash + ./jperl -S script.pl + ``` + +- **`-g`** - Read all input before executing (slurp mode for `-n`/`-p`) + +## Debugging Options + +- **`--debug`** - Show debug information + ```bash + ./jperl --debug -E 'say "test"' + ``` + +- **`--disassemble`** - Show disassembled bytecode + ```bash + ./jperl --disassemble script.pl + ``` + +- **`--tokenize`** - Show lexer output + ```bash + ./jperl --tokenize -E '$x = 1' + ``` + +- **`--parse`** - Show parser output (AST) + ```bash + ./jperl --parse -E 'my $x = 1' + ``` + +- **`-V`** - Print detailed configuration + ```bash + ./jperl -V + ``` + +## Environment Variables + +- **`PERL5LIB`** - Directories to add to `@INC` + ```bash + export PERL5LIB=/path/to/lib + ./jperl script.pl + ``` + +- **`PERL5OPT`** - Default options for perl + ```bash + export PERL5OPT='-Mwarnings -Mstrict' + ./jperl script.pl + ``` + +## Combining Options + +Options can be combined for powerful one-liners: + +```bash +# Replace text in all files +./jperl -i.bak -pe 's/old/new/g' *.txt + +# Process CSV with auto-split +./jperl -F, -ane 'print "$F[2]\n"' data.csv + +# Count lines matching pattern +./jperl -ne 'END {print $.} /pattern/ or next' file.txt + +# Sum numbers in a column +./jperl -ane '$sum += $F[2]; END {print $sum}' data.txt +``` + +## Not Implemented + +The following standard Perl options are not yet implemented: + +- **`-T`** - Taint checks +- **`-t`** - Taint checks with warnings +- **`-u`** - Dumps core after compiling +- **`-U`** - Allows unsafe operations +- **`-d[t][:debugger]`** - Run under debugger +- **`-D[number/list]`** - Set debugging flags +- **`-C [number/list]`** - Control Unicode features + +## See Also + +- [Quick Start](../../QUICKSTART.md) - Getting started guide +- [One-liners](../getting-started/oneliners.md) - Practical examples +- [Feature Matrix](feature-matrix.md) - Supported features diff --git a/docs/reference/configure.md b/docs/reference/configure.md new file mode 100644 index 000000000..c16513677 --- /dev/null +++ b/docs/reference/configure.md @@ -0,0 +1,328 @@ +# Configure.pl Reference + +The `Configure.pl` script manages configuration settings and dependencies for PerlOnJava. + +## Synopsis + +```bash +./Configure.pl [options] +./Configure.pl -D key=value +./Configure.pl --search keyword +./Configure.pl --direct group:artifact:version +./Configure.pl --upgrade +``` + +## Options + +### Help and Information + +**`-h, --help`** +- Show help message and usage instructions + +```bash +./Configure.pl --help +``` + +### Configuration Management + +**`-D key=value`** +- Set configuration values in `Configuration.java` +- Can specify multiple key-value pairs +- String values are automatically quoted +- Boolean/numeric values are used as-is + +```bash +./Configure.pl -D perlVersion=v5.40.0 +./Configure.pl -D jarVersion=3.0.1 +./Configure.pl -D strict_mode=true -D enable_optimizations=false +``` + +**Special behavior for `jarVersion`:** +- Automatically updates all references to the JAR filename throughout the repository +- Updates from `perlonjava-OLD.jar` to `perlonjava-NEW.jar` in all text files + +**View current configuration:** +```bash +./Configure.pl +``` + +### Dependency Management + +**`--search keyword`** +- Search Maven Central for artifacts by keyword, class name, or group:artifact +- Interactive selection if multiple matches found +- Useful for finding JDBC drivers and other libraries + +```bash +# Search by keyword +./Configure.pl --search h2 +./Configure.pl --search mysql + +# Search by driver class name +./Configure.pl --search org.h2.Driver +./Configure.pl --search com.mysql.cj.jdbc.Driver + +# Search by group:artifact +./Configure.pl --search org.postgresql:postgresql +``` + +**Search behavior:** +- Class names (ending in `.Driver`): Searches by fully qualified class name +- Keywords with `:`: Searches by `group:artifact` pattern +- Other keywords: Searches in artifact name and text fields +- Returns top 10 most relevant results ranked by JDBC relevance +- Prompts for selection if multiple matches found + +**`--direct group:artifact:version`** +- Add dependency using direct Maven coordinates +- No search required - immediately updates build files +- Format must be: `group:artifact:version` + +```bash +./Configure.pl --direct com.h2database:h2:2.2.224 +./Configure.pl --direct org.postgresql:postgresql:42.7.1 +./Configure.pl --direct com.mysql:mysql-connector-j:8.2.0 +``` + +**`--verbose`** +- Enable verbose output for debugging +- Shows Maven Central API URLs +- Displays full search results as JSON +- Useful for troubleshooting search issues + +```bash +./Configure.pl --search h2 --verbose +``` + +### Dependency Upgrades + +**`--upgrade`** +- Upgrade all project dependencies to their latest versions +- Updates both Maven (`pom.xml`) and Gradle (`build.gradle`) dependencies +- Uses `mvn versions:use-latest-versions` for Maven +- Uses `./gradlew versionCatalogUpdate` for Gradle + +```bash +./Configure.pl --upgrade +``` + +**Requirements:** +- Maven must be installed for Maven upgrades +- Gradle wrapper must be present for Gradle upgrades + +## Workflow + +### Adding JDBC Drivers + +**Recommended workflow:** + +1. Search for the driver: +```bash +./Configure.pl --search mysql-connector-java +``` + +2. Or use direct coordinates if you know them: +```bash +./Configure.pl --direct com.mysql:mysql-connector-j:8.2.0 +``` + +3. Rebuild the project to include the driver: +```bash +make +``` + +The driver is now bundled in the PerlOnJava JAR. + +**Alternative: Manual CLASSPATH** + +Instead of bundling drivers, you can load them at runtime: + +```bash +# Download driver manually +wget https://repo1.maven.org/maven2/com/h2database/h2/2.2.224/h2-2.2.224.jar + +# Use with CLASSPATH +CLASSPATH=/path/to/h2-2.2.224.jar ./jperl script.pl +``` + +## How It Works + +### Dependency Management + +When you add a dependency with `--search` or `--direct`: + +1. **Updates `build.gradle`** (if present): + - Adds `implementation "group:artifact:version"` to dependencies block + +2. **Updates `pom.xml`** (if present): + - Adds `` entry with groupId, artifactId, version + +3. **Requires rebuild**: + - Run `make` or `./gradlew build` to download and bundle the dependency + +### Search Ranking + +The Maven Central search ranks results by JDBC relevance: + +- **+5 points**: `jdbc` in group or artifact name +- **+4 points**: Class name ends with `Driver` +- **+3 points**: Database keywords (mysql, postgresql, oracle, sqlserver, database) +- **+2 points**: `jdbc` in version +- **Bonus**: Logarithm of download count + +This ensures JDBC drivers appear first in search results. + +### Configuration Updates + +When you set configuration with `-D`: + +1. Reads `src/main/java/org/perlonjava/Configuration.java` +2. Finds `public static final Type key = value;` declarations +3. Replaces value with new value +4. Writes updated file back + +For `jarVersion` updates, also: +- Scans all text files in the repository +- Replaces old JAR filename with new one +- Skips binary files and hidden directories + +## Examples + +### View Current Configuration + +```bash +./Configure.pl +``` + +Output: +``` +Current configuration: + +perlVersion = "v5.40.0" +jarVersion = "3.0.1" +strict_mode = true +``` + +### Update Configuration + +```bash +./Configure.pl -D perlVersion=v5.42.0 -D jarVersion=3.1.0 +``` + +### Search and Add JDBC Driver + +```bash +# Search for PostgreSQL driver +./Configure.pl --search postgresql + +# Output shows: +# Multiple matches found: +# [0] org.postgresql:postgresql:42.7.1 +# [1] com.impossibl.pgjdbc-ng:pgjdbc-ng:0.8.9 +# [2] ... +# Select number [0-9]: 0 + +# Updates build.gradle and pom.xml +# Updated build.gradle +# Updated pom.xml + +# Rebuild to include driver +make +``` + +### Add Driver with Direct Coordinates + +```bash +# Add H2 database driver +./Configure.pl --direct com.h2database:h2:2.2.224 + +# Rebuild +make +``` + +### Upgrade All Dependencies + +```bash +./Configure.pl --upgrade +``` + +Output: +``` +Upgrading project dependencies to latest versions... +Updating Maven dependencies to latest versions... +Maven dependencies updated successfully. +Updating Gradle dependencies to latest versions using version catalog... +Gradle dependencies updated successfully. +``` + +## Common Use Cases + +### Adding a Database Driver + +```bash +# Option 1: Search and select +./Configure.pl --search mysql +make + +# Option 2: Direct coordinates +./Configure.pl --direct com.mysql:mysql-connector-j:8.2.0 +make + +# Option 3: Manual CLASSPATH (no rebuild needed) +CLASSPATH=/path/to/mysql-connector.jar ./jperl script.pl +``` + +### Updating Project Version + +```bash +./Configure.pl -D jarVersion=4.0.0 +# This updates Configuration.java and all references to perlonjava-*.jar +``` + +### Finding Available Drivers + +```bash +# Search by database name +./Configure.pl --search postgresql --verbose + +# Search by driver class +./Configure.pl --search org.postgresql.Driver --verbose +``` + +## Troubleshooting + +### Search Returns No Results + +**Problem**: `./Configure.pl --search keyword` finds nothing + +**Solutions**: +- Try broader keywords: `mysql` instead of `mysql-connector-java-8.2.0` +- Search by driver class: `./Configure.pl --search com.mysql.cj.jdbc.Driver` +- Use `--verbose` to see search URL and results +- Use `--direct` if you know the exact coordinates + +### Dependencies Not Found After Adding + +**Problem**: Added dependency with Configure.pl but not available at runtime + +**Solution**: You must rebuild after adding dependencies: +```bash +./Configure.pl --direct group:artifact:version +make # This downloads and bundles the dependency +``` + +### Version Conflicts + +**Problem**: Multiple versions of same library + +**Solution**: Edit `build.gradle` or `pom.xml` manually to resolve conflicts, or use: +```bash +./gradlew dependencies # Show dependency tree +mvn dependency:tree # Show Maven dependency tree +``` + +## See Also + +- **[Installation Guide](../getting-started/installation.md)** - Build and setup +- **[Database Access Guide](../guides/database-access.md)** - Using JDBC drivers +- **[CLI Options](cli-options.md)** - jperl command-line options diff --git a/docs/FEATURE_MATRIX.md b/docs/reference/feature-matrix.md similarity index 100% rename from docs/FEATURE_MATRIX.md rename to docs/reference/feature-matrix.md diff --git a/docs/TESTING.md b/docs/reference/testing.md similarity index 77% rename from docs/TESTING.md rename to docs/reference/testing.md index bfac94e6f..bac2e173e 100644 --- a/docs/TESTING.md +++ b/docs/reference/testing.md @@ -134,6 +134,68 @@ make test-all # Review test_results.json for any regressions ``` +### Tracking Test Progress Over Time + +For long-running development work, track test results over time to monitor progress and catch regressions: + +#### 1. Run Tests with Timestamped Logs + +```bash +# Clean up and run comprehensive tests +killall java +rm -rf perl5_t/t/tmp_* + +# Run with extended timeout and save to dated log +perl dev/tools/perl_test_runner.pl \ + --timeout 300 \ + --output out.json \ + perl5_t/t \ + 2>&1 > logs/test_$(date +%Y%m%d_%H%M%S).log +``` + +**Workflow details:** +- `killall java` - Stop any hung Java processes from previous runs +- `rm -rf perl5_t/t/tmp_*` - Clean temporary test files +- `--timeout 300` - Allow 5 minutes per test (for slower tests) +- `logs/test_YYYYMMDD_HHMMSS.log` - Timestamped log for tracking history + +#### 2. Compare Test Runs + +Compare two test runs to see what changed: + +```bash +perl dev/tools/compare_test_logs.pl \ + logs/test_20260206_090000.log \ + logs/test_20260206_102400.log +``` + +**Output shows:** +- Tests that started passing +- Tests that started failing +- Changes in test counts +- New tests added or removed +- Summary of improvements or regressions + +**Use cases:** +- Before/after implementing a feature +- Daily progress tracking on a branch +- Identifying when a regression was introduced +- Measuring impact of optimizations + +#### 3. Log Organization + +Suggested `logs/` directory structure: + +``` +logs/ +├── test_20260206_090000.log # Baseline +├── test_20260206_102400.log # After Feature A +├── test_20260206_153000.log # After Bug Fix +└── test_20260207_101500.log # Latest +``` + +Keep baseline logs for major milestones to track long-term progress. + ### CI/CD Pipeline ```bash @@ -303,11 +365,14 @@ To import Perl test files and verify their behavior under PerlOnJava: perl dev/import-perl5/sync.pl ``` -This will copy all files from `perl5/` to `perl5_t/`, creating a complete Perl 5 test environment including: -- Core tests (`perl5_t/t/`) +This script reads `dev/import-perl5/config.yaml` and copies configured files from the `perl5/` directory (Perl 5 source repository) to `perl5_t/` at the project root, creating: +- Core tests in `perl5_t/t/` +- Module tests in `perl5_t/[Module]/` - Test infrastructure (`TestInit.pm`, `MANIFEST`) - Supporting files (`Porting/` directory) +The script also applies patches from `dev/import-perl5/patches/` for PerlOnJava compatibility. + ### Running Imported Tests To run the imported Perl5 tests: @@ -323,7 +388,7 @@ See `dev/import-perl5/README.md` for more details on: ## See Also -- [Build Guide](BUILD.md) - Building PerlOnJava -- [Architecture](ARCHITECTURE.md) - System architecture -- [Import System](../dev/import-perl5/README.md) - Importing Perl5 tests +- [Installation Guide](../getting-started/installation.md) - Building PerlOnJava +- [Architecture](architecture.md) - System architecture +- [Import System](../../dev/import-perl5/README.md) - Importing Perl5 tests