Skip to content

Commit 1afee08

Browse files
fglockfglock
authored
Merge pull request #498 from fglock/feature/io-pty
feat: implement IO::Tty/IO::Pty module with FFM native PTY bindings
2 parents 7a3992f + c75125a commit 1afee08

24 files changed

Lines changed: 3616 additions & 15 deletions

File tree

‎.agents/skills/port-native-module/SKILL.md‎

Lines changed: 374 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,374 @@
1+
# Port Native/FFM CPAN Module to PerlOnJava
2+
3+
## When to Use This Skill
4+
5+
Use this skill when porting a CPAN module that requires **native system calls**
6+
(POSIX functions, ioctl, terminal control, etc.) via Java's Foreign Function &
7+
Memory (FFM) API. This extends the standard `port-cpan-module` skill with the
8+
additional FFM layer.
9+
10+
Examples of modules that need this approach:
11+
- IO::Tty / IO::Pty (pty allocation, ioctl, termios)
12+
- Term::ReadKey (terminal mode control via tcgetattr/tcsetattr)
13+
- IPC::SysV (shared memory, semaphores, message queues)
14+
- Sys::Syslog (syslog system calls)
15+
- Any module whose XS code calls libc/POSIX functions directly
16+
17+
For modules that only need Java standard library equivalents (crypto, time,
18+
encoding), use the standard `port-cpan-module` skill instead.
19+
20+
## Key Principle: Windows Support
21+
22+
**Always support Windows when possible. When not possible, match the original
23+
CPAN module's behavior on Windows.**
24+
25+
Concretely:
26+
27+
1. **If the original module works on Windows** (e.g., Term::ReadKey uses
28+
`Win32::Console`), provide a Windows implementation in `FFMPosixWindows.java`
29+
using either Windows API calls via FFM or pure Java emulation.
30+
31+
2. **If the original module explicitly rejects Windows** (e.g., IO::Tty dies
32+
with `"OS unsupported"` on `$^O eq 'MSWin32'`), replicate that same behavior
33+
at each layer:
34+
- `FFMPosixWindows.java`: throw `UnsupportedOperationException` (Java-internal,
35+
matches existing convention for unimplemented FFM functions)
36+
- Java perlmodule: catch `UnsupportedOperationException` and translate to a
37+
Perl-visible error via `WarnDie.die()` or `PerlCompilerException`
38+
- Perl `.pm` shim: `die` with the same message the original module uses, so
39+
users see identical behavior
40+
41+
3. **If a partial Windows implementation is feasible** (e.g., some functions
42+
work via ConPTY or Java APIs but others don't), implement what you can and
43+
clearly document which functions are Windows-only stubs.
44+
45+
The goal: a user switching from `perl` + CPAN to `jperl` + `jcpan` should see
46+
**identical platform support** — if the module worked on their OS before, it
47+
should work on PerlOnJava too; if it didn't, it should fail the same way.
48+
49+
## Prerequisites
50+
51+
- Read the standard `port-cpan-module` skill first for general porting patterns
52+
- Read `docs/guides/module-porting.md` for naming conventions and checklists
53+
- Familiarity with Java FFM API (java.lang.foreign.*)
54+
55+
## Architecture Overview
56+
57+
PerlOnJava calls native C functions via a layered FFM architecture:
58+
59+
```
60+
Perl code (use IO::Pty; $pty->new)
61+
|
62+
v
63+
Java perlmodule (IOTty.java) <-- Perl API in Java
64+
|
65+
v
66+
NativeUtils / ExtendedNativeUtils <-- Cross-platform dispatch
67+
|
68+
v
69+
PosixLibrary (thin facade)
70+
|
71+
v
72+
FFMPosixInterface <-- Java interface (contract)
73+
|
74+
v
75+
FFMPosix.get() <-- Factory, detects OS
76+
|
77+
v
78+
FFMPosixLinux | FFMPosixMacOS | FFMPosixWindows <-- Platform impls
79+
```
80+
81+
### Key Files
82+
83+
| File | Role |
84+
|------|------|
85+
| `src/main/java/org/perlonjava/runtime/nativ/ffm/FFMPosixInterface.java` | Interface defining all POSIX function signatures |
86+
| `src/main/java/org/perlonjava/runtime/nativ/ffm/FFMPosix.java` | Factory: detects OS, returns correct implementation |
87+
| `src/main/java/org/perlonjava/runtime/nativ/ffm/FFMPosixLinux.java` | Core FFM implementation (Linux + base for macOS) |
88+
| `src/main/java/org/perlonjava/runtime/nativ/ffm/FFMPosixMacOS.java` | macOS overrides (extends Linux; override only what differs) |
89+
| `src/main/java/org/perlonjava/runtime/nativ/ffm/FFMPosixWindows.java` | Windows emulation or UnsupportedOperationException |
90+
| `src/main/java/org/perlonjava/runtime/nativ/NativeUtils.java` | High-level: symlink, link, getppid, getuid, etc. |
91+
| `src/main/java/org/perlonjava/runtime/nativ/ExtendedNativeUtils.java` | High-level: user/group info, network, SysV IPC |
92+
| `src/main/java/org/perlonjava/runtime/nativ/PosixLibrary.java` | Facade: `PosixLibrary.getFFM()` returns `FFMPosix.get()` |
93+
94+
### FFM Configuration
95+
96+
PerlOnJava targets Java 21+ with FFM enabled:
97+
- `build.gradle` / `pom.xml` includes `--enable-native-access=ALL-UNNAMED`
98+
- FFM is enabled by default; disable with `-Dperlonjava.ffm.enabled=false`
99+
- The `jperl` wrapper script passes the required JVM flags
100+
101+
## Step-by-Step Process
102+
103+
### Phase 1: Analysis (same as port-cpan-module, plus)
104+
105+
1. **Study the XS source** to identify which C/POSIX functions are called
106+
2. **Classify each function call:**
107+
108+
| Category | Example | FFM Complexity |
109+
|----------|---------|----------------|
110+
| Simple (int→int) | `getuid()`, `setsid()`, `umask()` | Trivial |
111+
| String arg | `chmod(path, mode)`, `open(path, flags)` | Easy (Arena string alloc) |
112+
| String return | `strerror(errno)`, `ptsname(fd)` | Easy (readCString) |
113+
| Struct I/O | `stat()`, `tcgetattr()` | Medium (struct layout) |
114+
| Variadic | `ioctl(fd, req, ...)` | Medium (firstVariadicArg) |
115+
| Callback | `signal handlers` | Hard (upcall stubs) |
116+
117+
3. **Check what's already implemented** in `FFMPosixInterface.java`
118+
4. **Identify platform differences** (macOS vs Linux struct layouts, constant values)
119+
5. **Check if existing IOHandle implementations** can be reused or if a new one is needed
120+
121+
### Phase 2: Add FFM Bindings
122+
123+
Follow this pattern for each new native function:
124+
125+
#### Step 2a: Add method to FFMPosixInterface.java
126+
127+
```java
128+
// In the appropriate section (Terminal Functions, Process Functions, etc.)
129+
130+
/**
131+
* Brief description of what the function does.
132+
* @param fd File descriptor
133+
* @return 0 on success, -1 on error (check errno)
134+
*/
135+
int myFunction(int fd);
136+
```
137+
138+
#### Step 2b: Add MethodHandle + implementation in FFMPosixLinux.java
139+
140+
```java
141+
// 1. Declare at class level:
142+
private static MethodHandle myFunctionHandle;
143+
144+
// 2. Initialize in ensureInitialized():
145+
myFunctionHandle = linker.downcallHandle(
146+
stdlib.find("myFunction").orElseThrow(),
147+
FunctionDescriptor.of(ValueLayout.JAVA_INT, ValueLayout.JAVA_INT),
148+
captureErrno // include if errno capture needed
149+
);
150+
151+
// 3. Implement the interface method:
152+
@Override
153+
public int myFunction(int fd) {
154+
ensureInitialized();
155+
try (Arena arena = Arena.ofConfined()) {
156+
MemorySegment capturedState = arena.allocate(
157+
Linker.Option.captureStateLayout());
158+
int result = (int) myFunctionHandle.invokeExact(capturedState, fd);
159+
if (result == -1) {
160+
int err = capturedState.get(ValueLayout.JAVA_INT, errnoOffset);
161+
setErrno(err);
162+
}
163+
return result;
164+
} catch (Throwable e) {
165+
setErrno(1);
166+
return -1;
167+
}
168+
}
169+
```
170+
171+
#### Step 2c: Override in FFMPosixMacOS.java (only if macOS differs)
172+
173+
Most functions are identical on Linux and macOS. Override only when:
174+
- Struct layouts differ (different field offsets or sizes)
175+
- Constants differ (ioctl request codes)
176+
- Function signatures differ
177+
178+
#### Step 2d: Handle Windows in FFMPosixWindows.java
179+
180+
Check how the **original CPAN module** behaves on Windows, then match it:
181+
182+
- **If the original module supports Windows**: provide a real implementation
183+
using Windows API calls via FFM (e.g., `kernel32.dll` functions) or pure
184+
Java emulation.
185+
- **If the original module rejects Windows** (e.g., `die "OS unsupported"`):
186+
throw `UnsupportedOperationException` with a message matching the original.
187+
The Perl `.pm` shim should also replicate the original's Windows guard.
188+
- **If partial support is feasible**: implement what you can, document the rest
189+
as unsupported, and throw for the gaps.
190+
191+
Never silently return success for a function that isn't actually working.
192+
193+
### Common FFM Patterns
194+
195+
#### Simple function (no args, returns int)
196+
```java
197+
// getuid() — no errno needed
198+
getuidHandle = linker.downcallHandle(
199+
stdlib.find("getuid").orElseThrow(),
200+
FunctionDescriptor.of(ValueLayout.JAVA_INT)
201+
);
202+
203+
public int getuid() {
204+
ensureInitialized();
205+
try { return (int) getuidHandle.invokeExact(); }
206+
catch (Throwable e) { return -1; }
207+
}
208+
```
209+
210+
#### Function with string argument
211+
```java
212+
// chmod(path, mode) — needs Arena for string, captures errno
213+
public int chmod(String path, int mode) {
214+
ensureInitialized();
215+
try (Arena arena = Arena.ofConfined()) {
216+
MemorySegment pathSegment = arena.allocateFrom(path);
217+
MemorySegment capturedState = arena.allocate(
218+
Linker.Option.captureStateLayout());
219+
int result = (int) chmodHandle.invokeExact(
220+
capturedState, pathSegment, mode);
221+
if (result == -1) {
222+
setErrno(capturedState.get(ValueLayout.JAVA_INT, errnoOffset));
223+
}
224+
return result;
225+
} catch (Throwable e) { setErrno(5); return -1; }
226+
}
227+
```
228+
229+
#### Function returning C string
230+
```java
231+
// ptsname(fd) — returns char*
232+
public String ptsname(int fd) {
233+
ensureInitialized();
234+
try {
235+
MemorySegment result = (MemorySegment) ptsnameHandle.invokeExact(fd);
236+
if (result.address() == 0) return null;
237+
return result.reinterpret(1024).getString(0);
238+
} catch (Throwable e) { return null; }
239+
}
240+
```
241+
242+
#### Struct I/O (reading fields at offsets)
243+
```java
244+
// Platform-specific struct offsets (set in init method)
245+
private static long FIELD_OFFSET;
246+
247+
private static void initStructOffsets() {
248+
if (IS_MACOS) {
249+
FIELD_OFFSET = 8; // macOS layout
250+
} else {
251+
FIELD_OFFSET = 4; // Linux layout
252+
}
253+
}
254+
255+
// Read struct from native memory
256+
MyRecord readStruct(MemorySegment ptr) {
257+
MemorySegment s = ptr.reinterpret(STRUCT_SIZE);
258+
int field1 = s.get(ValueLayout.JAVA_INT, FIELD1_OFFSET);
259+
String field2 = readCString(s.get(ValueLayout.ADDRESS, FIELD2_OFFSET));
260+
return new MyRecord(field1, field2);
261+
}
262+
```
263+
264+
#### Variadic function (ioctl)
265+
```java
266+
// ioctl(fd, request, ...) — variadic after arg index 2
267+
ioctlHandle = linker.downcallHandle(
268+
stdlib.find("ioctl").orElseThrow(),
269+
FunctionDescriptor.of(
270+
ValueLayout.JAVA_INT, // return
271+
ValueLayout.JAVA_INT, // fd
272+
ValueLayout.JAVA_LONG, // request (unsigned long)
273+
ValueLayout.ADDRESS // variadic arg: pointer
274+
),
275+
Linker.Option.firstVariadicArg(2),
276+
captureErrno
277+
);
278+
```
279+
280+
### Phase 3: Bridge Native FDs to PerlOnJava I/O (if needed)
281+
282+
Some native modules produce raw POSIX file descriptors (e.g., `posix_openpt()`
283+
returns an int fd). PerlOnJava's I/O system uses Java `IOHandle` objects. To
284+
bridge the gap:
285+
286+
1. **Create a new IOHandle implementation** (e.g., `NativeFdIOHandle`) that:
287+
- Stores the raw POSIX fd
288+
- Implements `read()` / `write()` / `close()` via FFM `read()`/`write()`/`close()`
289+
- Implements `fileno()` returning the native fd
290+
- Implements `sysread()` / `syswrite()` for unbuffered I/O
291+
- Registers in `FileDescriptorTable` for `select()` support
292+
293+
2. **Register the handle** so Perl code can use it:
294+
```java
295+
int fd = FileDescriptorTable.register(nativeHandle);
296+
// Perl's fdopen($fd, "r+") can now find it
297+
```
298+
299+
3. **Key classes to understand:**
300+
- `IOHandle` interface — base contract for all I/O handles
301+
- `FileDescriptorTable` — maps simulated fd numbers to IOHandle objects
302+
- `RuntimeIO` — wraps IOHandle for the Perl runtime
303+
- `DupIOHandle` — wraps handles for dup/dup2 operations
304+
- `IOOperator.findFileHandleByDescriptor()` — looks up handles by fd number
305+
306+
### Phase 4: Create Java perlmodule
307+
308+
Follow the standard `port-cpan-module` pattern:
309+
- File: `src/main/java/org/perlonjava/runtime/perlmodule/ModuleName.java`
310+
- Extends `PerlModuleBase`
311+
- Static `initialize()` method called by XSLoader
312+
- Methods call through to FFM layer via `PosixLibrary.getFFM()` or directly
313+
314+
### Phase 5: Create Perl shim (.pm)
315+
316+
Follow the standard `port-cpan-module` pattern:
317+
- File: `src/main/perl/lib/Module/Name.pm`
318+
- Calls `XSLoader::load('Module::Name', $VERSION)`
319+
- Pure Perl helper methods wrap Java-backed functions
320+
- Preserve original CPAN module's API exactly
321+
322+
### Phase 6: Testing
323+
324+
Same as `port-cpan-module`, plus:
325+
- Test on both macOS and Linux if struct layouts differ
326+
- Test error paths (invalid fd, permission denied, etc.)
327+
- Test errno propagation
328+
- Verify `isatty()` returns correct results for new handle types
329+
330+
## Checklist (extends port-cpan-module checklist)
331+
332+
### FFM Layer
333+
- [ ] Identify all C/POSIX functions needed
334+
- [ ] Check which are already in `FFMPosixInterface.java`
335+
- [ ] Add new methods to `FFMPosixInterface.java` with Javadoc
336+
- [ ] Implement in `FFMPosixLinux.java` with errno capture
337+
- [ ] Override in `FFMPosixMacOS.java` if platform-specific
338+
- [ ] Handle Windows in `FFMPosixWindows.java` (emulate or throw)
339+
- [ ] Add any new data records to `FFMPosixInterface.java`
340+
341+
### I/O Bridge (if raw fds are involved)
342+
- [ ] Create new `IOHandle` implementation if needed
343+
- [ ] Register handles in `FileDescriptorTable`
344+
- [ ] Wire into `IOOperator.findFileHandleByDescriptor()`
345+
- [ ] Test `fileno()`, `sysread()`, `syswrite()`, `close()`
346+
- [ ] Test `select()` integration if applicable
347+
348+
### Platform Constants
349+
- [ ] Identify platform-specific constants (ioctl codes, struct sizes)
350+
- [ ] Use `IS_MACOS` flag for conditional initialization
351+
- [ ] Document constant values and their sources
352+
353+
## Existing FFM Functions (reference)
354+
355+
Already implemented in `FFMPosixInterface`:
356+
- Process: `kill`, `getppid`, `waitpid`
357+
- User/Group: `getuid`, `geteuid`, `getgid`, `getegid`, `getpwnam`, `getpwuid`, `getpwent`, `setpwent`, `endpwent`
358+
- File: `stat`, `lstat`, `chmod`, `link`, `utimes`
359+
- Terminal: `isatty`
360+
- File control: `fcntl`, `umask`
361+
- Error: `errno`, `setErrno`, `strerror`
362+
363+
Also available outside FFM:
364+
- `symlink` (NativeUtils, via Java NIO)
365+
- `ioctl` (IOOperator, currently a stub returning false)
366+
367+
## References
368+
369+
- `port-cpan-module` skill — standard XS→Java porting workflow
370+
- `docs/guides/module-porting.md` — authoritative naming/layout guide
371+
- `dev/modules/` — per-module implementation plans
372+
- Java FFM tutorial: https://docs.oracle.com/en/java/javase/22/core/foreign-function-and-memory-api.html
373+
- `src/main/java/org/perlonjava/runtime/nativ/ffm/` — existing FFM code (best reference)
374+
- `src/main/java/org/perlonjava/runtime/io/` — I/O handle implementations

‎.gitignore‎

Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -106,3 +106,6 @@ test-*.html
106106

107107
# Ignore generated lib/ directory at top level
108108
/lib/
109+
110+
# Ignore heap dumps
111+
*.hprof

0 commit comments

Comments
 (0)