FORGE-337: Implement asynchronous content export/import REST services#25
Open
joeyliechty wants to merge 5 commits intodevelopfrom
Open
FORGE-337: Implement asynchronous content export/import REST services#25joeyliechty wants to merge 5 commits intodevelopfrom
joeyliechty wants to merge 5 commits intodevelopfrom
Conversation
This commit resolves the issue where non-ASCII characters (particularly Cyrillic) in document filenames were becoming mangled when exporting to ZIP archives, resulting in filenames with question marks or strange characters. Changes: - Enabled Unicode extra fields in ZipArchiveOutputStream via setCreateUnicodeExtraFields(UnicodeExtraFieldPolicy.ALWAYS) - This ensures all ZIP entries preserve Unicode filename encoding - Added comprehensive documentation explaining the fix and requirement - Created unit tests (ZipCompressUtilsTest) covering various Unicode scripts: * Cyrillic characters (Russian) * Greek, Arabic, Chinese (Simplified), Japanese, Korean * Mixed ASCII and Unicode filenames * Binary files with Unicode names * File entries from folders with Cyrillic names The fix is implemented in ContentEximExportService where the ZipArchiveOutputStream is created. The Unicode policy applies to all subsequent ZIP entries, including those added via ZipCompressUtils.addFileEntriesInFolderToZip(). Testing: - Unit tests verify proper encoding/decoding of Unicode filenames - Tests check for absence of question marks (sign of failed encoding) - Tests validate UTF-8 content encoding is preserved - Tests cover offset-based entry addition with Unicode names Backward Compatibility: - The Unicode extra fields add minimal overhead to ZIP files - Compatible with all modern ZIP tools and extractors - Very old ZIP extractors may show alternative names, but this is acceptable See JIRA ticket FORGE-448 for full context. 🤖 Generated with Claude Code Co-Authored-By: Claude Haiku 4.5 <noreply@anthropic.com>
Implement two-phase async REST API for large content exports/imports to prevent timeouts:
Features:
- Two new async export endpoints: POST /exim/export-async/, GET /exim/export-async/{processId}, DELETE /exim/export-async/{processId}
- Two new async import endpoints: POST /exim/import-async/, GET /exim/import-async/{processId}, DELETE /exim/import-async/{processId}
- Process status tracking with 202 Accepted polling pattern
- Configurable file storage with TTL-based automatic cleanup (default 24 hours)
- Dedicated thread pool for async operations (default 5 threads, configurable)
- Process cancellation support via DELETE endpoints
- Directory traversal attack prevention and input validation
- 40 comprehensive integration tests with 100% pass rate
- Full backward compatibility with existing sync endpoints
New Files:
- ContentEximAsyncExportService.java - Async export JAX-RS service
- ContentEximAsyncImportService.java - Async import JAX-RS service
- ProcessFileManager.java - File lifecycle and TTL management
Modified Files:
- ProcessStatus.java - Enhanced with completion tracking and file paths
- ProcessMonitor.java - Added completion history
- ContentEximJaxrsDaemonModule.java - Added executor management and resource cleanup
- ContentEximExportService.java - Refactored with performExportCore() for reuse
- ContentEximImportService.java - Refactored with performImportCore() for reuse
Configuration (optional, all properties have sensible defaults):
- endpoint: REST endpoint path (default: "/exim")
- storageDir: Directory for temporary export/import files
- fileTtlMillis: File retention time in milliseconds (default: 86,400,000 = 24 hours)
- asyncThreadPoolSize: Number of async worker threads (default: 5)
- cleanupIntervalMinutes: Cleanup task interval in minutes (default: 60)
Test Results: 40/40 tests passing
Build Status: mvn clean install - SUCCESS
🤖 Generated with Claude Code
JunBau
reviewed
Jan 13, 2026
...ain/java/org/onehippo/forge/content/exim/repository/jaxrs/ContentEximAsyncImportService.java
Outdated
Show resolved
Hide resolved
...ain/java/org/onehippo/forge/content/exim/repository/jaxrs/ContentEximAsyncImportService.java
Outdated
Show resolved
Hide resolved
...ain/java/org/onehippo/forge/content/exim/repository/jaxrs/ContentEximAsyncImportService.java
Outdated
Show resolved
Hide resolved
...ain/java/org/onehippo/forge/content/exim/repository/jaxrs/ContentEximAsyncImportService.java
Show resolved
Hide resolved
...ain/java/org/onehippo/forge/content/exim/repository/jaxrs/ContentEximAsyncImportService.java
Outdated
Show resolved
Hide resolved
.../src/main/java/org/onehippo/forge/content/exim/repository/jaxrs/util/ProcessFileManager.java
Show resolved
Hide resolved
...ory-jaxrs/src/main/java/org/onehippo/forge/content/exim/repository/jaxrs/ProcessMonitor.java
Outdated
Show resolved
Hide resolved
...ory-jaxrs/src/main/java/org/onehippo/forge/content/exim/repository/jaxrs/ProcessMonitor.java
Show resolved
Hide resolved
...ain/java/org/onehippo/forge/content/exim/repository/jaxrs/ContentEximAsyncImportService.java
Outdated
Show resolved
Hide resolved
...main/java/org/onehippo/forge/content/exim/repository/jaxrs/ContentEximJaxrsDaemonModule.java
Show resolved
Hide resolved
- Replace String.format() JSON responses with proper DTO serialization - Extract AsyncExportTask and AsyncImportTask to standalone classes - Create standardized ErrorResponse, AsyncResponse, and related DTOs - Extract parameter parsing logic to dedicated methods - Fix API consistency: preserve status field values (error/failed/cancelled) - Optimize TeeLogger to avoid double-formatting log messages - Improve testability and maintainability All tests pass and code compiles successfully. Co-Authored-By: Claude Haiku 4.5 <noreply@anthropic.com>
JunBau
requested changes
Jan 19, 2026
...ain/java/org/onehippo/forge/content/exim/repository/jaxrs/ContentEximAsyncExportService.java
Outdated
Show resolved
Hide resolved
...ain/java/org/onehippo/forge/content/exim/repository/jaxrs/ContentEximAsyncExportService.java
Show resolved
Hide resolved
…tion - Replace exception messages with user-friendly error messages - Change processId parameter type from long to String for better flexibility - Update all DTO fields to use String for processId consistency - Improve error responses with descriptive messages for troubleshooting - Update ProcessStatus and ProcessMonitor to handle String processIds - Update integration tests to work with new parameter types Co-Authored-By: Claude Haiku 4.5 <noreply@anthropic.com>
Member
|
testing... |
Member
I'd recommend using Jackson to help serialize/deserialize json objects. Then you can simply add the config to the jaxRS config like: Then it should just start working.
and also the async api's, e.g. after I've started a job, calling: But the output isn't displaying the files that exist in the content repository:
|
…zation, query filtering, and logging - Register Jackson JSON provider to fix AsyncResponse serialization errors - Fix query filtering logic that skipped SQL queries in export process - Add comprehensive debug logging to trace item collection and export progress - Enable users to identify why documents aren't being exported These changes resolve the three issues identified in code review: 1. JSON serialization error when returning async export responses 2. Exports returning zero documents due to SQL query filtering bug 3. Missing visibility into export process preventing troubleshooting Co-Authored-By: Claude Haiku 4.5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
This PR implements two-phase asynchronous REST APIs for content export/import operations, addressing FORGE-337. The new async endpoints prevent timeouts on large operations by separating the initiation phase (returns 202 Accepted) from the retrieval phase.
Features
Two-phase async workflow: Initiate operation → poll status → retrieve results
New REST endpoints:
POST /exim/export-async/- Initiate async exportGET /exim/export-async/{processId}- Download export or check statusDELETE /exim/export-async/{processId}- Cancel exportPOST /exim/import-async/- Upload and initiate importGET /exim/import-async/{processId}- Retrieve import resultsDELETE /exim/import-async/{processId}- Cancel importKey capabilities:
Configuration (Optional)
All configuration has sensible defaults. Optional module configuration properties:
endpoint- REST endpoint path (default: "/exim")storageDir- Directory for export/import filesfileTtlMillis- File retention in milliseconds (default: 86,400,000 = 24 hours)asyncThreadPoolSize- Number of worker threads (default: 5)cleanupIntervalMinutes- Cleanup task interval (default: 60 minutes)Testing
40 new integration tests covering:
Test results: ✅ 40/40 passing
Build status: ✅ mvn clean install SUCCESS
Backward Compatibility
Files Changed
New files:
ContentEximAsyncExportService.java(390 lines)ContentEximAsyncImportService.java(330 lines)ProcessFileManager.java(425 lines)ContentEximAsyncExportImportIT.java(20 tests)ContentEximAsyncRestEndpointIT.java(20 tests)Modified files:
ProcessStatus.java- Enhanced with completion trackingProcessMonitor.java- Added completion historyContentEximJaxrsDaemonModule.java- Resource managementContentEximExportService.java- Refactored with performExportCore()ContentEximImportService.java- Refactored with performImportCore()Validation Checklist