feat: add MemoryFileSystem module#6573
Open
lloydrichards wants to merge 1 commit into
Open
Conversation
🦋 Changeset detectedLatest commit: cda9755 The changes in this PR will be included in the next version bump. This PR includes changesets to release 27 packages
Not sure what this means? Click here to learn what changesets are. Click here if you're a maintainer who wants to add another changeset to this PR |
|
No actionable comments were generated in the recent review. 🎉 ℹ️ Recent review info⚙️ Run configurationConfiguration used: Repository UI Review profile: CHILL Plan: Pro Plus Run ID: 📒 Files selected for processing (4)
🚧 Files skipped from review as they are similar to previous changes (4)
📝 WalkthroughWalkthroughAdds an isolated in-memory ChangesMemoryFileSystem
Estimated code review effort: 5 (Critical) | ~120 minutes Sequence Diagram(s)sequenceDiagram
participant FileSystem
participant Volume
participant WatchStream
FileSystem->>Volume: mutate filesystem state
Volume->>Volume: commit normalized transition
Volume->>WatchStream: publish filesystem event
WatchStream-->>FileSystem: emit watched path event
Suggested labels: 🚥 Pre-merge checks | ✅ 2✅ Passed checks (2 passed)
Comment |
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
Actionable comments posted: 1
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@packages/effect/src/internal/memoryFileSystem.ts`:
- Around line 769-810: Update the recursive makeDirectory creation flow to
accumulate every newly created path instead of overwriting createdPath on each
iteration. Emit a Create event for each created directory in creation order,
including intermediate paths such as /a and /a/b, while preserving the existing
empty-event behavior when no directory is created.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Repository UI
Review profile: CHILL
Plan: Pro Plus
Run ID: 054f14df-a2f1-4ce3-8637-2eefbe37f869
📒 Files selected for processing (4)
.changeset/puny-queens-spend.mdpackages/effect/src/MemoryFileSystem.tspackages/effect/src/internal/memoryFileSystem.tspackages/effect/test/MemoryFileSystem.test.ts
github-project-automation
Bot
moved this from Discussion Ongoing
to Waiting on Author
in PR Backlog
lloydrichards
force-pushed
the
feat/memory-file-system
branch
from
July 24, 2026 12:54
a52dfd4 to
cda9755
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Type
Description
I've been looking for an in-memory layer for the
FileSystemfor a little while, something useful for testing filesystem-dependent code without creating and cleaning up temp directories, and for exercising filesystem configuration or dry-run flows without touching the host filesystem. My main motivating use case is stack-effect#108.I spoke briefly with @fubhy about this on discord and wanted to try and push this along based on his work in
effect-smol(see effect-smol/pull/456) and the test suite I build in #6555. I set some clankers on this initially and have been working back and forth for the last couple of days to get it to a stage where its more a questions of improvement than implementation (all 79 tests pass).MemoryFileSystemkeeps an explicit in-memory inode model (directory entries, files, links, and per-open descriptors) on top of the existing v4FileSystemservice. The adapter implements the core filesystem operations, whileFileSystem.makecontinues to supply its derived helpers. Consumers just need to provideMemoryFileSystem.layerin place of a host-backed filesystem.Review guide
Its a chunky 2,650 loc module so I'm not expecting this to just get pushed in without some criticism and rework but I thought its a good place to start and inspire some action. The main implementation is intentionally self-contained:
packages/effect/src/MemoryFileSystem.tspackages/effect/src/internal/memoryFileSystem.tspackages/effect/test/MemoryFileSystem.test.tsThe highest-value review areas are:
The focused test suite currently passes. I’ve also included an architecture overview for readers who want the full model before reviewing the implementation or you can explore call stack graph below:
Call Stack Graph
--- config: flowchart: curve: monotoneX --- flowchart LR F[FileSystem.make facade] F --> access --> withState --> resolve F --> stat --> withState stat --> resolve F --> realPath --> withState realPath --> resolve F --> readLink --> withState readLink --> resolveNoFollow[resolve: do not follow final symlink] resolve -. metadata .-> fileInfo resolveNoFollow -. raw target .-> target F --> readDirectory --> mutate --> resolve F --> readFile --> mutate readFile --> resolve readDirectory --> atime[replace inode atime] --> commit readFile --> atime F --> chmod --> validateMode --> mutate F --> chown --> validateOwner --> mutate F --> utimes --> validateTimes --> mutate chmod --> resolve chown --> resolve utimes --> resolve chmod --> inodeUpdateEvents --> commit chown --> inodeUpdateEvents utimes --> inodeUpdateEvents F --> truncate --> validateSize --> mutate truncate --> resolve --> allocateBytes --> inodeUpdateEvents F --> makeDirectory --> mutate makeDirectory --> resolveParent --> createDirectory --> attachDirectory --> commit F --> link --> mutate link --> resolve link --> resolveParent --> linkInode --> commit F --> symlink --> mutate symlink --> resolveParent symlink --> createSymbolicLink --> linkInode F --> remove --> mutate --> resolveEntry --> detachEntry[recursive detachEntry] --> reclaimInode --> commit F --> rename --> mutate rename --> resolveEntry rename --> resolveParent rename --> topologyChecks[topology checks and optional detach] --> commit copy --> copyEntryUnlocked --> validateCopy[recursive validate/copy] --> cloneInode --> commit copyFile --> copyFileUnlocked --> resolveCopyDestination --> cloneOrUpdate[clone or update] --> commit F --> copyFile --> mutate F --> copy --> mutate F --> open --> acquireRelease --> openDescriptor --> openDescriptorUnlocked --> MemoryFile F --> writeFile --> mutate MemoryFile --> closeDescriptor writeFile --> openDescriptorUnlocked --> writeDescriptorUnlocked --> closeDescriptorUnlocked --> commit F --> makeTempDirectory --> mutate makeTempDirectory --> allocateTempDirectory --> allocateToken[allocate token] --> createDirectory F --> makeTempDirectoryScoped --> acquireRelease makeTempDirectoryScoped --> makeTempDirectory F --> makeTempFile --> mutateInterruptibly --> allocateTempDirectory makeTempFile --> allocateToken --> createFile --> linkInode F --> makeTempFileScoped --> acquireRelease makeTempFileScoped --> makeTempFile F --> glob --> validateGlob --> withState glob --> resolve glob --> directoryWalk[sorted directory walk] --> globMatcher[glob matcher] F --> watch --> StreamUnwrap[Stream.unwrap] --> acquireRelease watch --> subscription[subscription + callback queue] mutate --> commit mutateInterruptibly --> commit commit --> publishWatchEvents F --> exists[exists derived] --> access F --> readFileString[readFileString derived] --> readFile F --> writeFileString[writeFileString derived] --> writeFile F --> stream[stream derived] --> open stream --> readAlloc[MemoryFile.readAlloc] F --> sink[sink derived] --> open sink --> writeAll[MemoryFile.writeAll] classDef read fill:#123247,stroke:#55B6E8,stroke-width:1.5px,color:#E6F6FF classDef namespace fill:#3B2B12,stroke:#E5AE4B,stroke-width:1.5px,color:#FFF4D8 classDef content fill:#153D2A,stroke:#63C287,stroke-width:1.5px,color:#E7FAED classDef lifecycle fill:#302044,stroke:#B494E8,stroke-width:1.5px,color:#F2EBFF classDef derived fill:#273344,stroke:#9BAFC9,stroke-width:1.5px,color:#EEF4FC class access,stat,realPath,readLink,readDirectory,readFile,chmod,chown,utimes,truncate,glob read class makeDirectory,link,symlink,remove,rename namespace class copyFile,copy,open,writeFile content class makeTempDirectory,makeTempDirectoryScoped,makeTempFile,makeTempFileScoped,watch lifecycle class exists,readFileString,writeFileString,stream,sink derived linkStyle default stroke-width:1.5px linkStyle 0,1,2,3,4,5,6,7,8,9,10,11,12,13 stroke-width:1px,stroke-dasharray:4 3Related
Summary by CodeRabbit
Summary by CodeRabbit
New Features
MemoryFileSystemmodule providing an isolated in-memory virtual file system.Bug Fixes