Skip to content

feat(dag): add --local-only to dag export and import#11229

Open
ChayanDass wants to merge 6 commits intoipfs:masterfrom
ChayanDass:feat/dag-export-import-local-only
Open

feat(dag): add --local-only to dag export and import#11229
ChayanDass wants to merge 6 commits intoipfs:masterfrom
ChayanDass:feat/dag-export-import-local-only

Conversation

@ChayanDass
Copy link

Add --local-only to dag export and dag import

Summary

Adds support for partial CAR export and import: export only blocks that exist locally (skipping missing blocks), and import such partial CARs without pinning roots.

Changes

dag export

  • --local-only (bool): When set, only blocks present in the local blockstore are exported; missing blocks are skipped (partial CAR). User must pass --offline explicitly for a local-only DAG walk; the command does not set offline automatically.

dag import

  • --pin-roots (bool, default true): Pin optional roots listed in the CAR headers after importing.
  • For partial CARs, user must pass --pin-roots=false and --local-only (not both with pin-roots true).

Usage

Export a partial CAR (local blocks only; user must pass both flags):

ipfs dag export --local-only --offline <root> > partial.car

Import that partial CAR without pinning roots:

ipfs dag import --pin-roots=false partial.car
# or
ipfs dag import --local-only --pin-roots=false partial.car

References

- Export: only export blocks present locally; skip missing (partial CAR).
  --local-only with --offline. Support both binary and base58 link keys.
- Import: support partial CARs; --local-only with -- pin-roots=false (error if
  both --pin-roots and --local-only set).
- Fix cidFromBinString to accept base58 key format from link implementations.

Signed-off-by: Chayan Das <01chayandas@gmail.com>
@ChayanDass ChayanDass requested a review from a team as a code owner March 7, 2026 08:01
@ChayanDass ChayanDass force-pushed the feat/dag-export-import-local-only branch from 2e5e224 to 9ad0c3b Compare March 7, 2026 08:03
ChayanDass and others added 2 commits March 7, 2026 13:36
- remove local replace directive for go-car/v2
- upgrade to v2.16.1-0.20260306172652-7d2f4aceb070
Copy link
Member

@lidel lidel left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you @ChayanDass, quick first pass review below.

Also, we need to wait for go-car release with your fix before this can be merged.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Test guidance

@ChayanDass please add tests in test/cli, e.g. in dag_test.go. The existing tests there (TestDag, TestDagImportFastProvide) and the harness helpers IPFSDagExport/IPFSDagImport show the patterns to follow.

How to create partially local DAG

Perhaps useful: to test partial DAG export/import you need multi-block test data where some blocks are missing locally. The setup is:

  1. ipfs add enough data (e.g. 300KB with --raw-leaves) to produce a root with multiple child blocks
  2. Use ipfs refs <root> to list child block CIDs
  3. Use ipfs block rm <child-cid> to remove one or more blocks from the local blockstore
  4. Now the DAG is partially local -- dag export --offline should fail, but dag export --local-only --offline should succeed and produce a partial CAR

Specific tests to add

  • TestDagExportLocalOnly -- set up a partial DAG (add content, remove a child block). Confirm dag export --offline fails. Confirm dag export --local-only --offline succeeds and the resulting CAR has fewer blocks than the full DAG (import on a second node with --pin-roots=false --stats to compare block counts).

  • TestDagExportLocalOnlyRequiresOffline -- verify that --local-only without --offline either errors or behaves as documented. Currently there is no enforcement, and without --offline the node may fetch missing blocks from the network, making --local-only misleading.

  • TestDagImportPartialCAR -- export a partial CAR with --local-only --offline, import it on a fresh node with --pin-roots=false, verify success.

  • TestDagImportLocalOnlyPinRootsConflict -- verify dag import --local-only --pin-roots errors with the expected message.

Signed-off-by: Chayan Das <01chayandas@gmail.com>
Signed-off-by: Chayan Das <01chayandas@gmail.com>
@ChayanDass ChayanDass requested a review from lidel March 11, 2026 06:42
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants