Skip to content

feat: add ILM (lifecycle/tiering) and bucket replication commands#60

Merged
overtrue merged 5 commits intomainfrom
feat/ilm-replication
Mar 20, 2026
Merged

feat: add ILM (lifecycle/tiering) and bucket replication commands#60
overtrue merged 5 commits intomainfrom
feat/ilm-replication

Conversation

@cxymds
Copy link
Contributor

@cxymds cxymds commented Mar 20, 2026

Type of Change

  • New Feature
  • Bug Fix
  • Documentation
  • Performance Improvement
  • Test/CI
  • Refactor
  • Other:

Summary of Changes

Add three major feature groups to the rc CLI: lifecycle management (ILM), storage tiering, and bucket replication. These features follow the MinIO mc CLI UX conventions and communicate with the RustFS backend via standard S3 APIs and Admin APIs.

1. Lifecycle Rule Management (rc ilm rule)

  • add — Add lifecycle rules with expiry/transition/noncurrent-version settings, prefix filter
  • edit — Modify existing rules by ID (partial update of provided fields)
  • list — Display rules as table (ID, Status, Prefix, Expiry, Transition, Storage Class) or JSON
  • remove — Remove rules by ID or all rules
  • export — Export full lifecycle configuration as JSON
  • import — Import lifecycle configuration from JSON file

Supported rule properties: --expiry-days, --expiry-date, --transition-days, --transition-date, --storage-class, --noncurrent-expiry-days, --noncurrent-transition-days, --noncurrent-transition-storage-class, --prefix, --expired-object-delete-marker, --newer-noncurrent-versions, --disable

2. Storage Tier Management (rc ilm tier)

  • add — Add remote storage tiers (S3, RustFS, MinIO, Aliyun, Tencent, Huaweicloud, Azure, GCS, R2)
  • edit — Update tier credentials
  • list — List all configured tiers
  • info — Show tier details/statistics
  • remove — Remove tiers (with --force option)

Tier config matches the RustFS admin API JSON format exactly (polymorphic structure with type + sub-config per tier type).

3. Object Restore (rc ilm restore)

  • Restore transitioned (archived) objects with configurable retention days

4. Bucket Replication (rc replicate)

  • add — Add replication rules (orchestrates both S3 and Admin APIs: registers remote target → gets ARN → builds replication rule)
  • update — Modify existing replication rules by ID
  • list — Display replication rules as table or JSON
  • status — Show replication metrics via Admin API
  • remove — Remove rules by ID or all rules
  • export/import — Full replication configuration JSON export/import

Architecture

The implementation follows the existing three-layer pattern:

rc-core (domain types + traits) ← rc-s3 (SDK implementations) ← rustfs-cli (CLI commands)

Core Layer (rc-core)

  • New modules: lifecycle.rs, replication.rs, admin/tier.rs
  • Extended ObjectStore trait: +7 methods (get/set/delete lifecycle, restore, get/set/delete replication)
  • Extended AdminApi trait: +9 methods (5 tier ops + 4 replication target ops)
  • Extended Capabilities: added lifecycle and replication fields

S3 Layer (rc-s3)

  • S3Client: Implements lifecycle methods via aws-sdk-s3 (PutBucketLifecycleConfiguration, GetBucketLifecycleConfiguration, etc.), replication via standard S3 replication APIs, and RestoreObject
  • AdminClient: Implements tier management via HTTP+SigV4 (/rustfs/admin/v3/tier*) and remote target management (/rustfs/admin/v3/set-remote-target, etc.)

CLI Layer (rustfs-cli)

  • New command modules: ilm/mod.rs, ilm/rule.rs, ilm/tier.rs, ilm/restore.rs, replicate.rs
  • Follows existing patterns (event.rs for S3 subcommands, quota.rs for admin subcommands)
  • All commands support --json output, --force capability bypass, and human-readable table output

Files Changed

New Files (8)

File Purpose
crates/core/src/lifecycle.rs Lifecycle domain types
crates/core/src/replication.rs Replication + BucketTarget domain types
crates/core/src/admin/tier.rs Tier config types (matches RustFS admin API JSON)
crates/cli/src/commands/ilm/mod.rs ILM top-level dispatcher
crates/cli/src/commands/ilm/rule.rs Lifecycle rule subcommands
crates/cli/src/commands/ilm/tier.rs Tier management subcommands
crates/cli/src/commands/ilm/restore.rs Object restore command
crates/cli/src/commands/replicate.rs Replication subcommands

Modified Files (10)

File Changes
crates/core/src/lib.rs Added lifecycle/replication modules + re-exports
crates/core/src/admin/mod.rs Added tier module, 9 new AdminApi trait methods
crates/core/src/traits.rs Added 7 new ObjectStore methods, extended Capabilities
crates/s3/src/client.rs Implemented 7 ObjectStore methods
crates/s3/src/admin.rs Implemented 9 AdminApi methods
crates/s3/src/capability.rs Added lifecycle/replication capability checks
crates/cli/src/commands/mod.rs Registered ilm + replicate command modules
crates/cli/tests/help_contract.rs Added ~15 help contract test cases
README.md Added ILM, tier, and replication documentation
.gitignore Added CLAUDE.md

Checklist

  • cargo fmt --all --check passes
  • cargo clippy --workspace -- -D warnings passes (zero warnings)
  • cargo test --workspace passes (all 481+ tests)
  • No changes to CLI/JSON/config contracts (exit codes unchanged)
  • New behaviors have unit tests
  • Each new command module has unit tests with exit code scenarios
  • Help contract tests added for all new commands
  • No sensitive information in logs
  • README updated with new commands documentation

Validation

Verified against live RustFS instance (rustfs/rustfs:latest):

# Lifecycle rules
rc ilm rule add local/test-bucket --expiry-days 30 --prefix "logs/"
rc ilm rule list local/test-bucket         # ✓ table output
rc ilm rule list local/test-bucket --json  # ✓ JSON output
rc ilm rule export local/test-bucket       # ✓ JSON export
rc ilm rule remove local/test-bucket --id rule-xxx  # ✓ removal

# Tier management
rc ilm tier list local  # ✓ empty list

# Replication
rc replicate list local/test-bucket  # ✓ no config found
rc replicate --help                  # ✓ all subcommands listed

# All existing commands
rc ls local/           # ✓ unchanged
rc admin info cluster local  # ✓ unchanged

Impact

  • Breaking change (compatibility)
  • Requires doc/config/deployment update
  • Other impact: adds 3 new top-level command groups (ilm, replicate) with no changes to existing commands

Add three major feature groups to the rc CLI:

1. Lifecycle rule management (rc ilm rule add/edit/list/remove/export/import)
2. Storage tier management (rc ilm tier add/edit/list/info/remove)
3. Object restore (rc ilm restore)
4. Bucket replication (rc replicate add/update/list/status/remove/export/import)

Core changes:
- Add LifecycleRule, LifecycleConfiguration types to rc-core
- Add TierConfig, TierType, TierCreds types matching RustFS admin API format
- Add ReplicationConfiguration, ReplicationRule, BucketTarget types
- Extend ObjectStore trait with lifecycle/replication/restore methods
- Extend AdminApi trait with tier and replication target methods
- Add lifecycle and replication fields to Capabilities
- Implement all ObjectStore methods in S3Client (aws-sdk-s3)
- Implement all AdminApi methods in AdminClient (HTTP+SigV4)
- Add help contract tests for all new commands
- Update README with ILM, tier, and replication documentation
@cxymds cxymds requested a review from overtrue March 20, 2026 01:55
Copy link
Contributor

@overtrue overtrue left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for putting this feature set together. I found a few correctness issues in the new lifecycle and replication flows that look blocking before merge.

  • ilm rule add can overwrite an existing lifecycle configuration after any failed read.
  • The lifecycle and replication read/modify/write paths do not preserve tag-based filters from existing rules.
  • replicate export and replicate import do not round-trip the admin-side remote target state.
  • Removing replication rules leaves the registered remote targets behind.
  • replicate update accepts target-level flags that are never applied.

I also ran cargo test --workspace; it passes, but the new coverage is mostly help and path parsing and does not exercise these stateful round-trips.

Copy link
Contributor

@overtrue overtrue left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Plz check the review comments

@cxymds
Copy link
Contributor Author

cxymds commented Mar 20, 2026

Addressed the requested review changes and pushed the updates to feat/ilm-replication.

Summary:

  • stopped ilm rule add from overwriting lifecycle config after read failures
  • preserved lifecycle and replication tag-based filters through read/modify/write round-trips
  • made replication export/import include admin remote-target state and ARN remapping
  • cleaned up remote targets during replication removal
  • applied target-level update flags through the admin remote-target API

Validation run locally:

  • cargo fmt --all
  • cargo clippy --workspace -- -D warnings
  • cargo test --workspace

@overtrue overtrue merged commit 4d1528b into main Mar 20, 2026
15 checks passed
@overtrue overtrue deleted the feat/ilm-replication branch March 20, 2026 09:14
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants