feat(cleanup): add progress logging for manifest reads and delete phase

[jan-exa]

Summary

Adds progress logging to the cleanup operation so operators can monitor long-running cleanup jobs instead of waiting blindly for hours.

Three new log points:

1. Manifest read progress — logs every ~5% of manifests processed:

   [INFO] Manifest progress: 500/9703 (5.2%)
   [INFO] Manifest progress: 1000/9703 (10.3%)
   ...
   [INFO] Successfully processed all 9703 manifests
   

2. Inspection summary — after manifest processing, logs what was found:

   [INFO] Inspection summary: 7926 old manifests, 9706 referenced data files,
          75045 referenced deletion files, 0 referenced tx files, 0 referenced indices
   

3. Delete progress — logs every 1000 files submitted for deletion, plus completion:

   [INFO] Delete progress: 1000 files submitted
   [INFO] Delete progress: 2000 files submitted
   ...
   [INFO] Delete phase complete: 35288 files deleted (7926 old manifests)
   

All logging uses info! level — visible with default RUST_LOG settings.

Review & Testing Checklist for Human

• Verify log interval makes sense for your dataset size (logs every max(total/20, 100) manifests)
• After merge, rebuild the pylance wheel and upload to S3 for the Databricks cleanup job

Notes

• No functional changes — only adds logging
• Uses AtomicU64 for lock-free progress counting across concurrent manifest reads
• The log interval for manifests is max(total/20, 100) — for 9703 manifests this logs ~20 times during the read phase

Link to Devin session: https://app.devin.ai/sessions/bb810ab5769542e0a12b08c2505d2ae1
Requested by: @jan-exa

[devin-ai-integration]

🤖 Devin AI Engineer

I'll be helping with this pull request! Here's what you should know:

✅ I will automatically:

• Address comments on this PR. Add '(aside)' to your comment to have me ignore it.
• Look at CI failures and help fix them

Note: I can only respond to comments from users who have write access to this repository.

⚙️ Control Options:

• Disable automatic comment and CI monitoring

[github-actions]

ACTION NEEDED
Lance follows the Conventional Commits specification for release automation.

The PR title and description are used as the merge commit message. Please update your PR title and description to match the specification.

For details on the error please inspect the "PR Title Check" action.