Tapes db by Boog900 · Pull Request #587 · Cuprate/cuprate

Boog900 · 2026-02-21T16:01:49Z

No description provided.

Boog900 · 2026-03-04T02:55:47Z

I think this is about done. This was a massive task but the outcome is great, the database is now small and faster both reading and writing (practically, some operations on some tables may be slower).

This is probably going to take some time to review, the old docs have all been removed as I didn't want to spend the time updating it all while everything is still changing. Its the same with the lower level tests, the higher level tests are still present.

The format of the code is still roughly the same.

Otherwise, this will error with debug builds

redsh4de

some comments

redsh4de · 2026-03-25T09:31:53Z

 #[inline]
-pub fn remove_tx(tx_hash: &TxHash, tables: &mut impl TablesMut) -> DbResult<(TxId, Transaction)> {
-    //------------------------------------------------------ Transaction data
-    let tx_id = tables.tx_ids_mut().take(tx_hash)?;


main used to have .take() which deletes the tx_hash, i dont see the remove_tx_from_dynamic_tables deleting the tx_hash from tx_ids - only key images and outputs

edit: looks like it impacts add_alt_transaction_blob in src/ops/alt_block/tx.rs as well - sees tx hash exists, skips adding the blob

Oops forgot to add the explicit removal when I move to a write batch instead of an atomic tx, should be fixed now

redsh4de

mostly nitpicks

redsh4de · 2026-03-26T20:04:08Z

-    monero_oxide::io::VarInt::write(&block_txs.len(), &mut block)
-        .expect("The number of txs per block will not exceed u64::MAX");
+    let cumulative_rct_outs = tapes
+        .read_entry(&db.block_infos, block_height as u64 - 1)?


nit, to match L215

Suggested change

.read_entry(&db.block_infos, block_height as u64 - 1)?

.read_entry(&db.block_infos, block_height.saturating_sub(1) as u64)?

redsh4de · 2026-03-26T20:59:18Z

+    let top_block_height = tapes
+        .fixed_sized_tape_len(&db.block_infos)
+        .expect("Required tape not open")
+        - 1;


style nit: could port the same defensive check from chain_height

unnecessary as genesis block always gets added but good for consistency

Suggested change

let top_block_height = tapes

.fixed_sized_tape_len(&db.block_infos)

.expect("Required tape not open")

- 1;

let chain_height = tapes

.fixed_sized_tape_len(&db.block_infos)

.expect("Required tape not open");

if chain_height == 0 {

return Err(BlockchainError::NotFound);

}

let top_block_height = chain_height - 1;

redsh4de · 2026-03-26T21:01:10Z

+        block_height(db, &tx_ro, &first_known_block_hash)?.ok_or(BlockchainError::NotFound)?;

-    let chain_height = crate::ops::blockchain::chain_height(table_block_heights)?;
+    let chain_height = crate::ops::blockchain::chain_height(db, &tapes)?;


same here

Suggested change

let chain_height = crate::ops::blockchain::chain_height(db, &tapes)?;

let chain_height = crate::ops::blockchain::chain_height(db, &tapes)?;

if chain_height == 0 {

return Err(BlockchainError::NotFound):

}

redsh4de · 2026-03-26T21:16:41Z

+    let height = u64_to_usize(
+        tapes
+            .fixed_sized_tape_len(&db.block_infos)
+            .expect("Require tape not found")


Suggested change

.expect("Require tape not found")

.expect("Required tape not found")

redsh4de · 2026-03-26T21:17:12Z

-    table_block_heights.len().map(|height| height as usize)
+    Ok(tapes
+        .fixed_sized_tape_len(&db.block_infos)
+        .expect("Required tape must exists") as usize)


here too

Suggested change

.expect("Required tape must exists") as usize)

.expect("Required tape must exist") as usize)

redsh4de · 2026-03-26T22:21:24Z

+    tapes.commit(Persistence::Buffer)?;
+
+    let mut pre_rct_numb_outputs_cache = db.pre_rct_numb_outputs_cache.lock().unwrap();
+
+    let mut tx_rw = db.fjall.batch().durability(Some(PersistMode::Buffer));
+
+    for block in blocks {
+        crate::ops::block::add_block_to_dynamic_tables(
+            db,
+            &block.block,
+            &block.block_hash,
+            block.txs.iter().map(|tx| Cow::Borrowed(&tx.tx)),
+            &mut numb_transactions,
+            &mut tx_rw,
+            &mut pre_rct_numb_outputs_cache,
+        )?;
    }
+
+    tx_rw.commit()?;


note: with graceful shutdown all errors after tapes commit, but before fjall successfully commits should be distinguished due to us being out of sync after that. BlockchainError::DatabaseOutOfSync?

handler.rs does panic on error now so not a issue rn, but graceful shutdown changes this

idea: recovery function that replays only the out of sync blocks from tapes into fjall when we get that kind of error instead instead of a full chain nuke on desync

Yeah I was going to do that originally, however full fjall nuke is much easier to handle + is not too slow. You can test it by just deleting the fjall DB but keeping the tapes.

redsh4de · 2026-03-26T22:43:27Z

+/// - The blockchain has 0 blocks => this returns `Err(BlockchainError::KeyNotFound)`
 /// - The blockchain has 1 block (height 0) => this returns `Ok(0)`
 /// - The blockchain has 2 blocks (height 1) => this returns `Ok(1)`
 ///
 /// Note that in cases where no blocks have been written to the
-/// database yet, an error is returned: `Err(RuntimeError::KeyNotFound)`.
+/// database yet, an error is returned: `Err(BlockchainError::KeyNotFound)`.


KeyNotFound -> NotFound

hinto-janai · 2026-04-13T18:53:01Z

nit: I don't think you care to change this but in the storage/ diffs there are many u32/u64/usize as ops that could use cuprate_helper::cast already in scope.

hinto-janai · 2026-04-13T19:09:25Z

+    // Split the blocks at the point the pruning stripe changes.
+    let start_height = blocks[0].height;
+    let first_block_pruning_seed = cuprate_pruning::DecompressedPruningSeed::new(
+        cuprate_pruning::get_block_pruning_stripe(start_height, usize::MAX, 3).unwrap(),


nit:

rg "usize::MAX, 3"

storage/blockchain/src/ops/tx.rs 259: cuprate_pruning::get_block_pruning_stripe(tx_info.height, usize::MAX, 3).unwrap(); storage/blockchain/src/ops/block.rs 111: cuprate_pruning::get_block_pruning_stripe(start_height, usize::MAX, 3).unwrap(), 143: cuprate_pruning::get_block_pruning_stripe(blocks[0].height, usize::MAX, 3).unwrap(); 405: let stripe = cuprate_pruning::get_block_pruning_stripe(block_height, usize::MAX, 3).unwrap();

// cuprate_pruning pub fn get_default_block_pruning_stripe(start_height) -> u32 { get_block_pruning_stripe( block_height, usize::MAX, CRYPTONOTE_PRUNING_LOG_STRIPES ).unwrap() }

hinto-janai · 2026-04-13T19:21:23Z


-/// Adds a [`VerifiedTransactionInformation`] from an alt-block
-/// if it is not already in the DB.
+/// Adds a [`VerifiedTransactionInformation`] from an alt-block if it is not already in the DB.


nit: there's probably a lot of inaccurate docs at this point; this function replaces it unconditionally.

hinto-janai · 2026-04-13T19:31:35Z

+    let cumulative_rct_outs = tapes
+        .read_entry(&db.block_infos, block_height as u64 - 1)?


#587 (comment)

Call stack:

cuprated::blockchain::manager::pop_blocks

BlockchainWriteRequest::PopBlocks

cuprate_blockchain::service::write::pop_blocks

cuprate_blockchain::ops::block::pop_blocks

AFAICT input that can pop the genesis is allowed? I think this will underflow.

hinto-janai · 2026-04-13T19:37:54Z

+/// **THIS IS NOT ATOMIC**
+///
 pub fn update_alt_chain_info(
+    db: &BlockchainDatabase,


Shouldn't this be given a tx_rw: &mut fjall::OwnedWriteBatch? It is only called from add_alt_block which has it open already.

hinto-janai · 2026-04-13T20:43:39Z

+fn pop_blocks(db: &BlockchainDatabase, numb_blocks: usize) -> ResponseResult {
+    let mut tapes = db.linear_tapes.truncate();
+    let mut tx_rw = db.fjall.batch();
+
+    // flush all the current alt blocks as they may reference blocks to be popped.
+    crate::ops::alt_block::flush_alt_blocks(db)?;
+
+    // generate a `ChainId` for the popped blocks.
+    let old_main_chain_id = ChainId(rand::random());
+
+    // pop the blocks
+    for _ in 0..numb_blocks {
+        crate::ops::block::pop_block(db, Some(old_main_chain_id), &mut tx_rw, &mut tapes)?;
    }
+
+    tx_rw.commit()?;
+    tapes.commit(Persistence::SyncAll)?;
+    Ok(BlockchainResponse::PopBlocks(old_main_chain_id))


Shouldn't tx_rw be passed to flush_alt_blocks?

hinto-janai · 2026-04-13T21:15:00Z

+    pub(crate) linear_tapes: Tapes,
+    pub(crate) fjall: fjall::Database,
+
+    pub(crate) block_heights: fjall::Keyspace,


Optional but perhaps (K,V) and their encoding should be documented since the types/tables are gone, even monerod's DB is easier to understand at a glance now 😭

Suggested change

pub(crate) block_heights: fjall::Keyspace,

/// K = `[u8; 32]` (block hash), V = [`usize`], encoding = little endian.

pub(crate) block_heights: fjall::Keyspace,

hinto-janai · 2026-04-13T22:31:50Z

-            .build()
+        cuprate_blockchain::config::Config {
+            blob_dir: path_with_network(&self.fs.fast_data_directory, self.network),
+            index_dir: path_with_network(&self.fs.slow_data_directory.clone(), self.network),


nit:

Suggested change

index_dir: path_with_network(&self.fs.slow_data_directory.clone(), self.network),

index_dir: path_with_network(&self.fs.slow_data_directory, self.network),

cache_sizes below can be Copy but meh

hinto-janai · 2026-04-13T22:37:30Z

        /// | macOS   | "/Users/Alice/Library/Application Support/Cuprate/" |
        /// | Linux   | "/home/alice/.local/share/cuprate/"                 |
-        pub data_directory: PathBuf,
+        pub fast_data_directory: PathBuf,


Is there a reason to split this into two? They're both the same path.

hinto-janai · 2026-04-13T22:50:20Z

+    db.tx_infos.insert(tx_hash, bytemuck::bytes_of(&tx_info))?;
+
+    if !db.tx_blobs.contains_key(tx_hash)? {
+        db.tx_infos.remove(tx_hash)?;


Shouldn't this be through a OwnedWriteBatch?

hinto-janai · 2026-04-13T23:31:01Z

I still don't have a good mental model of the atomic-ness between fjall/tapes, there are separate transactions everywhere, sometimes directly into a keyspace, sometimes in different commit orders. It doesn't seem like we can have a deterministic safe sync mode (as much as monerod is). Is this just a tradeoff being taken?

Boog900 · 2026-04-15T14:39:26Z

The DB is now 2 systems that are for ACID purposes independent. The tapes DB will do 1 update atomically, so will fjall, the 2 updates combined are not atomic.

This does lead to issues with the fact they could get out of sync. On start-up we solve this by using the tapes DB as the source of truth, if fjall is out of sync, we rebuild it from the tapes (doesn't take that long for me). On some requests where we need an fjall and tapes TX to be in sync we should check if they are then just open another set of txs until they are in sync (this is not done yet).

The only exception is alt blocks, for which fjall is not atomic. This is ok as we clear alt blocks on startup anyway and request should be built to handle only some alt block data for a given block being present.

This all means Cuprated should be able to have a safe sync mode.

Boog900 added 3 commits February 16, 2026 15:08

init new db

70a33aa

Merge branch 'main' into new-db

4c469a8

fix some issues

40933a8

Boog900 added 2 commits February 21, 2026 22:56

remove x86_64 specific code

f8766d6

working clippy & tests

711af23

github-actions bot added the A-p2p Area: Related to P2P. label Feb 24, 2026

Boog900 added 4 commits February 24, 2026 23:23

todo docs

7c6c0c8

typos

babe62e

remove old doc tests

b5c40f8

remove redb test

5dd77e5

github-actions bot added the A-ci Area: Related to CI. label Feb 25, 2026

Boog900 added 3 commits February 25, 2026 15:28

remove empty doc tests

b424cd6

fix all tests

6f0db57

fix hack errors

5b5a0be

hinto-janai added this to the cuprated v0.0.9 milestone Feb 26, 2026

Boog900 added 3 commits February 26, 2026 23:56

add required features to txpool

0c5ed11

clean up docs and uncomment code

8364266

fix bug in alt blocks and more clean up

5fdf47d

Boog900 marked this pull request as ready for review March 4, 2026 02:56

Boog900 requested a review from hinto-janai March 4, 2026 02:56

Boog900 added 9 commits March 4, 2026 12:53

add required features for cargo hack

87b4255

add config option for setting memory limit

c22754d

sort imports correctly

3e06505

update tapes

5d53901

fix syncer restarting loop

3a790dd

Merge branch 'main' into new-db

f45705b

fmt

9475c26

set fjall to 25% of memory by default

5599378

commit tapes changes before changing fjall when adding blocks

c024ed7

Otherwise, this will error with debug builds

redsh4de reviewed Mar 26, 2026

View reviewed changes

Boog900 added 2 commits March 26, 2026 13:42

redsh4de review fixes

a3d50b4

redsh4de review fixes 2

8f9f2a9

redsh4de reviewed Mar 26, 2026

View reviewed changes

Comment thread storage/blockchain/src/ops/alt_block/tx.rs Outdated

always write tx blobs to alt table

f93b430

redsh4de reviewed Mar 26, 2026

View reviewed changes

redsh4de approved these changes Mar 27, 2026

View reviewed changes

hinto-janai reviewed Apr 13, 2026

View reviewed changes

	.read_entry(&db.block_infos, block_height as u64 - 1)?
	.read_entry(&db.block_infos, block_height.saturating_sub(1) as u64)?

-    let top_block_height = tapes
-        .fixed_sized_tape_len(&db.block_infos)
-        .expect("Required tape not open")
-        - 1;
+    let chain_height = tapes
+      .fixed_sized_tape_len(&db.block_infos)
+      .expect("Required tape not open");
+    if chain_height == 0 {
+        return Err(BlockchainError::NotFound);
+    }
+    let top_block_height = chain_height - 1;

	.expect("Require tape not found")
	.expect("Required tape not found")

	.expect("Required tape must exists") as usize)
	.expect("Required tape must exist") as usize)

		let cumulative_rct_outs = tapes
		.read_entry(&db.block_infos, block_height as u64 - 1)?

	pub(crate) block_heights: fjall::Keyspace,
	/// K = `[u8; 32]` (block hash), V = [`usize`], encoding = little endian.
	pub(crate) block_heights: fjall::Keyspace,

	index_dir: path_with_network(&self.fs.slow_data_directory.clone(), self.network),
	index_dir: path_with_network(&self.fs.slow_data_directory, self.network),

Conversation

Boog900 commented Feb 21, 2026

Uh oh!

Boog900 commented Mar 4, 2026

Uh oh!

redsh4de left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

redsh4de left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

hinto-janai commented Apr 13, 2026

Uh oh!

Boog900 commented Apr 15, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants