all keys in memory

[gra-moore]

Badger talks about keys being in-memory when doing key only iteration "It is several order of magnitudes faster than regular iteration, because it involves access to the LSM-tree only, which is usually resident entirely in RAM"

Obviously at some point there isnt enough memory, but is there some strategy for having "all" keys in memory in the skiplist structure. See issue on key compression as well.

[Drawaes]

The skiplist always is all memory. If we come up with a strategy for moving the values off to a value log then the skiplist will be able to be a lot larger before we require a flush.

[gra-moore]

Sure, but once it is flushed they are not in memory? I read the badger thing to mean that in general they tried to keep as many of the keys in memory at any one time, including after they have been flushed to disk.