Remote Execution Failure:
Failed Precondition: status: NotFound, message: "Object xxxx not found in either fast or slow store. If using multiple workers, ensure all workers share the same CAS storage path
When using an Existence Cache that wraps a RedisStore and Redis is configured to use an evicting maxmemory-policy (e.g. allkeys-lru), the existence cache keeps reporting blobs present after Redis evicts the keys causing build failures as shown above.
Seems like after the change in #1947 the remove-callbacks in redis_store.rs makes it a silent no-op, refer comment here.
Seems like that comment may not be true as already noted here.
When using an Existence Cache that wraps a RedisStore and Redis is configured to use an evicting maxmemory-policy (e.g. allkeys-lru), the existence cache keeps reporting blobs present after Redis evicts the keys causing build failures as shown above.
Seems like after the change in #1947 the remove-callbacks in redis_store.rs makes it a silent no-op, refer comment here.
Seems like that comment may not be true as already noted here.