Skip to content

GCAS Erasure Coding #38

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wk-y wants to merge 4 commits into
base: main
Choose a base branch
from
Open

GCAS Erasure Coding #38

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
4 commits
Select commit Hold shift + click to select a range
8748636
gcas: switch to garbage collection to handle deleted chunks
wk-y May 12, 2026
601f96e
gcas: treat node already having hash as success in gcas.Put
wk-y May 16, 2026
f48094f
gcas: only use internal db to list
wk-y May 16, 2026
d01fec9
gcas: implement error-correction striping
wk-y May 17, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions go.mod
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -44,6 +44,7 @@ require (
github.com/google/uuid v1.6.0 // indirect
github.com/json-iterator/go v1.1.12 // indirect
github.com/klauspost/cpuid/v2 v2.3.0 // indirect
github.com/klauspost/reedsolomon v1.14.0 // indirect
github.com/leodido/go-urn v1.4.0 // indirect
github.com/modern-go/concurrent v0.0.0-20180306012644-bacd9c7ef1dd // indirect
github.com/modern-go/reflect2 v1.0.2 // indirect
Expand Down
2 changes: 2 additions & 0 deletions go.sum
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -61,6 +61,8 @@ github.com/json-iterator/go v1.1.12 h1:PV8peI4a0ysnczrg+LtxykD8LfKY9ML6u2jnxaEnr
github.com/json-iterator/go v1.1.12/go.mod h1:e30LSqwooZae/UwlEbR2852Gd8hjQvJoHmT4TnhNGBo=
github.com/klauspost/cpuid/v2 v2.3.0 h1:S4CRMLnYUhGeDFDqkGriYKdfoFlDnMtqTiI/sFzhA9Y=
github.com/klauspost/cpuid/v2 v2.3.0/go.mod h1:hqwkgyIinND0mEev00jJYCxPNVRVXFQeu1XKlok6oO0=
github.com/klauspost/reedsolomon v1.14.0 h1:5YSZeclzSYg5nl349+GDG/agDtQ6MZiwUYXvVKN1Jx0=
github.com/klauspost/reedsolomon v1.14.0/go.mod h1:yjqqjgMTQkBUHSG97/rm4zipffCNbCiZcB3kTqr++sQ=
github.com/leodido/go-urn v1.4.0 h1:WT9HwE9SGECu3lg4d/dIA+jxlljEa1/ffXKmRjqdmIQ=
github.com/leodido/go-urn v1.4.0/go.mod h1:bvxc+MVxLKB4z00jd1z+Dvzr47oO32F/QSNjSBOlFxI=
github.com/lib/pq v1.10.9 h1:YXG7RB+JIjhP29X+OtkiDnYaXQwpS4JEWq7dtCCRUEw=
Expand Down
2 changes: 1 addition & 1 deletion server/gcas/db.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -19,7 +19,7 @@ const pragmaString = `PRAGMA journal_mode=WAL;
//go:embed migrations/*.sql
var migrations embed.FS

const dbVersion = 1
const dbVersion = 2

func OpenDB(dbPath string) (*sql.DB, error) {
return OpenDBWithVersion(dbPath, dbVersion)
Expand Down
4 changes: 4 additions & 0 deletions server/gcas/gcas.go
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,5 +1,7 @@
package gcas

import "context"

// GCAS is a content-addressible storage service that combines multiple CAS nodes into a single CAS.
// It uses erasure coding to provide efficient redundancy.
// The erasure coding used is Reed-Solomon coding.
Expand All @@ -8,6 +10,8 @@ type GCAS interface {
AddNode(node NamedCAS)
RemoveNode(nodeName string)
ReplaceNode(node NamedCAS) // replaces the node with the same name with the new node
RunMaintenance(ctx context.Context) error
Repair(ctx context.Context) error
}

type NamedCAS interface {
Expand Down
Loading