Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
34 commits
Select commit Hold shift + click to select a range
a4b9dc3
refactor: update database schema
Mar 7, 2026
540c6fc
refactor: add updated_at trigger for container, chunk, logical_file
Mar 7, 2026
2d22b93
refactor: update timestamp fields to use TIMESTAMPZ
Mar 7, 2026
6c0bfe4
refactor : incorrect timestamptz
Mar 7, 2026
df0d929
refactor : reorg project into packages
Mar 7, 2026
c0e0f86
refactor: update .gitignore and add storage management functions
Mar 8, 2026
6485e7a
refactor: reorganize Dockerfile and main application structure
Mar 8, 2026
a0e04ee
refactor: enhance garbage collection and statistics reporting with qu…
Mar 8, 2026
d77884b
refactor: implement system recovery process and enhance container man…
Mar 8, 2026
e6b9b90
refactor: enhance logical file statistics reporting with additional s…
Mar 9, 2026
22d3d02
refactor: enhance claimLogicalFile and claimChunk functions to return…
Mar 9, 2026
724f8c9
refactor: improve transaction handling in claimLogicalFile and claimC…
franchoy Mar 10, 2026
1f0aff0
refactor: rename StorageDir to ContainersDir
franchoy Mar 10, 2026
79e7445
refactor: enhance chunk handling and statistics reporting
Mar 10, 2026
a533a3d
refactor: enhance smoke test with garbage collection and file removal…
Mar 10, 2026
1c54ee5
refactor: improve stats
Mar 10, 2026
b8f5057
refactor: add retry statistics
Mar 10, 2026
605fe24
refactor: add container max size handling and recovery tests
Mar 10, 2026
91c26da
refactor : rename db local variable to dbconn to avoid shadow db package
franchoy Mar 11, 2026
c55ee41
refactor : stats deadratio by healthyContainer
franchoy Mar 11, 2026
4507406
refactor : stats remove unused chunkstats
franchoy Mar 11, 2026
6dc3913
refactor : gitignore filter coldkeep build file
franchoy Mar 11, 2026
f408b78
Remove coldkeep binary from version control
franchoy Mar 11, 2026
d0d12e8
refactor : gc wrong where clause
franchoy Mar 11, 2026
a6813cd
refactor : system_recovery add control to not fail if container dir d…
franchoy Mar 11, 2026
0d32f3c
refactor : move version to 0.2
franchoy Mar 11, 2026
10008d0
refactor : v0.2 change details for readme and changelog
franchoy Mar 12, 2026
b97ba0f
refactor : system_recovery insert orphan query issue and log info
franchoy Mar 12, 2026
cee35a8
refactor : system_recovery log info
franchoy Mar 12, 2026
f2f675b
refactor : readme change to experimental
Mar 14, 2026
ffc9ad7
refactor : integration_test dbconn name issue
Mar 14, 2026
167ed05
refactor : integration_test dbconn name issue 2
Mar 14, 2026
e04f9bf
refactor : integration_test dbconn name issue 3
Mar 14, 2026
8c8192e
refactor : integration_test hex.EncodeToString issue
Mar 14, 2026
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 2 additions & 2 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -20,15 +20,15 @@ go.work
# =========================
# coldkeep specific binaries
# =========================
coldkeep
app/coldkeep
coldkeep

# binaries generated during development
*.exe
*.out
# =========================
# Local storage directories
# =========================
storage/
storage/containers/
storage/output/
pgdata/
Expand Down
51 changes: 48 additions & 3 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,13 +2,58 @@

All notable changes to this project will be documented in this file.

This project follows a lightweight, research-friendly versioning
approach. Version numbers indicate conceptual milestones rather than
This project follows a lightweight, prototype-friendly versioning
approach.

Version numbers indicate conceptual milestones rather than
production stability.

------------------------------------------------------------------------

## \[0.1.0\] - 2026-02-24
## [0.2.0]- 2026-03-11

Crash-consistency foundation for the storage engine

### Added

- Logical file lifecycle management
- Chunk lifecycle management
- Retry handling for interrupted operations
- Startup recovery system
- Container quarantine mechanism
- Extended storage statistics
- Smoke test improvements
- Durable container writes with fsync to guarantee on-disk persistence

### Improved

- Concurrent file ingestion
- Garbage collection safety
- Operational observability

### Notes

This version introduces the core reliability model for the storage
engine.

The on-disk format and APIs may still change in future releases.

### Known Limitations

- Basic crash recovery exists, but full end-to-end crash consistency across
filesystem and database layers is still evolving.
- No encryption at rest or in transit.
- No authentication or authorization model.
- Whole-container compression is not suitable for efficient random-access
restores.
- Concurrency behavior has not been heavily stress-tested under high parallel
workloads.
- No background integrity verification or automatic container scrubbing.
- On-disk storage format may change before v1.0.

------------------------------------------------------------------------

## [0.1.0] - 2026-02-24

Initial public research prototype (POC).

Expand Down
14 changes: 9 additions & 5 deletions app/Dockerfile → Dockerfile
Original file line number Diff line number Diff line change
Expand Up @@ -6,16 +6,21 @@ WORKDIR /app
# Install git (required for some Go modules)
RUN apk add --no-cache git

# Copy module files first (better layer caching)
COPY go.mod go.sum ./
RUN go mod download

COPY app/ .
RUN CGO_ENABLED=0 GOOS=linux GOARCH=amd64 go build -o coldkeep .
# Copy source
COPY cmd/ ./cmd/
COPY internal/ ./internal/

# Build binary
RUN CGO_ENABLED=0 GOOS=linux GOARCH=amd64 go build -o coldkeep ./cmd/coldkeep


# ---------- Runtime ----------
FROM alpine:3.19

# Install runtime dependencies
RUN apk add --no-cache \
ca-certificates \
bash
Expand All @@ -25,9 +30,8 @@ WORKDIR /app
# Copy binary
COPY --from=builder /app/coldkeep /usr/local/bin/coldkeep

# Copy scripts for smoke/testing
# Copy scripts
COPY scripts/ ./scripts/

# Default command
ENTRYPOINT ["coldkeep"]
CMD ["stats"]
Loading
Loading