Skip to content

Commit cd4854d

Browse files
author
molty3000
committed
docs: add SaveEmbedder/LoadEmbedder to README and landing page
The v1.2.0 release introduced embedder state persistence (rp_persistence.go) but the feature was undocumented in both the README and the website. This closes the gap: README.md: - New 'Embedder Persistence' subsection under Persistence with full Go example (fit → save → load → embed) - Added SaveEmbedder / LoadEmbedder to the RandomProjections API bullet list docs/index.html: - New 'Embedder state persistence' card in the Persistence section with syntax-highlighted code block
1 parent 1b72235 commit cd4854d

2 files changed

Lines changed: 43 additions & 0 deletions

File tree

README.md

Lines changed: 32 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -61,6 +61,8 @@ The `Embedder` interface lets you swap backends: bring your own OpenAI, Ollama,
6161

6262
## Persistence
6363

64+
### Store Persistence
65+
6466
```go
6567
// Save to disk (gob — compact binary)
6668
store.Save("/data/vectors.db")
@@ -75,6 +77,34 @@ restored.Load("/data/vectors.db")
7577

7678
Gob-encoded stores are compact (~4 bytes per float32 + overhead). For a 10K × 1536d store, expect ~60 MB on disk and ~200ms save/load times.
7779

80+
### Embedder Persistence
81+
82+
```go
83+
// Fit an embedder on your corpus
84+
rp := vector.NewRandomProjections(256)
85+
rp.Fit([]string{
86+
"machine learning is fascinating",
87+
"deep neural networks transform AI",
88+
"the weather today is sunny",
89+
})
90+
91+
// Save the embedder state — vocab, projection matrix, dimensions
92+
if err := rp.SaveEmbedder("/data/embedder.gob"); err != nil {
93+
log.Fatal(err)
94+
}
95+
96+
// Later — load without refitting
97+
restored, err := vector.LoadEmbedder("/data/embedder.gob")
98+
if err != nil {
99+
log.Fatal(err)
100+
}
101+
102+
// Embeddings are deterministic — same text, same vector
103+
v, _ := restored.Embed("machine learning")
104+
```
105+
106+
`SaveEmbedder` preserves the vocabulary, projection matrix, output dimension, and scaling factor. `LoadEmbedder` reconstructs a ready-to-use `*RandomProjections` that produces identical vectors to the original.
107+
78108
## API
79109

80110
### Vector Type
@@ -139,6 +169,8 @@ Johnson-Lindenstrauss sparse random projection (Achlioptas 2003). Projects token
139169
- `Embed(text string) (Vector, error)` — embed text (L2-normalized output)
140170
- `VocabSize() int` — number of unique tokens in vocabulary
141171
- `Dims() int` — output dimensionality
172+
- `SaveEmbedder(path string) error` — persist embedder state to gob file
173+
- `LoadEmbedder(path string) (*RandomProjections, error)` — restore embedder from gob file
142174

143175
## Performance
144176

docs/index.html

Lines changed: 11 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -372,6 +372,17 @@ <h3>JSON — human-readable</h3>
372372
<div class="code-mini"><pre>store.<span class="fn">SaveJSON</span>(<span class="str">"/data/vectors.json"</span>)
373373
restored.<span class="fn">LoadJSON</span>(<span class="str">"/data/vectors.json"</span>)</pre></div>
374374
</div>
375+
376+
<div class="usecase">
377+
<div>
378+
<h3>Embedder state persistence</h3>
379+
<p>Save and restore the <code>RandomProjections</code> embedder — vocabulary, projection matrix, dimensions. No refitting needed across restarts.</p>
380+
</div>
381+
<div class="code-mini"><pre>rp.<span class="fn">SaveEmbedder</span>(<span class="str">"/data/embedder.gob"</span>)
382+
383+
restored, _ := <span class="pkg">vector</span>.<span class="fn">LoadEmbedder</span>(<span class="str">"/data/embedder.gob"</span>)
384+
<span class="cmt">// Same text → identical vector</span></pre></div>
385+
</div>
375386
</div>
376387
</section>
377388

0 commit comments

Comments
 (0)