Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
56 changes: 43 additions & 13 deletions docs/ARCHITECTURE.md
Original file line number Diff line number Diff line change
Expand Up @@ -66,7 +66,7 @@
| `components/pipeline/StageCard.tsx` | Visualizza singolo stage (token, retry info) |
| `components/document/ConfigDrawer.tsx` | Drawer config pipeline: mode, lingue, stage, persona, glossary |
| `components/layout/Header.tsx` | Project/pipeline selector |
| `components/workspace/WorkspaceHome.tsx` | Dashboard workspace: switch/create/config workspace, progetti, preset Phrase Memory workspace |
| `components/workspace/WorkspaceHome.tsx` | Dashboard workspace: switch/create/config workspace, progetti, configurazione extractor Phrase Memory |
| `components/workspace/WorkspaceWizard.tsx` | Primo avvio: crea il primo workspace reale |

---
Expand All @@ -78,8 +78,8 @@ Glossa 2.0 separa tre livelli:
| Livello | Dove si configura | Cosa contiene |
|---|---|---|
| App | `SettingsModal` | Provider/API key, Ollama, segmentazione default, layout, backup/pricing |
| Workspace traduzioni | `WorkspaceHome` | Progetti di traduzione, modello embedding, preset Phrase Memory, memoria condivisa |
| Pipeline/progetto | `ConfigDrawer` | Lingue, persona, stage, prompt, glossario assegnato, toggle Phrase Memory |
| Workspace traduzioni | `WorkspaceHome` | Progetti di traduzione, modello embedding, extractor Phrase Memory, memoria condivisa |
| Pipeline/progetto | `ConfigDrawer` | Lingue, persona, stage, prompt, glossario assegnato, toggle/search Phrase Memory |

Il workspace attuale Γ¨ specifico per l'area **Traduzioni**. Biblioteca e Trascrizioni sono future macro-aree separate; non devono condividere implicitamente la Phrase Memory delle traduzioni.

Expand Down Expand Up @@ -204,7 +204,7 @@ flushPendingTokenBatch() β†’ un solo setState per frame (O(1) chunk update)
| File | ResponsabilitΓ  |
|---|---|
| `src-tauri/src/lib.rs` | Entry point Tauri, registrazione comandi, StreamRegistry state |
| `src-tauri/src/llm/pipeline.rs` | Comandi Tauri: run_stage, run_stage_stream, judge_translation, run_coherence_for_chunk, preflight_pipeline, compute_blobs, cancel_stream |
| `src-tauri/src/llm/pipeline.rs` | Comandi Tauri: run_stage, run_stage_stream, judge_translation, run_coherence_for_chunk, preflight_pipeline, compute_blobs, extract_phrase_memory_pairs, cancel_stream |
| `src-tauri/src/llm/blobs.rs` | Algoritmo assegnazione blob (globale vs finestre) |
| `src-tauri/src/llm/prompts.rs` | Costruzione prompt 3-block, glossario, markdown rules, persona |
| `src-tauri/src/llm/provider.rs` | Trait LlmProvider, struct LlmRequest |
Expand All @@ -216,6 +216,38 @@ flushPendingTokenBatch() β†’ un solo setState per frame (O(1) chunk update)

---

## Phrase Memory

**Configurazione:**
- Workspace: `memoryExtractorProvider`, `memoryExtractorModel`, `memoryExtractorPrompt`; prompt templates con context `memory`.
- Pipeline: `usePhraseMemory`, `autoSearchPhraseMemory`, `phraseMemorySimilarityThreshold`, `phraseMemoryMaxResults`.

**Salvataggio memoria:**
```
Chunk originale + draft/traduzione finale
↓
phraseMemoryService.savePhrasePairs()
↓
Tauri: extract_phrase_memory_pairs(provider, model, prompt, sourceText, targetText, languages)
↓
LLM JSON mode β†’ { pairs: [{ sourcePhrase, targetPhrase, confidence }] }
↓
Validazione verbatim frontend + backend su source e target
↓
Embedding solo su sourcePhrase
↓
vec_save_locked_phrases(..., confidence)
```

Non esiste fallback locale: se extractor, JSON parsing o validazione falliscono, il chunk non salva coppie. Le coppie vecchie e i preset vengono purgati dal bump schema perchΓ© il formato precedente non Γ¨ compatibile.

**Ricerca memoria:**
- Auto-search parte solo se `usePhraseMemory` Γ¨ attivo e `autoSearchPhraseMemory !== false`.
- Il tab Memory puΓ² sempre lanciare refresh manuale per il chunk corrente quando la memoria Γ¨ abilitata.
- La query embedding usa solo il testo sorgente del chunk; i match selezionati sono gli unici iniettati nel prompt di run/rerun.

---

## Schema DB (SQLite)

```
Expand All @@ -226,6 +258,7 @@ projects

workspaces
id, name, description, embedding_model, created_at
memory_extractor_provider, memory_extractor_model, memory_extractor_prompt
active_workspace_id vive in app_settings

pipelines ← multi-pipeline per progetto (feat/multi-pipeline)
Expand All @@ -237,6 +270,8 @@ pipelines ← multi-pipeline per progetto (feat/multi-pipeline)
persona, custom_source_language, custom_target_language
blob_budget_tokens, blob_overlap
review_provider_options JSON
use_phrase_memory, auto_search_phrase_memory
phrase_memory_similarity_threshold, phrase_memory_max_results
run_status ('idle'|'running'|'completed'|'interrupted')
last_run_config JSON (fingerprint per resume)
created_at, updated_at
Expand All @@ -256,7 +291,7 @@ glossaries / glossary_entries / project_glossaries
CRUD standard, many-to-many project↔glossary

prompt_templates
id, name, prompt, context ('stage'|'audit'|'persona')
id, name, prompt, context ('stage'|'audit'|'persona'|'memory')
default_model, default_provider

operation_logs
Expand All @@ -272,19 +307,15 @@ app_settings
phrase_memory
id, workspace_id FK, source_phrase, target_phrase
source_language, target_language, author, work, domain, tags, notes
chunk_id, project_id, embedding, created_at

phrase_memory_presets
id, workspace_id nullable, name, is_builtin, config JSON, created_at
built-in globali + custom scoped al workspace
chunk_id, project_id, confidence, embedding, created_at

source_phrase_embeddings
id, project_id, chunk_id, source_phrase, embedding, created_at
```

**Persistito vs in-memory:**
- βœ… Persistito: source, config, stage_results, translations, run_status, operation_logs
- βœ… Persistito workspace: progetti, preset Phrase Memory custom, memoria frasi
- βœ… Persistito workspace: progetti, configurazione extractor Phrase Memory, memoria frasi
- ❌ Solo in-memory: token stream real-time (ricostruito da stage_results su resume)

---
Expand Down Expand Up @@ -316,12 +347,11 @@ source_phrase_embeddings
| Area | Descrizione | PrioritΓ  |
|---|---|---|
| `hooks/usePipeline.ts` | 3 blocchi blob assembler identici da estrarre in helper condiviso | bassa (cosmesi) |
| `services/phraseMemoryService.ts` β†’ `savePhrasePairs()` | Zip silenzioso source↔target: se lo splitter produce lunghezze diverse, le frasi extra vengono scartate senza notifica all'utente. Da segnalare in UI o loggare nel dettaglio. | media |
| `src-tauri/src/llm/providers/anthropic.rs` | Supporto reasoning da rivedere (verifica integrazione, parametri, formato risposta) | alta |
| `src-tauri/src/llm/providers/deepseek.rs` | Supporto reasoning da rivedere (verifica integrazione, parametri, formato risposta) | alta |
| `src-tauri/src/llm/providers/gemini.rs` | Supporto reasoning da rivedere (verifica integrazione, parametri, formato risposta) | alta |
| OpenAI gpt-5.x β€” prompt caching | Bug lato OpenAI: prefix caching non funziona in modo affidabile su tutta la famiglia gpt-5 (gpt-5, gpt-5-mini, gpt-5-nano, gpt-5.4). Su gpt-4o funziona al 100%. Thread community aperto da ott 2025, non risolto a gen 2026. Da monitorare; non fixabile lato Glossa. Ref: [community.openai.com/t/1359574](https://community.openai.com/t/caching-is-borked-for-gpt-5-models/1359574) | monitoraggio |

---

*Ultimo aggiornamento: 2026-06-04 β€” branch feat/phrase-memory*
*Ultimo aggiornamento: 2026-06-06 β€” branch feat/phrase-memory*
3 changes: 1 addition & 2 deletions src-tauri/src/lib.rs
Original file line number Diff line number Diff line change
Expand Up @@ -73,6 +73,7 @@ pub fn run() {
llm::pipeline::test_provider_connection,
llm::pipeline::preflight_pipeline,
llm::pipeline::run_coherence_for_chunk,
llm::pipeline::extract_phrase_memory_pairs,
keystore::save_api_key,
keystore::get_api_key_status,
keystore::delete_api_key,
Expand All @@ -85,8 +86,6 @@ pub fn run() {
documents::extract_pdf_text,
vector::vec_ping,
vector::embedding::get_embeddings,
vector::embedding::split_phrases_llm,
vector::embedding::vec_upsert_source_phrase,
vector::embedding::vec_list_phrase_memory,
vector::embedding::vec_delete_phrase_memory,
vector::embedding::vec_update_phrase_memory,
Expand Down
84 changes: 84 additions & 0 deletions src-tauri/src/llm/pipeline.rs
Original file line number Diff line number Diff line change
Expand Up @@ -48,6 +48,20 @@ pub struct StageResult {
pub cache_miss_input_tokens: Option<u32>,
}

#[derive(serde::Serialize, serde::Deserialize)]
#[serde(rename_all = "camelCase")]
pub struct MemoryExtractorPair {
pub source_phrase: String,
pub target_phrase: String,
pub confidence: f64,
}

#[derive(serde::Serialize)]
#[serde(rename_all = "camelCase")]
pub struct MemoryExtractorResponse {
pub pairs: Vec<MemoryExtractorPair>,
}

#[tauri::command]
#[allow(clippy::too_many_arguments)]
pub async fn run_stage(
Expand Down Expand Up @@ -354,6 +368,76 @@ pub async fn refine_prompt(
prov.call(&client, &req).await.map(|r| r.content)
}

#[tauri::command]
#[allow(clippy::too_many_arguments)]
pub async fn extract_phrase_memory_pairs(
app: AppHandle,
provider: String,
model: String,
prompt: String,
source_text: String,
target_text: String,
source_language: String,
target_language: String,
ollama_base_url: Option<String>,
) -> Result<MemoryExtractorResponse, String> {
let prov = get_provider(&provider, ollama_base_url)?;
prov.preflight(&model).await?;
let api_key = get_api_key(&app, &provider)?;
let client = prov.http_client()?;
let structured = crate::llm::types::StructuredPrompt {
system: vec![crate::llm::types::PromptBlock {
text: prompt,
cacheable: false,
}],
user: format!(
"Source language: {source_language}\nTarget language: {target_language}\n\nOriginal source chunk:\n<<<SOURCE\n{source_text}\nSOURCE>>>\n\nFinal/current translation:\n<<<TARGET\n{target_text}\nTARGET>>>\n\nReturn JSON only with key \"pairs\"."
),
};
let req = LlmRequest {
model: &model,
structured: &structured,
api_key: &api_key,
json_mode: true,
provider_options: None,
};

let result_text = prov.call(&client, &req).await?.content;
let sanitized = sanitize_llm_json_output(&result_text);
let parsed: serde_json::Value = serde_json::from_str(sanitized)
.map_err(|e| format!("Failed to parse phrase memory JSON: {e}"))?;
let pairs = parsed["pairs"]
.as_array()
.ok_or_else(|| "Phrase memory extractor returned JSON without a pairs array".to_string())?;

let validated = pairs
.iter()
.filter_map(|entry| {
let source_phrase = entry["sourcePhrase"].as_str()?.trim();
let target_phrase = entry["targetPhrase"].as_str()?.trim();
if source_phrase.is_empty() || target_phrase.is_empty() {
return None;
}
if !source_text.contains(source_phrase) || !target_text.contains(target_phrase) {
log::warn!(
"phrase_memory.extract_pairs.discard_non_verbatim source_chars={} target_chars={}",
source_phrase.len(),
target_phrase.len()
);
return None;
}
let confidence = entry["confidence"].as_f64()?.clamp(0.0, 1.0);
Some(MemoryExtractorPair {
source_phrase: source_phrase.to_string(),
target_phrase: target_phrase.to_string(),
confidence,
})
})
.collect();

Ok(MemoryExtractorResponse { pairs: validated })
}

#[tauri::command]
pub async fn run_coherence_for_chunk(
app: AppHandle,
Expand Down
Loading
Loading