Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
13 changes: 13 additions & 0 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,8 @@ node_modules
CLAUDE.md
.vercel
.env
key.pem
*.pem

.vscode/
.github/agents
Expand All @@ -11,3 +13,14 @@ recordings/

# Local design specs and implementation plans (not pushed)
docs/

# AI build artifacts and local dev files
ai/core/server/build/
ai/**/.venv/
ai/**/venv/
ai/llm-core/models/
ai/llm-core/sup-it-lora-v3/
ai/llm-core/unsloth_compiled_cache/
ai/microservices/**/llm/
*.wav
*.mp4
25 changes: 24 additions & 1 deletion ai/core/server/inc/network/MicroservicesManager.hpp
Original file line number Diff line number Diff line change
Expand Up @@ -27,7 +27,8 @@
#include <condition_variable>
#include <queue>
#include <memory>
#include <crow.h>
#include <functional>
#include <future>

#include "ExceptionManager.hpp"

Expand Down Expand Up @@ -74,6 +75,14 @@ namespace talkup_network {
*/
static void send_to_sts_microservice(const nlohmann::json &data, ResponseCallback callback);

/**
* @brief Push structured simulation context (company, job offer) to STS for one interview.
* @return true if STS acknowledged registration.
*/
static bool send_simulation_context_to_sts(
const std::string &interview_id,
const nlohmann::json &context_data);

/**
* @brief Initialize WebSocket connections to all registered microservices.
* It's establishes persistent WebSocket connections to each microservice
Expand Down Expand Up @@ -108,9 +117,18 @@ namespace talkup_network {
protected:
private:
struct WebSocketConnection {
enum class StsJobKind {
StreamChunk,
SimulationContext,
};

struct StsJob {
StsJobKind kind = StsJobKind::StreamChunk;
nlohmann::json data;
ResponseCallback callback;
std::string interview_id;
nlohmann::json context_data;
std::shared_ptr<std::promise<bool>> context_promise;
};

std::shared_ptr<boost::asio::io_context> io_context;
Expand Down Expand Up @@ -168,5 +186,10 @@ namespace talkup_network {
* @param data The JSON data containing the job information.
*/
static void process_sts_job(const nlohmann::json &data, ResponseCallback callback);

static void process_simulation_context_job(
const std::string &interview_id,
const nlohmann::json &context_data,
const std::shared_ptr<std::promise<bool>> &result_promise);
};
}
3 changes: 3 additions & 0 deletions ai/core/server/inc/network/WebsocketManager.hpp
Original file line number Diff line number Diff line change
Expand Up @@ -86,6 +86,9 @@ namespace talkup_network {
void handle_stream_chunk(const nlohmann::json& json, crow::websocket::connection& conn,
std::shared_ptr<MicroservicesManager> microservices_manager);

void handle_simulation_context(const nlohmann::json& json, crow::websocket::connection& conn,
std::shared_ptr<MicroservicesManager> microservices_manager);

private:
std::unordered_map<std::string, std::function<void(const nlohmann::json&,
crow::websocket::connection&, std::shared_ptr<MicroservicesManager>)>> _type_handlers;
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -84,6 +84,17 @@ Type format can be `audio`, `video`, `image` or `text` depending on the stream c
```

---

## 4.5 Simulation context — company and job offer

Before audio `stream_chunk` messages, send `simulation_context` so the STS LLM receives company/job context in its system prompt.

**Order:** WebSocket open → `simulation_context` → `simulation_context_ack` → `stream_chunk`.

See the French protocol document for the full JSON schema (`company`, `jobOffer`, `additionalInfo`, `language`, `interviewType`).

---

## 5. Advanced Level — AI Server ↔ Microservices
The protocol is designed to evolve into a modular architecture where the AI Server delegates tasks to Python microservices.

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -85,6 +85,53 @@ Le format de type peut être `audio`, `video`, `image` ou `text` selon le conten

---

## 4.5 Contexte de simulation — entreprise et offre d'emploi

Avant d'envoyer des `stream_chunk` audio, le Frontend **doit** enregistrer le contexte métier via `simulation_context`. Ce contexte alimente le prompt système du LLM (STS).

**Ordre recommandé :** connexion WS → `simulation_context` → `simulation_context_ack` → `stream_chunk`.

### Exemple : Frontend → Serveur IA

```json
{
"key": "exemple_key",
"type": "simulation_context",
"stream_id": "019bf2a0-....",
"format": "json",
"timestamp": 1739592334,
"data": {
"language": "French",
"interviewType": "technical",
"company": {
"name": "Acme Digital",
"description": "ESN specialisee en transformation digitale."
},
"jobOffer": {
"title": "Developpeur Backend Java",
"description": "Mission SI bancaire microservices.",
"requirements": "Java 17, Spring Boot, Kafka."
},
"additionalInfo": "Notes complementaires (CV resume, objectifs)."
}
}
```

### Exemple : Serveur IA → Frontend

```json
{
"key": "exemple_key",
"type": "simulation_context_ack",
"stream_id": "019bf2a0-....",
"format": "text",
"timestamp": 1739592335,
"data": "simulation context registered"
}
```

---

## 5. Niveau avancé — Serveur IA ↔ Microservices
Le protocole est conçu pour évoluer vers une architecture modulaire où le Serveur IA délègue certaines tâches à des microservices Python.

Expand Down
Loading
Loading