FEATURE-REQUEST: What about Key rotation?

I asked Deekwiki to give me full Guidance to Add AI Provider API Keys Rotation, and here is their respond.

---

<html>
<body>
<p>complete, final guide — every file, every line, from zero to end.</p>
<h2 data-heading="Step 1 — Create &#x60;src/cache/CacheService.ts&#x60; (new file)">Step 1 — Create <code>src/cache/CacheService.ts</code> (new file)</h2>
<p>Create this file from scratch:</p>
<pre><code class="language-typescript">// src/cache/CacheService.ts

class CacheService {
  private readonly store = new Map&#x3C;string, unknown>();

  get&#x3C;T>(key: string): T | undefined {
    return this.store.get(key) as T | undefined;
  }

  set&#x3C;T>(key: string, value: T): void {
    this.store.set(key, value);
  }
}

/** Process-lifetime singleton. Resets on restart. */
export const cache = new CacheService();
</code></pre>
<hr>
<h2 data-heading="Step 2 — Create &#x60;src/llm/keyRotator.ts&#x60; (new file)">Step 2 — Create <code>src/llm/keyRotator.ts</code> (new file)</h2>
<p>Create this file from scratch:</p>
<pre><code class="language-typescript">// src/llm/keyRotator.ts

import { cache } from '../cache/CacheService.js';

export interface ApiKeyEntry {
  id: string;        // UUID
  key: string;       // the actual API key string
  label?: string;    // human-readable name (optional)
  isEnabled: boolean;
}

/**
 * Advances the round-robin cursor and returns the next enabled key.
 * The cursor is stored in the in-memory cache, namespaced by providerId.
 * If the previously cached key ID no longer exists in the enabled list
 * (deleted or disabled between calls), findIndex returns -1 and
 * (-1 + 1) % length = 0, so rotation restarts from index 0 automatically.
 */
export function nextApiKey(providerId: string, entries: ApiKeyEntry[]): string {
  const enabled = entries.filter((e) => e.isEnabled);
  if (enabled.length === 0) {
    throw new Error(`No enabled API keys configured for provider: ${providerId}`);
  }
  const cacheKey = `apikey:rotation:${providerId}`;
  const lastUsedId = cache.get&#x3C;string>(cacheKey);
  let nextIndex = 0;
  if (lastUsedId !== undefined) {
    const currentIndex = enabled.findIndex((e) => e.id === lastUsedId);
    nextIndex = (currentIndex + 1) % enabled.length;
  }
  const chosen = enabled[nextIndex]!;
  cache.set(cacheKey, chosen.id);
  return chosen.key;
}

/** Returns true if at least one key is enabled. Used for startup validation
 *  without advancing the rotation cursor. */
export function hasEnabledKey(entries: ApiKeyEntry[]): boolean {
  return entries.some((e) => e.isEnabled);
}
</code></pre>
<hr>
<h2 data-heading="Step 3 — Modify &#x60;src/config/config.ts&#x60;">Step 3 — Modify <code>src/config/config.ts</code></h2>
<p>Two additions. The rest of the file is untouched.</p>
<p><strong>Addition 1</strong>: Insert after line 46 (after <code>export type PluginConfig = z.infer&#x3C;typeof PluginConfig>;</code>):</p>
<pre><code class="language-typescript">const ApiKeyEntry = z.object({
  id: z.string().uuid(),
  key: z.string().min(1),
  label: z.string().optional(),
  isEnabled: z.boolean().default(true),
});
export type ApiKeyEntry = z.infer&#x3C;typeof ApiKeyEntry>;
</code></pre>
<p><strong>Addition 2</strong>: Inside <code>ConfigSchema</code>, insert after line 60 (after <code>api_key: z.string().default(''),</code>):</p>
<pre><code class="language-typescript">  api_keys: z.array(ApiKeyEntry).default([]),
</code></pre>
<p>The relevant section of <code>ConfigSchema</code> will look like this after the change:</p>
<pre><code class="language-typescript">const ConfigSchema = z.object({
  backend: Backend.default(''),
  model: z.string().default(''),
  base_url: z.string().default(''),
  api_key: z.string().default(''),
  api_keys: z.array(ApiKeyEntry).default([]),   // ← new line
  skills_dirs: z.array(z.string()).default([]),
  // ... rest unchanged
</code></pre>
<hr>
<h2 data-heading="Step 4 — Modify &#x60;src/llm/openai.ts&#x60;">Step 4 — Modify <code>src/llm/openai.ts</code></h2>
<p>Five targeted changes. Everything else in the file is untouched.</p>
<p><strong>Change 1</strong> — Replace line 98 (field declaration):</p>
<pre><code class="language-typescript">// Before:
  readonly apiKey: string;

// After:
  private readonly resolveKey: () => string;
  private cachedKey: string = '';
</code></pre>
<p><strong>Change 2</strong> — Replace line 107 (constructor parameter):</p>
<pre><code class="language-typescript">// Before:
    apiKey: string,

// After:
    apiKey: string | (() => string),
</code></pre>
<p><strong>Change 3</strong> — Replace line 114 (constructor assignment):</p>
<pre><code class="language-typescript">// Before:
    this.apiKey = apiKey;

// After:
    this.resolveKey = typeof apiKey === 'string' ? () => apiKey : apiKey;
</code></pre>
<p><strong>Change 4</strong> — Replace lines 137–138 inside <code>ping()</code>:</p>
<pre><code class="language-typescript">// Before:
    if (this.apiKey) headers.Authorization = `Bearer ${this.apiKey}`;

// After:
    // Reuse the last key resolved for a real request; don't advance the cursor.
    const key = this.cachedKey || this.resolveKey();
    if (key) headers.Authorization = `Bearer ${key}`;
</code></pre>
<p><strong>Change 5</strong> — Replace lines 329–330 inside <code>headers()</code>:</p>
<pre><code class="language-typescript">// Before:
    if (this.apiKey) h.Authorization = `Bearer ${this.apiKey}`;

// After:
    this.cachedKey = this.resolveKey();   // advance cursor, cache for ping reuse
    if (this.cachedKey) h.Authorization = `Bearer ${this.cachedKey}`;
</code></pre>
<p>After all changes, the class header and the two affected methods look like:</p>
<pre><code class="language-typescript">export class OpenAIClient implements Client, StreamingClient, Pinger {
  readonly baseURL: string;
  private readonly resolveKey: () => string;
  private cachedKey: string = '';
  readonly modelID: string;
  readonly label: string;
  private readonly extraHeaders: Record&#x3C;string, string>;
  private readonly temperature?: number;
  private readonly maxTokens?: number;

  constructor(
    baseURL: string,
    apiKey: string | (() => string),
    model: string,
    label = 'openai-compat',
    extraHeaders: Record&#x3C;string, string> = {},
    genOpts: { temperature?: number; maxTokens?: number } = {},
  ) {
    this.baseURL = baseURL;
    this.resolveKey = typeof apiKey === 'string' ? () => apiKey : apiKey;
    this.modelID = model;
    // ... rest unchanged
  }

  async ping(signal?: AbortSignal): Promise&#x3C;void> {
    const headers: Record&#x3C;string, string> = {};
    const key = this.cachedKey || this.resolveKey();
    if (key) headers.Authorization = `Bearer ${key}`;
    const resp = await fetch(`${this.baseURL}/models`, { method: 'GET', headers, signal });
    if (resp.status >= 500) {
      throw new Error(`${this.label} status ${resp.status}`);
    }
  }

  // ... chat(), chatOnce(), chatStream(), openStream() all unchanged ...

  private headers(): Record&#x3C;string, string> {
    const h: Record&#x3C;string, string> = { ...this.extraHeaders, 'Content-Type': 'application/json' };
    this.cachedKey = this.resolveKey();
    if (this.cachedKey) h.Authorization = `Bearer ${this.cachedKey}`;
    return h;
  }
</code></pre>
<p>Note: <code>lmStudio()</code> passes <code>''</code> — this becomes <code>() => ''</code>, and the <code>if (key)</code> guard handles it correctly with no behavior change. </p>
<hr>
<h2 data-heading="Step 5 — Modify &#x60;src/llm/gemini.ts&#x60;">Step 5 — Modify <code>src/llm/gemini.ts</code></h2>
<p>Four targeted changes. Everything else in the file is untouched.</p>
<p><strong>Change 1</strong> — Replace lines 38–39 (field declarations):</p>
<pre><code class="language-typescript">// Before:
  readonly apiKey: string;
  readonly modelID: string;

// After:
  private readonly resolveKey: () => string;
  private cachedKey: string = '';
  readonly modelID: string;
</code></pre>
<p><strong>Change 2</strong> — Replace line 41 (constructor signature):</p>
<pre><code class="language-typescript">// Before:
  constructor(baseURL: string, apiKey: string, model: string) {

// After:
  constructor(baseURL: string, apiKey: string | (() => string), model: string) {
</code></pre>
<p><strong>Change 3</strong> — Replace line 43 (constructor assignment):</p>
<pre><code class="language-typescript">// Before:
    this.apiKey = apiKey;

// After:
    this.resolveKey = typeof apiKey === 'string' ? () => apiKey : apiKey;
</code></pre>
<p><strong>Change 4</strong> — Replace <code>ping()</code> and the fetch line inside <code>chat()</code>:</p>
<pre><code class="language-typescript">  async ping(signal?: AbortSignal): Promise&#x3C;void> {
    // Reuse the last key resolved for a real request; don't advance the cursor.
    const key = this.cachedKey || this.resolveKey();
    const resp = await fetch(`${this.baseURL}/models?key=${encodeURIComponent(key)}`, {
      method: 'GET',
      signal,
    });
    if (resp.status >= 500) {
      throw new Error(`gemini status ${resp.status}`);
    }
  }

  async chat(req: ChatRequest, signal?: AbortSignal): Promise&#x3C;ChatResponse> {
    const body = encodeRequest(req);
    const combinedSignal = withTimeout(signal, CHAT_TIMEOUT_MS);
    this.cachedKey = this.resolveKey();   // advance cursor, cache for ping reuse
    let resp: Response;
    try {
      resp = await fetch(
        `${this.baseURL}/${withModelsPrefix(req.model || this.modelID)}:generateContent?key=${encodeURIComponent(this.cachedKey)}`,
        {
          method: 'POST',
          headers: { 'Content-Type': 'application/json' },
          body: JSON.stringify(body),
          signal: combinedSignal,
        },
      );
    // ... rest of chat() unchanged
</code></pre>
<hr>
<h2 data-heading="Step 6 — Replace &#x60;src/llm/factory.ts&#x60; entirely">Step 6 — Replace <code>src/llm/factory.ts</code> entirely</h2>
<pre><code class="language-typescript">// src/llm/factory.ts
// Build the right Client from the parsed Config.

import type { Config } from '../config/config.js';
import type { Client } from './client.js';
import { GeminiClient } from './gemini.js';
import { hasEnabledKey, nextApiKey } from './keyRotator.js';
import { OllamaClient } from './ollama.js';
import { OpenAIClient } from './openai.js';
import {
  DEEPSEEK_DEFAULT_BASE_URL,
  DEEPSEEK_DEFAULT_MODEL,
  GEMINI_DEFAULT_BASE_URL,
  GEMINI_DEFAULT_MODEL,
  GROQ_DEFAULT_BASE_URL,
  GROQ_DEFAULT_MODEL,
  KIMI_DEFAULT_BASE_URL,
  KIMI_DEFAULT_MAX_TOKENS,
  KIMI_DEFAULT_MODEL,
  OPENROUTER_DEFAULT_BASE_URL,
  OPENROUTER_DEFAULT_MODEL,
} from './providers.js';

/**
 * Creates a per-request key resolver for the given backend.
 * If api_keys has enabled entries, each call to the resolver advances the
 * round-robin cursor (stored in the in-memory CacheService).
 * Falls back to the static api_key string for single-key setups.
 */
function createKeyResolver(cfg: Config, backend: string): () => string {
  if (cfg.api_keys.length > 0 &#x26;&#x26; hasEnabledKey(cfg.api_keys)) {
    return () => nextApiKey(backend, cfg.api_keys);
  }
  return () => cfg.api_key;
}

/**
 * Returns true if the config has at least one usable key.
 * Used for startup validation without advancing the rotation cursor.
 */
function hasKey(cfg: Config): boolean {
  if (cfg.api_keys.length > 0) return hasEnabledKey(cfg.api_keys);
  return Boolean(cfg.api_key);
}

export function newFromConfig(cfg: Config): Client {
  // Generation knobs shared by the OpenAI-compatible backends. temperature is
  // sent only to models that accept it (see OpenAIClient.encodeRequest).
  const gen = { temperature: cfg.temperature, maxTokens: cfg.max_tokens };
  switch (cfg.backend) {
    case 'ollama':
    case '':
      return new OllamaClient(cfg.base_url, cfg.model);
    case 'lmstudio':
      return OpenAIClient.lmStudio(cfg.base_url, cfg.model);
    case 'openai-compat':
      if (!cfg.base_url) {
        throw new Error('openai-compat backend requires base_url');
      }
      return new OpenAIClient(
        cfg.base_url,
        createKeyResolver(cfg, 'openai-compat'),
        cfg.model,
        'openai-compat',
        {},
        gen,
      );
    case 'kimi':
      if (!hasKey(cfg)) {
        throw new Error('kimi backend requires api_key or MOONSHOT_API_KEY');
      }
      return new OpenAIClient(
        cfg.base_url || KIMI_DEFAULT_BASE_URL,
        createKeyResolver(cfg, 'kimi'),
        cfg.model || KIMI_DEFAULT_MODEL,
        'kimi',
        {},
        // Kimi can't be tuned down via temperature (locked to 1), so default a
        // response cap to keep it from narrating unbounded; user override wins.
        { temperature: cfg.temperature, maxTokens: cfg.max_tokens ?? KIMI_DEFAULT_MAX_TOKENS },
      );
    case 'groq':
      if (!hasKey(cfg)) {
        throw new Error('groq backend requires api_key or GROQ_API_KEY');
      }
      return new OpenAIClient(
        cfg.base_url || GROQ_DEFAULT_BASE_URL,
        createKeyResolver(cfg, 'groq'),
        cfg.model || GROQ_DEFAULT_MODEL,
        'groq',
        {},
        gen,
      );
    case 'openrouter':
      if (!hasKey(cfg)) {
        throw new Error('openrouter backend requires api_key or OPENROUTER_API_KEY');
      }
      return new OpenAIClient(
        cfg.base_url || OPENROUTER_DEFAULT_BASE_URL,
        createKeyResolver(cfg, 'openrouter'),
        cfg.model || OPENROUTER_DEFAULT_MODEL,
        'openrouter',
        {
          'HTTP-Referer': 'https://github.com/pentesterflow/agent',
          'X-OpenRouter-Title': 'PentesterFlow',
        },
        gen,
      );
    case 'deepseek':
      if (!hasKey(cfg)) {
        throw new Error('deepseek backend requires api_key or DEEPSEEK_API_KEY');
      }
      return new OpenAIClient(
        cfg.base_url || DEEPSEEK_DEFAULT_BASE_URL,
        createKeyResolver(cfg, 'deepseek'),
        cfg.model || DEEPSEEK_DEFAULT_MODEL,
        'deepseek',
        {},
        gen,
      );
    case 'gemini':
      if (!hasKey(cfg)) {
        throw new Error('gemini backend requires api_key or GEMINI_API_KEY');
      }
      return new GeminiClient(
        cfg.base_url || GEMINI_DEFAULT_BASE_URL,
        createKeyResolver(cfg, 'gemini'),
        cfg.model || GEMINI_DEFAULT_MODEL,
      );
    default: {
      const _exhaustive: never = cfg.backend;
      throw new Error(`unknown backend: ${String(_exhaustive)}`);
    }
  }
}
</code></pre>
<hr>
<h2 data-heading="Step 7 — Modify &#x60;src/cli/index.ts&#x60;">Step 7 — Modify <code>src/cli/index.ts</code></h2>
<p>Four targeted changes. Everything else in the file is untouched.</p>
<p><strong>Change 1</strong> — Replace line 245 (<code>--api-key</code> flag):</p>
<pre><code class="language-typescript">// Before:
  if (flags.apiKey) cfg.api_key = flags.apiKey;

// After:
  if (flags.apiKey) {
    cfg.api_key = flags.apiKey;
    cfg.api_keys = [];  // single-key CLI override disables rotation
  }
</code></pre>
<p><strong>Change 2</strong> — Replace lines 247–261 (env var fallbacks). Add <code>&#x26;&#x26; !cfg.api_keys.some((e) => e.isEnabled)</code> to each guard:</p>
<pre><code class="language-typescript">// Before:
  if (cfg.backend === 'kimi' &#x26;&#x26; !cfg.api_key) {
    cfg.api_key = process.env.MOONSHOT_API_KEY || process.env.KIMI_API_KEY || '';
  }
  if (cfg.backend === 'groq' &#x26;&#x26; !cfg.api_key) {
    cfg.api_key = process.env.GROQ_API_KEY || '';
  }
  if (cfg.backend === 'openrouter' &#x26;&#x26; !cfg.api_key) {
    cfg.api_key = process.env.OPENROUTER_API_KEY || '';
  }
  if (cfg.backend === 'deepseek' &#x26;&#x26; !cfg.api_key) {
    cfg.api_key = process.env.DEEPSEEK_API_KEY || '';
  }
  if (cfg.backend === 'gemini' &#x26;&#x26; !cfg.api_key) {
    cfg.api_key = process.env.GEMINI_API_KEY || '';
  }

// After:
  if (cfg.backend === 'kimi' &#x26;&#x26; !cfg.api_key &#x26;&#x26; !cfg.api_keys.some((e) => e.isEnabled)) {
    cfg.api_key = process.env.MOONSHOT_API_KEY || process.env.KIMI_API_KEY || '';
  }
  if (cfg.backend === 'groq' &#x26;&#x26; !cfg.api_key &#x26;&#x26; !cfg.api_keys.some((e) => e.isEnabled)) {
    cfg.api_key = process.env.GROQ_API_KEY || '';
  }
  if (cfg.backend === 'openrouter' &#x26;&#x26; !cfg.api_key &#x26;&#x26; !cfg.api_keys.some((e) => e.isEnabled)) {
    cfg.api_key = process.env.OPENROUTER_API_KEY || '';
  }
  if (cfg.backend === 'deepseek' &#x26;&#x26; !cfg.api_key &#x26;&#x26; !cfg.api_keys.some((e) => e.isEnabled)) {
    cfg.api_key = process.env.DEEPSEEK_API_KEY || '';
  }
  if (cfg.backend === 'gemini' &#x26;&#x26; !cfg.api_key &#x26;&#x26; !cfg.api_keys.some((e) => e.isEnabled)) {
    cfg.api_key = process.env.GEMINI_API_KEY || '';
  }
</code></pre>
<p><strong>Change 3</strong> — Replace lines 695–700 (<code>readConfig</code>):</p>
<pre><code class="language-typescript">// Before:
        readConfig: () => ({
          backend: cfg.backend,
          baseURL: cfg.base_url,
          apiKey: cfg.api_key,
          model: cfg.model,
        }),

// After:
        readConfig: () => ({
          backend: cfg.backend,
          baseURL: cfg.base_url,
          // For the UI model picker, supply the first enabled key from the
          // rotation pool (or fall back to the single api_key). listModels()
          // only needs one valid key — it doesn't participate in rotation.
          apiKey: cfg.api_keys.find((e) => e.isEnabled)?.key ?? cfg.api_key,
          model: cfg.model,
        }),
</code></pre>
<p><strong>Change 4</strong> — Replace line 709 inside <code>applyProvider</code>:</p>
<pre><code class="language-typescript">// Before:
          if (change.apiKey !== undefined) cfg.api_key = change.apiKey;

// After:
          if (change.apiKey !== undefined) {
            cfg.api_key = change.apiKey;
            // Only wipe the rotation pool if the incoming key is not already
            // in it (e.g. user just confirmed the same backend without changing
            // the key — readConfig returns the first enabled key, which IS in
            // the pool, so we must not clear it).
            const alreadyInPool = cfg.api_keys.some(
              (e) => e.key === change.apiKey &#x26;&#x26; e.isEnabled,
            );
            if (!alreadyInPool) cfg.api_keys = [];
          }
</code></pre>
<hr>
<h2 data-heading="How to configure &#x60;~/.pentesterflow/config.json&#x60;">How to configure <code>~/.pentesterflow/config.json</code></h2>
<p>After applying all changes, add <code>api_keys</code> to your config. Leave <code>api_key</code> as <code>""</code> when using the rotation pool:</p>
<pre><code class="language-json">{
  "backend": "groq",
  "model": "llama-3.3-70b-versatile",
  "api_key": "",
  "api_keys": [
    { "id": "11111111-1111-1111-1111-111111111111", "key": "gsk_AAA...", "label": "key-1", "isEnabled": true },
    { "id": "22222222-2222-2222-2222-222222222222", "key": "gsk_BBB...", "label": "key-2", "isEnabled": true },
    { "id": "33333333-3333-3333-3333-333333333333", "key": "gsk_CCC...", "label": "key-3", "isEnabled": false }
  ]
}
</code></pre>
<p>Request 1 → <code>gsk_AAA...</code>, Request 2 → <code>gsk_BBB...</code>, Request 3 → <code>gsk_AAA...</code> (key-3 is skipped, wraps around). Rotation resets on process restart.</p>
<hr>
<h2 data-heading="Summary of all files">Summary of all files</h2>

File | Action
-- | --
src/cache/CacheService.ts | Create — in-memory Map singleton
src/llm/keyRotator.ts | Create — round-robin logic + hasEnabledKey
src/config/config.ts | Modify — add ApiKeyEntry schema + api_keys field
src/llm/openai.ts | Modify — resolveKey + cachedKey; fix ping() and headers()
src/llm/gemini.ts | Modify — same pattern; fix ping() and chat()
src/llm/factory.ts | Replace — createKeyResolver + hasKey; no more bare cfg.api_key
src/cli/index.ts | Modify — 4 spots: --api-key flag, env vars, readConfig, applyProvider


</body>
</html>

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

FEATURE-REQUEST: What about Key rotation? #15

Step 1 — Create `src/cache/CacheService.ts` (new file)

Step 2 — Create `src/llm/keyRotator.ts` (new file)

Step 3 — Modify `src/config/config.ts`

Step 4 — Modify `src/llm/openai.ts`

Step 5 — Modify `src/llm/gemini.ts`

Step 6 — Replace `src/llm/factory.ts` entirely

Step 7 — Modify `src/cli/index.ts`

How to configure `~/.pentesterflow/config.json`

Summary of all files

Metadata

Assignees

Labels

Projects

Milestone

Relationships

Development

File	Action
src/cache/CacheService.ts	Create — in-memory Map singleton
src/llm/keyRotator.ts	Create — round-robin logic + hasEnabledKey
src/config/config.ts	Modify — add ApiKeyEntry schema + api_keys field
src/llm/openai.ts	Modify — resolveKey + cachedKey; fix ping() and headers()
src/llm/gemini.ts	Modify — same pattern; fix ping() and chat()
src/llm/factory.ts	Replace — createKeyResolver + hasKey; no more bare cfg.api_key
src/cli/index.ts	Modify — 4 spots: --api-key flag, env vars, readConfig, applyProvider

FEATURE-REQUEST: What about Key rotation? #15

Description

Step 1 — Create src/cache/CacheService.ts (new file)

Step 2 — Create src/llm/keyRotator.ts (new file)

Step 3 — Modify src/config/config.ts

Step 4 — Modify src/llm/openai.ts

Step 5 — Modify src/llm/gemini.ts

Step 6 — Replace src/llm/factory.ts entirely

Step 7 — Modify src/cli/index.ts

How to configure ~/.pentesterflow/config.json

Summary of all files

Metadata

Metadata

Assignees

Labels

Projects

Milestone

Relationships

Development

Issue actions

Step 1 — Create `src/cache/CacheService.ts` (new file)

Step 2 — Create `src/llm/keyRotator.ts` (new file)

Step 3 — Modify `src/config/config.ts`

Step 4 — Modify `src/llm/openai.ts`

Step 5 — Modify `src/llm/gemini.ts`

Step 6 — Replace `src/llm/factory.ts` entirely

Step 7 — Modify `src/cli/index.ts`

How to configure `~/.pentesterflow/config.json`