Skip to content

Pass dangerouslyAllowBrowser to clean client constructor in isWrapped#170

Open
jamie-retool wants to merge 1 commit intobraintrustdata:mainfrom
jamie-retool:jd/pass-dangerous-into-clean-constructor
Open

Pass dangerouslyAllowBrowser to clean client constructor in isWrapped#170
jamie-retool wants to merge 1 commit intobraintrustdata:mainfrom
jamie-retool:jd/pass-dangerous-into-clean-constructor

Conversation

@jamie-retool
Copy link

This might be an artifact of our own unique setup where we are using JSDom for handling browser-related things in our environment.

The Problem

The isWrapped function creates a "clean" client to compare against the passed client to detect if it's already been wrapped/proxied. It does this by calling:

const Constructor = Object.getPrototypeOf(client).constructor;
const clean = new Constructor({ apiKey: "dummy" });

However, in browser environments (or environments using JSDom), the OpenAI client requires dangerouslyAllowBrowser: true or it throws an error. If a client was created with dangerouslyAllowBrowser: true, the isWrapped check would fail because the clean client constructor throws before the comparison can happen.

The Fix

Extract dangerouslyAllowBrowser from the client options (or deprecated options) and pass it through to isWrapped, so the clean client is constructed with the same browser setting as the original.

I added a test that shows a failure before the fix, and it passes after the fix.

Also, I have a reproducible block of code that shows this failing in autoevals 0.0.127 (the latest code fails in the same way).

    "openai": "^4.104.0",
    "autoevals": "^0.0.127",

Code you can run to see it fail in current versions:

import { JSDOM } from "jsdom";

const dom = new JSDOM("", { url: "http://localhost" });
(globalThis as any).window = dom.window;
(globalThis as any).document = dom.window.document;

(async () => {
  const { Factuality } = await import("autoevals");
  const { default: OpenAI } = await import("openai");

  const client = new OpenAI({
    apiKey: process.env.BRAINTRUST_API_KEY,
    baseURL: "https://api.braintrust.dev/v1/proxy",
    dangerouslyAllowBrowser: true,
  });

  Factuality({
    input: "What is 2+2?",
    output: "4",
    expected: "4",
    client,
  })
    .then((score) => console.log("Score:", score))
    .catch((err) => {
      console.error("Error:", err.message.split("\n")[0]);
      console.error(err.stack?.split("\n").slice(0, 6).join("\n"));
    });
})();

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant