fix(openai): include Responses API instructions in captured input#754
Open
mmahmed wants to merge 3 commits into
Open
fix(openai): include Responses API instructions in captured input#754mmahmed wants to merge 3 commits into
mmahmed wants to merge 3 commits into
Conversation
When using the OpenAI Responses API, the `instructions` parameter (system prompt) was not captured in the generation input. This merges `instructions` into the input, mirroring how chat.completions captures system messages in the prompt. Fixes langfuse/langfuse#9775 Equivalent of langfuse/langfuse-python#1565 for the JS/TS SDK.
|
@mmahmed is attempting to deploy a commit to the langfuse Team on Vercel. A member of the Team first needs to authorize it. |
Co-authored-by: greptile-apps[bot] <165735046+greptile-apps[bot]@users.noreply.github.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
When using the OpenAI Responses API with
observeOpenAI(), theinstructionsparameter (system prompt) is not captured in the generation input. This PR fixesparseInputArgsto mergeinstructionsinto the input, mirroring howchat.completionscaptures system messages in the prompt.This is the JS/TS equivalent of the Python SDK fix in langfuse-python#1565.
Changes
packages/openai/src/parseOpenAI.ts— 19 lines added toparseInputArgs:instructionsis present alongsideinput(string), merge into[{role: system}, {role: user}]message listinstructionsis present alongsideinput(array), prepend{role: system}entryinstructionsis present (no input), return{ instructions }packages/openai/src/parseOpenAI.test.ts— new unit test file:Behavior
instructionsinput"You are a pirate.""Hello!""Hello!"[{role: "system", content: "You are a pirate."}, {role: "user", content: "Hello!"}]"You are a pirate."[{role: "user", content: "Hello!"}][{role: "user", ...}][{role: "system", content: "You are a pirate."}, {role: "user", ...}]"You are a pirate."undefined{instructions: "You are a pirate."}"Hello!""Hello!""Hello!"(unchanged)Test plan
vitest run packages/openai/src/parseOpenAI.test.ts)Fixes langfuse/langfuse#9775
Disclaimer: Experimental PR review
Greptile Summary
This PR fixes a gap in the OpenAI integration where the
instructionsparameter (system prompt) from the Responses API was silently dropped and never captured in Langfuse's generation input. The fix extendsparseInputArgsinparseOpenAI.tsto detect and mergeinstructionsinto theinputfield — as a[{role: "system"}, {role: "user"}]message list for string/array inputs, or as a plain{ instructions }object when noinputis provided — mirroring the existing behaviour for Chat Completions system messages.Key points:
input, arrayinput, absentinput, and unknown-typeinput.messages/promptchecks; if bothmessagesandinstructionsare present (unusual but possible), the instructions-merged result is silently overwritten — adding a!("messages" in args)guard would make the precedence explicit.messages+instructionscombination is absent, leaving the interaction behaviour undocumented.Confidence Score: 4/5
messagesguard and missing edge-case test) are low-risk given that Responses API and Chat Completions API are always called on separate endpoints, making the overlap scenario unrealistic in production.Important Files Changed
parseInputArgs; logic is correct for all realistic API paths but lacks a guard against themessagesblock silently overwriting the merged input when bothmessagesandinstructionsare present.instructions+messagescombination, which would lock in the expected precedence behavior.Flowchart
%%{init: {'theme': 'neutral'}}%% flowchart TD A[parseInputArgs called] --> B{instructions present\nand is string?} B -- No --> G B -- Yes --> C{typeof input} C -- string --> D["Merge: system msg + user msg array"] C -- array --> E["Prepend system msg to array"] C -- falsy --> F["input = instructions object"] C -- other object --> H["input = instructions + input object"] D --> G{messages in args?} E --> G F --> G H --> G G -- Yes --> I["input = messages block\nWARN: overwrites instructions merge"] G -- No --> J{input still falsy?} J -- Yes --> K["input = args.prompt"] J -- No --> L[Return model + input + modelParameters] I --> L K --> LLast reviewed commit: "fix(openai): include..."
(2/5) Greptile learns from your feedback when you react with thumbs up/down!