What problem does this solve?
sample_input is collected during deploy but only used for validation. The LLM generates load() and predict() without knowing the concrete input type — it relies entirely on inferred hints. The fix loop receives only a traceback and the broken code, with no information about what input caused the failure.
This means the LLM is generating and fixing blind. Including sample_input is the single cheapest improvement to codegen quality.
Proposed solution
Include sample_input in all three LLM calls:
generate() prompt — append after the metadata block:
Sample input: [1.2, 0.4, 3.1]
predict() must handle this exact input type.
fix() prompt — append metadata summary and sample input so the LLM knows what input caused the failure, not just the traceback:
Artifact metadata: {"framework": "sklearn", "class_name": "RandomForestClassifier", ...}
Sample input used during validation: [1.2, 0.4, 3.1]
LLM interpretation prompt (A3) — already included by design; listed here for completeness.
Alternatives considered
Relying on input_hint from the inspector alone. input_hint is a string description; sample_input is a concrete value. "great movie" tells the LLM more about input handling than "raw text string".
Area
CLI (deploy / fix)
What problem does this solve?
sample_inputis collected during deploy but only used for validation. The LLM generatesload()andpredict()without knowing the concrete input type — it relies entirely on inferred hints. The fix loop receives only a traceback and the broken code, with no information about what input caused the failure.This means the LLM is generating and fixing blind. Including
sample_inputis the single cheapest improvement to codegen quality.Proposed solution
Include
sample_inputin all three LLM calls:generate()prompt — append after the metadata block:fix()prompt — append metadata summary and sample input so the LLM knows what input caused the failure, not just the traceback:LLM interpretation prompt (A3) — already included by design; listed here for completeness.
Alternatives considered
Relying on
input_hintfrom the inspector alone.input_hintis a string description;sample_inputis a concrete value."great movie"tells the LLM more about input handling than"raw text string".Area
CLI (deploy / fix)