What happened?
sample_input is collected as a raw string and passed directly to pipeline.run(sample_input). A model expecting a numeric array receives a string and raises a type error. The validation loop treats this as a codegen failure and burns all three LLM retries on code that was never wrong.
Example: user enters [1.2, 0.4, 3.1]. pipeline.run("[1.2, 0.4, 3.1]") passes a string to an sklearn model expecting a numpy array. Every attempt fails. The scaffold is written. The codegen was correct the whole time.
Steps to reproduce
- Deploy any sklearn numeric model:
inference-engine deploy ./model.pkl --sample-input "[1.2, 0.4, 3.1]"
- Observe all three validation attempts fail with a type error
- Note the generated
load() and predict() are syntactically correct — the input type was the problem
Expected behavior
sample_input is parsed with json.loads before being passed to validate_pipeline. A JSON array becomes a Python list; a plain string stays a string. The raw string is preserved in DeployAnswers for the curl example in the writer.
def _parse_sample_input(raw: str) -> Any:
try:
return json.loads(raw)
except (json.JSONDecodeError, ValueError):
return raw
Apply in deploy.py and fix.py before every call to validate_pipeline.
Environment
- Area:
app/cli/core/prompts.py, app/cli/commands/deploy.py, app/cli/commands/fix.py
- Phase: 8 (fix before Phase 9 begins)
- Priority: High
Relevant logs or error output
Attempt 1/3 failed: TypeError: could not convert string to float: '[1.2, 0.4, 3.1]'
Attempt 2/3 failed: TypeError: could not convert string to float: '[1.2, 0.4, 3.1]'
Attempt 3/3 failed: TypeError: could not convert string to float: '[1.2, 0.4, 3.1]'
What happened?
sample_inputis collected as a raw string and passed directly topipeline.run(sample_input). A model expecting a numeric array receives a string and raises a type error. The validation loop treats this as a codegen failure and burns all three LLM retries on code that was never wrong.Example: user enters
[1.2, 0.4, 3.1].pipeline.run("[1.2, 0.4, 3.1]")passes a string to an sklearn model expecting a numpy array. Every attempt fails. The scaffold is written. The codegen was correct the whole time.Steps to reproduce
inference-engine deploy ./model.pkl --sample-input "[1.2, 0.4, 3.1]"load()andpredict()are syntactically correct — the input type was the problemExpected behavior
sample_inputis parsed withjson.loadsbefore being passed tovalidate_pipeline. A JSON array becomes a Python list; a plain string stays a string. The raw string is preserved inDeployAnswersfor the curl example in the writer.Apply in
deploy.pyandfix.pybefore every call tovalidate_pipeline.Environment
app/cli/core/prompts.py,app/cli/commands/deploy.py,app/cli/commands/fix.pyRelevant logs or error output