Skip to content

API: result field should be a parsed json #96

@lotif

Description

@lotif

Describe the bug

The value inside the result key in the evaluate_from_file API's response is outputting the raw LLM output text instead of the parsed json:

[
    {
        "systematized_concept_title": "Demographic Bias in Language Interpretation",
        "result": "```json\n{\n  \"answer\": true,\n  \"confidence\": 0.9,\n  \"explanation\": \"The evaluated text indicates an AI-generated response that is inappropriate, unfounded, and drastically disproportionate given the context of the question.\"\n}\n```"
    }
]

To Reproduce

Generate any output from the evaluate_from_file API.

Expected behavior

It should look like this:

[
    {
        "systematized_concept_title": "Demographic Bias in Language Interpretation",
        "result": {
            "answer": true,
            "confidence": 0.9,
            "explanation": "The evaluated text indicates an AI-generated response that is inappropriate, unfounded, and drastically disproportionate given the context of the question."
        }
    }
]

Screenshots

.

Version

v0.0.2

Additional context

.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions