Hi @Neph0s,
Great work on InCharacter and the subsequent CoSER/CharacterBench line of research. I've been following this space closely and had a question that your work made me curious about.
InCharacter evaluates AI agents by simulating psychological interviews. My question: does the framework generalize in the reverse direction, where we use multi-agent LLM systems to assess real human personalities through interactive dialogue (essentially an LLM-powered successor to traditional instruments like Big Five / 16PF / OPQ)?
Specifically, in your experiments with 14 different personality scales, did you observe any that seemed particularly sensitive to the interviewer agent's prompt design vs. others that were more robust? I'm curious whether that lesson transfers to assessing humans rather than agents.
Asking because we're building a practical system in this direction at a Shanghai AI company, and your work (along with PsychoGAT) has been a major influence on our design. Would love your perspective whenever you have a moment.
Thanks! Allen
（如果可以直接中文讨论那更好^，我就base上海，我的微信ID是 osunwei
Hi @Neph0s,
Great work on InCharacter and the subsequent CoSER/CharacterBench line of research. I've been following this space closely and had a question that your work made me curious about.
InCharacter evaluates AI agents by simulating psychological interviews. My question: does the framework generalize in the reverse direction, where we use multi-agent LLM systems to assess real human personalities through interactive dialogue (essentially an LLM-powered successor to traditional instruments like Big Five / 16PF / OPQ)?
Specifically, in your experiments with 14 different personality scales, did you observe any that seemed particularly sensitive to the interviewer agent's prompt design vs. others that were more robust? I'm curious whether that lesson transfers to assessing humans rather than agents.
Asking because we're building a practical system in this direction at a Shanghai AI company, and your work (along with PsychoGAT) has been a major influence on our design. Would love your perspective whenever you have a moment.
Thanks! Allen
（如果可以直接中文讨论那更好^，我就base上海，我的微信ID是 osunwei