It's been suggested that rather than have a hard pass fail, a moderated message can be kicked back to the user for editing and refinement multiple times if necessary.
The user posts a new message, that is marked as pending. Local policy tools scan the post, first. If they find any violations, they explain to the user what those are. They instruct the user to make specific corrections. The user makes the corrections and saves the draft. The next pass is optional: an AI agent does a similar scan. But it is not allowed to take action. It reports its findings back to the policy tools that are not ai. The policy tools check AI's report against policy itself to avoid ai hallucination. If the ai's report is internally consistent with no hallucinations, the report data is factored in. If hallucination is detected, the tools ignore the feedback from ai. Whether ai is used or not, the policy tools then check the user's draft edit. If it passes, the post is either approved, or sent to a moderator, depending on configuration. If the post is still not right, it is sent to a human moderator for review. The moderator then types in feedback about what needs to be changed and why. If the user applies the changes correctly, the moderator will then approve the post. If the user refuses to make the necessary changes, the moderator will reject the post.
Its fine to start with the more absolute system you're building, because its just easier to start there. As things evolve however, please consider my suggestion for a multi-pass system that can optionally include an ai pass and local policy parsing tools.
It's been suggested that rather than have a hard pass fail, a moderated message can be kicked back to the user for editing and refinement multiple times if necessary.
This task could also be performed by AI
From Paradigms Shifting: