Quote review interface connected to Langfuse

A system that generates customer-facing content needs a person in the company who can stop or correct it before sending. That reviewer usually did not build the system and does not work with traces or scores; the AI engineer maintaining it needs their judgments to understand where the agent fails.

The interface presents one quote at a time for approval, without requiring the reviewer to read technical traces.

customer request → executing agent → automated judge → Langfuse queue → reviewer → Langfuse scores

The agent prepares a quote, PDF and message from the price catalogue. An automated judge evaluates the quote and message separately; a person decides whether to approve or correct them.

For the reviewer

• Sees the request, quote, message and automated evaluation together.
• Evaluates the quote and message separately and can correct the message before sending.
• Can defer or stop a case; sending is simulated in the demo.

For the AI engineer

• Langfuse links the agent output, automated verdict and human judgment to the same trace, making disagreements easy to examine.
• The company catalogue and data remain separate from the review logic.

The reviewer evaluates the result and says whether they agree with the automated judge. This helps distinguish a problem in the agent output from a problem in its automated evaluation, or both.

The demo uses sample requests to show the review flow. Judgments entered in the online demo are not saved; those already in Langfuse demonstrate the connection but do not measure agreement among independent reviewers.